This thesis reports advances in terahertz time-domain spectroscopy, relating to the development of new techniques and components that enhance the experimentalist control over the terahertz polarisation state produced by photoconductive emitters. It describes how utilising the dynamic magnetoelectric response at THz frequencies, in the form of electromagnons, can probe material properties at a transition between two magnetically ordered phases. Additionally, preliminary investigations into the properties of materials exposed to extreme terahertz optical electric fields are reported. The work presented in this thesis may have immediate impacts on the study of anisotropic media at THz frequencies, with photoconductive emitters and detectors being the most commonly used components for commercially available terahertz spectroscopy and imaging systems, and by providing a new way to study the nature of magnetic phase transitions in multiferroics. In the longer term the increased understanding of multiferroics yielded by ultrafast spectroscopic methods, including terahertz time-domain spectroscopy, may help develop new magnetoelectric and multiferroic materials for applications such as spintronics.