When I started playing with speakers as a teenager (in the early '70s, oh dear) the only test signal source I had was a sine wave generator. (I had square waves too, but they were of little value other than to demonstrate the utter inability of speakers of the day to reproduce them.) So it was very common to listen to sine sweeps, which revealed all sorts of odd, generally unharmonically-related, buzzes, rattles, and so forth. (Nothing smokes out an iffy HF driver like a sine sweep.) Now I have ARTA, LIMP, Smaart, my own analysis tools, and a nice measurement mic, and don't think about listening to sine sweeps very often.
The last few days I've been developing a new passive crossover for some 18 Sound 8CX400F 8 in. coax based utility speakers which I've had for a few years. I took many acoustic and electrical measurements, ran lots of simulations, consulted the Parts Express catalog for available element values, but never once listened to a sine sweep. Then, this morning, as a final sanity check before ordering parts, I hooked up the old analog sine wave generator and "swept it out". I was horrified to hear a note lower than the sine frequency (what some might call a "sub-harmonic", but let's not go there) in the crossover region. My first thought was oh crap, I've got a blown HF diaphragm. I swapped drivers and got the same sound, and furthermore, disconnecting and shorting the tweeter did not make it go away. It was coming from the woofers -- identically from both of them.
The bad news for me is, in the case of these little speakers, there's probably not much to be done. I don't want to move the crossover frequency any lower because of power and distortion concerns in the tweeter.
The takehome is, remember to do sine sweeps. It's quick, easy, and you learn things.
The question is, how often and to what extent is the difference in sound quality between speakers due to extraneous noises of this sort? We focus so much on what we routinely measure, but, other than distortion measurements, these measurements are blind to extraneous noises.