A Crisis in Headphone Measurement?

February 2018

I’d thought that the measuring of headphones was improving, but some recent developments have made me not so sure. Last year, the introduction of some excellent new lab gear made it possible for experienced engineers and technicians to make more useful measurements than ever before. But the introduction of some very cheap, nonstandard measurement gear has made it possible for almost anyone to make headphone measurements that are less useful than ever before.


On the positive side are the recent introductions of the first truly new headphone-measuring rigs in three decades. For me, the most exciting development is G.R.A.S. Sound and Vibration’s KB5000, a new, far more realistic artificial pinna, or human outer ear, that has allowed me to make far more accurate assessments of, for example, the isolation properties of earphones and earplugs.

G.R.A.S. and two competitors, Brüel & Kjær and Head Acoustics, also launched new ear simulators -- the component that mimics the acoustics of the average human ear, and incorporates a measurement microphone -- designed to deliver more accurate measurements above 10kHz. Because existing headphone-measuring products were designed largely for testing hearing aids, their performance with higher frequencies wasn’t of much concern, and they were inaccurate above 10kHz. New “high resolution” ear simulators, such as the G.R.A.S. RA0401 and RA0402, are designed with additional damping to reduce high-frequency resonance and deliver more consistent results above 10kHz without significantly affecting results below 10kHz. We don’t yet know much about how the performance of headphones in the top octave of human hearing affects their perceived sound quality, but with this new equipment we can learn.


The negative developments in headphone-measuring gear were summed up by Head-Fi founder Jude Mansilla in his presentation at the recent Association of Loudspeaker and Manufacturing Acoustics (ALMA) International Symposium and Expo (AISE) 2018, which took place in Las Vegas in early January, two days before the 2018 Consumer Electronics Show. In Mansilla’s talk, “Audio Engineering as Consumer Content,” he lamented the fact that the availability of cheap measurement microphones and free measurement software such as Room EQ Wizard has inspired many headphone enthusiasts to jury-rig their own measuring systems.

Most of these systems are little more than a consumer-grade measurement mike surrounded by a bit of foam and mounted in an improvised apparatus to hold the headphone under test. Little or no attempt is made to replicate the acoustics of the human ear according to the international standards adhered to by the companies that make professional headphone measurement gear. As a result, measurements taken with these systems look radically different from measurements taken using professional gear. The charts are wildly inaccurate, and impossible to compare meaningfully with anyone else’s measurements. Yet some enthusiasts are drawing conclusions about headphone performance based on these measurements.

Another development that had some audio professionals cringing was the introduction of the Earphone Audio Response System (EARS, $179), a new headphone-measuring rig from miniDSP, best known for its affordable digital-signal-processing (DSP) audio processors. EARS appears to be little more than a couple of cheap measurement mikes mounted in a couple of plastic ears. Considering that a comparable professional product incorporating ear simulators and standardized simulated pinnae would typically cost around $12,000, it’s highly unlikely that the EARS conforms to any established standards, and it doesn’t seem to have been designed by anyone familiar with those standards.


miniDSP seems to intend EARS primarily as a way for enthusiasts to fine-tune physical mods or the equalization settings they use for their headphones, and it might be somewhat useful for those tasks -- while it doesn’t appear to be useful for absolute measurements, it might help gauge the effect of a change. But on the EARS webpage miniDSP promises that it “brings headphone measurement capability to ‘the masses.’” Considering the increasing popularity of amateur headphone measurements, it’s hard to believe that this device won’t be snapped up by enthusiasts -- even a few reviewers -- who, unfamiliar with the established standards of headphone measuring, will likely proclaim it to be “not as good as a pro system, but close enough.”

Before I bought my headphone-measuring gear about six years ago, I experimented with making my own, using techniques similar to the ones Mansilla described. I can tell you that the results of such efforts were far from “close enough” to professional gear. They weren’t even remotely close.

In his presentation at AISE, Mansilla begged headphone manufacturers to publish their own measurements when they introduce their products, and ended by saying (I paraphrase) that if manufacturers don’t publish measurements, consumers will. But in talking with fellow attendees afterward, I wasn’t convinced that most of them got the message.

Meanwhile, I strongly encourage SoundStage! readers to ignore any headphone measurements produced using nonstandardized test gear. Not only are the results misleading, the expertise of those using this homemade gear is questionable, to say the least. I’ve been performing audio measurements professionally for more than 20 years, during which I’ve learned to measure speakers, subwoofers, amplifiers, and, most recently, headphones. Each of these skills was added to my repertoire through laborious experimentation, countless late nights wrestling with finicky analyzers and digging through decades-old research, and long discussions with fellow professionals -- and I know that the handful of professional audio reviewers who do their own measuring have all endured the same test. In my experience, it takes any technician about two years of performing a particular measurement before he or she becomes truly proficient at it.

The bottom line is, owning measurement gear -- even the very best professional, standardized measuring system -- doesn’t make you an expert measurer, any more than owning a trumpet makes you a trumpeter.

. . . Brent Butterworth

More SoundStage! Videos

  • PMC Fact.8 Signature Loudspeaker Review! (Take 2, Ep:11)
  • Which is Better? Elac Navis ARB-51 vs. Focal Shape 65 (Take 2, Ep:10)
  • Amphion for the Home, the Studio, and Billie Eilish - SoundStage! Talks (August 2020)
  • The NEW Hegel H95 Integrated Amplifier Review ! (Take 2, Ep:9)
  • Estelon: Loudspeaker Excellence and Originality from Estonia - SoundStage! Talks (August 2020)
  • Zero-Distortion Amplification: Halcro Out of Hiatus - SoundStage! Talks (August 2020)
  • Best Active Speaker? Elac Navis ARB-51 Speaker Review (Take 2, Ep:8)
  • Vivid Audio's Inventive and Spectacular S12 Loudspeaker - SoundStage! InSight (August 2020)
  • Qobuz Hi-Res Traction and Younger Listeners, with David Solomon - SoundStage! Talks (July 2020)
  • Schiit Audio Ragnarok 2 Integrated Amplifier Review (Take 2, Ep:7)
  • Schiit Audio Bifrost 2 DAC Review (Take 2, Ep:6)
  • Vinshine Audio: Kinki Studio, Magico, and More in Singapore - SoundStage! Talks (July 2020)
  • Focal Shape 65 Active Speaker is a Winner for Both Audiophiles and Studios (Take 2, Ep:5)
  • Audio Research's 50th Anniversary Products, Celebration, and Future - SoundStage! Talks (July 2020)
  • Get Closer to Your Music with the HEDD Type 07 Active Speaker -- Literally. (Take 2, Ep:4) #AMTfi
  • Back to the
  • Details on the Bowers & Wilkins 700-Series Signature Speakers - SoundStage! Talks (June 2020)
  • Totem Acoustic Skylight Speaker Review! Affordable Speakers with a Big Sound (Take 2, Ep:2)
  • Activities at Hi-Fi Super-Retailer Goodwin's High End - SoundStage! Talks (June 2020)
  • The Long-Term Investment in a Bryston Amplifier - SoundStage! Talks (June 2020)
  • PMC and the Flagship Fenestria Loudspeaker - 2019 SoundStage! Product of the Year Winner (June 2020)
  • Hegel H120 Integrated Amplifier Review. Living With It for the Past 3 Months! (Take 2, Ep:1)