Thx Keith for this very interesting and renewing article, cool addition to the Madshrimps site!

I hate to rain on this parade, but the article is severely flawed.

Most importantly, since there is no way to literally compare speakers side-by-side, as there is with video card image quality for example, the next best option is double-blind testing. The next best methodology after that is single blind testing- have someone else switching between speakers without telling the reviewer which ones they're hearing. The worst way to do it is to have the reviewer change the speakers themselves, as seems to be the case here. The observer-expectancy effect is unavoidable (you'll have to look it up, since I'm not allowed to post links yet), and throws the reviewer's objectivity into serious doubt. Credulous reviewers who eschew blind testing have brought us such laughable products as $14 000 speaker cables and $300 electrical outlet covers- look 'em up if you don't believe me.

Second, why on earth would you listen to MP3s (or game audio, which generally means MP3s) through an onboard sound chip? You might as well hook your $x000 setup to a tape deck! At least spend $100 for a decent sound card, and use uncompressed audio. While I realize that this is only part of the review, it is a rather pointless excercise.

haelduksf, keep in mind that Keith was testing the overall performance of the speakers, that's why he alsolistened to mp3 compressed music, as most people do.

@Keith: where's the cd-listening part in the A2 review ?

haelduksf as much as I dislike having my "parade rained upon" it's not actually "my" parade. The listening methods explained and carried out in my article are used by almost all leading review publications in High End Audio. Some, such as Stereophile may augment their evluations with electronic tests, however; the gist of the review will always be gleaned from the descriptions of the Audiophile/Journalist doing the listening. However it seems by your suggestions or criticims you haven't read the article through. Much of what your addressing about compression rates and such were mentioned, let me explain the sampling rates.

SOMA FM I offers several options for each station on the streaming and sample rate, such as 128kbps streaming, 44kHz sampling rate, however I did also listen to their 32kbps bit rate and 22kHz sample rate. The differences were notable through the soundcard, and Audioengine A2's more notable on the A5's. however this was clearly distinguishable using the Decco integrated amplifier with onboard USB DAC and the Peachtree Audio Era Design-4 speakers. 44kHz offered much more detail, depth, bottom end, essentially more information. The latter 22kHz was very forward and percieved as artificial. I've cropped a shot of the different samling rates which anyone can test (blind or double blind if you wish) by loading each into an ITune GUI / player and then simply clicking on one then the other.

As far as your mentioning audio cards, I will test PC-cards in the future, Pro cards seem to be the best option where pure neutrality is the goal although the Asus Xonar DX2 uses Burr Brown DACs which is a step in the right direction. Unfortunately IMO audiophile grade PC sound cards do not exist nor can they for one simple reason their mounted in a PC. Even if the card was encased in lead and used Burr Brown DACs the EMI inherent in the motherboard itself would render the sophisticated circuits moot.

This is why (as mentioned in the article) most of the upcoming component reviews will revolve around USB DAC based integrated ampifiers such as the Decco hybrid, Tecon Model 55 and soon to arrive GLOW AmpOne amplifer both intergrated vacuum tubes amplifiers with USB DACs. These intergated vacuum tube amplifers have become so affordable anyone can now hear High End sound. Mated to a pair of Cain & Cain Abby's for about what you pauid for your PC ($2,000) you can own a truly exotic High End system as you'll see when i review the Abby's whicu are a breed abart in floor standing speakers seen below.



The second reason is the isolation of DAC stage, eliminates a large degree of EMI. I cited several amplifiers already being tested as I write this which include on-board USB DACs thereby circumventing the normally poor quality of on-board sound no matter the card. In the first USB DAC component review I will absolutely be discussing different compression files and of course Lossless (as I mentioned) and uncompressed files.

I understand your concerns about blind testing, however; after many years of listening to hundreds of High End components I absolutely have no trouble trusting the opinions of these journalists and the expereinces of fellow Audiophiles as to their assement of a product's virtues and defects. I know these evaluations in most cases to be very accurate based on emprical evidence.

As I developed a passion for Audiophile grade hardware, yet wasn't able to afford much of the costlier items (at that time there were just a fraction of affordable High End components there are today) I began brokering (buy/sell) pre-owned Audiophile gear to gain access to more of it. Out of hundreds of high End components which passed thorugh my home, each was evaluated and the design scrutinized. Whenever products were available which had been reviewed by either Stereophile or The Absolute Sound, I would jump on them for two reasons. First they were easy to move once they got an endorsement. Second I wanted to hear for myself and determine whether I was able to distinguish these attributes audiophile journalists were hearing.

In almost every circumstance what the journalist reported hearing (using the same recording and associated equipment, although the latter wasn't always necessary) was distinguishable to me. Audiophile grade components have a sonic signature, despite trying to be "neutral." Oddly even their neutraility varies although this soudns like a misnomer.

Before I explain why Double Blind or Blind testing is just not used in High End (and it has been done to death by these same publications) let me address the use of electronic instruments for the evaluation of Audio gear, or objectivism vs. subjectivism.

There is no test instrument which can detect three dimensional imaging, this occurs in your mind as does the entire musical experience and for this reason its not so simple to say what cannot be measured cannot be percieved. Science is not always correct and we understand much less then we claim to I would take offense to anyone whom claimed I wasn't scientific, merely because I didn't use instruments where their use isn't warranted. The following quote from a reader response on this very subject in Stereophile expains some of the complexities:

Ask any physicist, "Are you an experimentalist or a theorist?" They'll know what you mean because the two kinds of work are so different. With Plato's devotion to mathematics and abstract concepts, theorists often need little more than a pad, a pen, and their mind. Experimentalists, on the other hand, are more like Aristotle. They'll spend weeks and months surrounded by instruments and machines, taking careful measurements or observing how nature behaves under conditions they've created in their labs.

You mentioned Blind and Double Blind testing which is considered untenable for High End (or any end for that matter) in evaluating audio gear. To put it simply People hear differently, based upon a pethora of mitigating factors, perhaps most important their life experiences. When I began listening, I mean really listening I already had the experience of growing up in a household filled with music. Mostly recorded, but never the less, there it was. When I began experiencing the kind of details only High End hardware can reveal it took a great deal of "training" my ears to discern between certain phenomena.

Then of course one must discriminate these instruments when amplified through vacuum tubes, in triode, or pentode, through transistors, or transistors with a tube input stage. Is there more warmth, or do details that were clean now sound as if someone threw a wet towel over the instrument.

Just as an athlete builds and trains certain parts of his/her anatomy the Audiophile has trained his/her ears and mind. This is not to say that the average person couldn't walk into a room and hear those same things, they just lack the tools to describe them.

As an example lets say you wanted to do a double blind test on two different pairs of running shoes. You select 100 people at random and ask them to run a 1/2 mile with sneaker A then sneaker B. Without a census there was no way to tell that half the people couldn't even run that 1/2mi and the other half would develop blisters. By the time the second pair of running shoes are tested only a few survived both tests, and all of these were runners. Some of the runners may have got blisters but they completed the tests and we're better able to communicate what was wrong with sneaker A or B and why. Perhaps one or two non-runners also did this, but that percentage was minute. What have you really proven other then people differ in their perceptions and experiences. Those whom ran daily were accustomed to wearing running shoes in the first place and had the stamina and strength to finish, those whom had never worn running shoes or had only worn them for walking could provide some data but how to extrapolate.

On that I'll end with these two links which may or may not open your "mind" as to why magazines such as Stereophile, 6moons, EnjoytheMusic, Audiophilia, TheAbsoluteSound Online, Sound Stage, and Stereo Times to name most online, all of which employ the same methods I used.

Please take he time to understansd the other side of the coin your grasping, and read: The blind leading the Blind and Blind Tests and Bus Stops.

The only relevant data blind tests reveal is whom the best listeners are in the room.

Wow Keith, just wow. Thanks for your reply, very interesting back story there and reasoning for blind testing.

Thank you

And haelduksf please don't take anything I've said as personal or an attack upon your knowledge of audio. I do understand the reasons behind blind testing I've chosen to follow another path for the reasons mentioned.

There is a rift in the audio world between Audiophiles and Engineers but there are many of us whom know we need each other.

Whoa- that's quite the manuscript! You needn't worry though, no offence was taken. I don't claim to be an audiophile myself- I am a computer science student, and the only setup i can boast of is a Xonar and a set of Logitech Z-560s. I am, however, both a scientist (in training, i suppose) and a computer hardware enthusiast. I write for one of the most scientifically rigorous hardware testing sites around. I wouldn't go so far as to critique your ears... all I am critiquing is the testing methodology.

I have read both the links you provided regarding blind testing, and I must admit that I'm not impressed. Those are a couple of lovely anecdotes... however, they offer little of substance. As Karl Sagan once said, "The plural of 'anecdote' is not 'evidence'". It maybe illuminating to note that John Atkinson, publisher of Stereophile magazine and author of the second of the articles quoted below, was invited to take the James Randi million dollar challenge on at least two occasions, and declined both. All he'd have to do for a million bucks is what he claims to have done already- tell two pieces of equipment apart in a blind test. And yet, he still hasn't put someone else's money where his mouth is.

If you are interested, there are a number of people who have done further blind listening tests on various components. For example (truncated, since I'm still not allowed URLs):
www.mastersonaudio.com/audio/20020901.htm
www.provide.net/~djcarlst/abx_wire.htm
http://www.hometheaterhifi.com/volum...s-12-2004.html
http://forums.audioholics.com/forums...2&postcount=28

These are all from the first page of Google results! Most of them are not performed on laypeople, but on self-professed audioholics, highly trained and experienced.

Frankly, I blame Stereophile et al's highly subjective testing for the laughable pseudoscience quackery of such vendors as Machina Dynamica (machinadynamica.com-- make sure you aren't drinking anything when you visit, I'm not responsible for any keyboards destroyed by laugh-propelled beverages). If you give someone a product, whether it be a loudspeaker or a wine or a car and tell them "this product is x", they are highly likely to agree with you, no matter the truth of x. For an excellent example, have a look at this ten-year-old's science fair experiment: randi.org/jr/121004science.html.

I have a working understanding of MP3s... not only am I a fan of Soma's Cliqhop and Secret Agent stations, I took a whole course in digital sampling and compression only 6 months ago My suggestion was not that you couldn't tell the difference between streams of varying bitrates, it was that MP3s are so lossy and corrupted to begin with that it's a waste to use them to test any speakers above $20, since the limitations of the compression will be reached long before that of the hardware used.

This methodology just doesn't seem to fit in with the site's normal excellent testing; cold hard facts, numbers, and repeatability.

they are not asking people what "sounds" better, which would make it a quality blind test; but answer what hardware was used to play back the music... hmmm

One reason I do not participate in forums as much as I once used to, are scenarios just like this. We can volley quotes over a simulated net of common experience endlessly and its very likely we will not have accomplished anything worth while.

To respond in kind to your subtle condescension of the cohesive thoughts of a devoted audio enthusiast would be the correct format, but the wrong thing to do. I'll simply say James Randi and the term paranormal. Both of which have done more harm then good to the field of psychology and have only widened the rift between proponents of blind testing and Audiophiles.

So as not to insult your intelligence and position albeit misguided by rhetoric I will admit my response to you was lazy in that I did not cite more "scientifically conclusive" data. Of course that's part of the conundrum. Qualia as discussed in some of the material I've cited, such as "redness," pain, imaging cannot be measured scientifically (at least not fully). I won't insult Carl Sagan, by quoting someone else to counter your quoting him and so on and so on it goes, round and round the Mulberry bush the monkey chased the weasel.

I will provide the following based on the fact between us, this person is much closer to knowing anything about the wonderful world of Randi because he's answered the challange and had actual contact with those behind the challenge: The Swiftboating of Audiophiles.

To appeal in some way to those whom support blind testing I will employ some method of blind testing in my next Audio review. I would suggest in turn you visit your local high End Shop and ask to listen to a tube ampifer and then compare it to a solid state amplifer driving the same speakers using the same source. This should NOT be a blind listening test.

And while I rarely practice this little known talent I am about to do something I haven't done in many years (at least in print) and that is reveal to you and readers of Madshrimps my extraordinary paranormal ability.

"I predict, even after reading the article linked above you will still disagree with my testing methodology. You will have come no closer to changing your position on the legitimacy of evaluating HiFi via blind and double blind methods. You will will or will not go to your local high End shop where they will or will not switch out the amplifer only between a tube and solid state model. And finally you will not remain an advocate of Randi's Challange, but you will remain in support of the principle"

To be sure my predictions are not effected or influenced I have chosen not to read up on, or Google any information pertaining to Randi's challange and I typed this with my eyes closed.

In the end the "Amazing Randio" is a retired magician with a 12th grade education? That may have no effect on his intellectual abilities, but what is important is how he applied these skills in life. I feel bad being so hard on James Randi, but he hasn't been so kind to others from what I've read. Gooday Mate

I have no intention of starting an internet argument, and this will be my last post on the subject. I only hope to have enlightened some of your readers (or at least as many as actually read these comments). Of course, I have no delusions of changing your own opinion of audiophile gear/testing; you have, at this point, invested far too much of your time, effort, money, and reputation to perform an about-face now.

I will not waste an AV shop owner's time, demoing equipment that I have neither the means nor the desire to buy. The best I am willing to do is comparing a friend's audiophile gear to his Peavy commercial amp... and to be frank, I've been there and done that, and heard nothing special. Of course, this only proves that I have unsophisticated ears, right? Either way, you may write as many reviews as you wish, using whatever methodology you wish. I have ignored all the "golden-eared" placebo-peddlers before you; one more should be doable.

As for your attacks on Mr. Randi, it is blindingly obvious that you've read up on him and his challenge; your smears are the same as have been repeated ad nauseam against Mr. Randi by quacks and psychics for the last 10 years, and I think they speak for themselves.

It's feelings and belief systems which (IMO) drive us (me) to react to criticism, albeit veiled, subtle or terse. My philosophy is that the only "corner of the universe I can be certain of improving, is my own self." This is not so easy if you "feel" attacked. Especially when someone is adamant in their position. When I choose to stand defiant thinking only about right and wrong, I fail to grow. This does not mean I must rearrange my personal mosaic, my belief systems each time someone confronts me. And while it seems like it just the method being confronted when comments are made (founded or not) about people like Mr Randi or Audiophiles with "golden ears" not only is there collateral damage, we get further from the truth.

In Philosophy an argument has no negative connotations, it is a means to an end. When I said I would use blond testing in my next review, I wasn't being patronizing nor sarcastic, I am going to try it because I feel it may have validity.

Your criticism is often more valuable then a string of compliments, since I cannot always see myself, nor am all knowing. I understand enough psychology to know one can certainly convince oneself a $600 pair of speaker cable sound better then a $200 pair, and then there's the science. If silver is a better conductor then Copper, and shielding protects from EMI, then why shouldn't that manifest itself as an improvement in sound?

None the less I am sincere when i say; thank you for your criticism, it may (I hope) give my future tests more validity.

Soon I'll be testing cables such as the Oyaide Tunami Nigo speaker cable below, that'll be fun...