[Table of Contents]


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [ARSCLIST] A proposed A/B test (was "dumbing down" of Downloaded Recordings)



Jon,

Well said.

The following AES pre-print from the 1991 (91st) convention is worth reading; I include three paragraphs below. --Marcos

TEN YEARS OF A/B/X TESTING
David Clark
DLC Design
Farmington Hills, MI 48335, USA
ABSTRACT
Experience from many years of double-blind listening tests
of audio equipment is summarized. The results are generally
consistent with threshold estimates from psychoacoustic
literature, that is, listeners often fail to prove they can hear
a difference after non-controlled listening suggested that there
was one. However, the "fantasy" of audible differences continues
despite the "fact" of audibility thresholds.
0 INTRODUCTION
Ten years ago the present author presented a paper to the
AES on double-blind testing using the A/B/X technique [1]. For
the next five years, a device to conveniently implement this test
was commercially available. It was thought by the author and his
associates that general use of this system would resolve "The
Great Debate" [2] of whether or not small differences in audio
components were audible. The debate rages on, however. It is
generally agreed by both sides that listeners fail to hear small
differences in double-blind tests, but some "believers" dismiss
the test as being inadequate to reveal what they hear clearly
when not being tested [3].
Questioning the test methodology is consistent with the
scientific method, particularly if it leads to a better test.
The problem here is that the challenge rarely leads to a better
test or vindication of the old test and acceptance of the
results. It could be argued that believing without scientific
confirmation is a harmless part of human nature. This is
acceptable for many of our personal choices from lucky charms to
comfortable furniture. Audio engineering, however, is the
business of delivering sound efficiently. It becomes an ethical
and perhaps legal question when it is claimed that improved sound
quality is delivered despite failure of tests to prove it.

Jon Noring wrote:
Jerry wrote:

I think with experience, we are learning to spot the faults in digital
recordings, which are not audible at first.

I do not dispute the fact that some can hear things that others do not. But
how much of this relates to the audio mikeing, mixing, amplification,
equalization, etc. process and how much to the digital part? Have those who
discern faults in digital recordings conducted blind comparisons of audio
and digital playbacks that originated from the same master?

I was definitely a little concerned whether to bring this up because the views concerning digitization of audio, whether for or against, oftentimes become almost religious.

And there are so many aspects to the digitization issue, it becomes
difficult to sort out even the known facts based on objective data
(which includes properly done listening tests.) And whether to go
digital or analog depend upon the particular purpose. In this group,
we are talking about the archiving of transfers from analog sources,
such as LPs, 78s, magnetic tape, etc.

As an example, in private email I was pointed to the 1984 Boston Audio
Society A/B test, which I remember studying a number of years ago but
forgot about until now. After reading it, it strongly indicated, but not
without some controversy, that even with 1984 digitization equipment
using 44.1/16-bit sampling, the "golden ears" could not discern any
better than guessing between "A" and "B", similar to the thought-
experiment I proposed in my prior message:

A: LP/78 playback --> pre-amp --> amplifier --> speakers

B: LP/78 playback --> pre-amp --> AD --> DA --> amplifier --> speakers

(There are a couple things that could obviously be done with the
Boston test to alleviate most of its controversies, such as using a
computer, or a coin toss, to reassign which is A and which is B for
each A/B comparison test, so as to remove the A or B preconceived
bias -- that is, the person may, after a few listens, develop a
preference for A or B for some subjective reason -- by mixing this up,
and letting the listener know A and B *may* switch at random at every
listen, the listener is less likely to develop a bias towards either A
or B due to subjective reasons having nothing to do with the test.)

I recall reading about a few other similar comparison tests done since
then, and I've concluded to myself for the above A and B chains,
that with today's much better digitization equipment, and the higher
96k/24bit sampling, that there will be no question NO ONE will be
able to tell the difference between A and B by careful listening
tests of actual audio material. (There might be faint discernment,
though, of the difference in noise floors when there is no input
signal. Inserting the A-D and D-A units in the stream adds a very
small amount of noise in the analog sections that we don't have in
the "A" system, which may actually favor "A". For archival purposes,
though, for the analog option we'd insert an analog recorder and
playback between the pre-amp and amp, and we know they will likewise
add a small amount of noise. This suggests refinements to the A/B
test.)

Of course, this is an assertion of belief based on my interpretation
of the data I've studied, so I propose this test be rerun, with
changes to alleviate the criticisms of the Boston Audio tests. Then
invite the "golden ears", especially those who believe digital audio of
any kind creates artifacts that the trained "golden ear" can discern,
to the listening, and see if any of the listeners can do better than
pure guessing (i.e., to meet statistical significance) as to which is
the full analog and which has the added A-D and D-A insertion.

Of course, from the archivist's perspective, there are two choices for
archiving:

1) place an analog recording system after the pre-amp

2) place a digital recording system after the pre-amp

One thing I do know, the analog recording system will distort the
signal (and add its own noise) much more upon recording than the
digital recording system will, and in my view this distortion is
unacceptable. It's bad enough that we have to deal with the vagaries
of the analog transfer (cartridge and pre-amp) which itself distorts
the signal from what was originally applied to the master.

(As an aside, it's amazing that the significance of the obvious
difference in "sound" between different cartridges among the
pro-analog crowd is totally lost on them. If each cartridge sounds
different, that's because each cartridge is distorting the audio in a
different way. This is "distortion", not accurate reproduction. Sure,
the "distortion" may be euphonic -- which I think contributes to the
belief that analog "sounds better" than digital, but nevertheless it
is an alteration of the analog signal which should be of concern to
the archival community.)

Thus, if the archivist has only one option for recording, they should
choose professional-grade 96k/24 bit (or better) digitization with
lossless compression. Doing both digital and analog recording, of
course, makes sense, and should be done, but not for audio quality
purposes, but rather for archival diversification. No one should do
only analog archival recording since that will be of lower accuracy to
the source due to inevitable distortion added by the analog recorder
(and during playback as well) and as stated above (yes as a statement
of belief) will not do any better, audio-quality-wise, than high-
quality digitization.

All in my rabid opinion, of course. <smile/>

Jon Noring


(p.s., just to note, I *love* the sound of good tube amplifiers, but I
know the reason I love the sound is because of euphonic distortion. I
have no difficulties with all the euphonic distortions added by analog
equipment for *listening enjoyment*, but for archival purposes, digital
is the only way to go as explained above. Now just tell those who love
"vinyl" over CD that the reason they like the vinyl sound better is
because of greater euphonic distortion from the whole analog chain of
mastering the LP and the cartridge playback. They believe it is
because the vinyl is more precise and accurate to the original... I
remember a statement by a professional violinist who is also an
audiophile, and he loved digital recording and playback because it
is more accurate to the original -- he should know. He stated that in
reality violins are pretty "harsh" instruments, and that the real
character of violins can only be brought out by digital recording and
playback. But most people prefer the sound of violins after having
passed through a number of analog stages of recording and playback
because of the reshaping of the sound by euphonic distortion. This to
me is further evidence that as archivists we must be concerned with
the most accurate reproduction of the original sound, and not be
swayed by the crowd who believe analog systems are more accurate
because "they sound better" -- they are not. One can always take the
digital recording and upon playback run it through a euphonic
"smoother" to recreate the sound of "vinyl" for those who just gotta
have that sound.)


[Subject index] [Index for current month] [Table of Contents]