Re: [ARSCLIST] A/B testing: another approach

[Matthew Barton <mbarton@xxxxxxx>]

It wasn't a scientific experiment--just an engineer having a bit of
fun--though he seems to have had a point to make, if not a detailedl
hypothesis to test. I just thought the structure was interesting, and
worth considering if our goal is to develop a listening test or tests.
Perhaps the thing that I find most interesting here is that it involved
a real-time, uninterrupted  listening experience of A, B, C, and D.
Perhaps the brain does respond differently to such a listening
experience.

Matthew Barton
MBRS
The Library of Congress
101 Independence Ave., SE
Washington, DC 20540-4696
202-707-5508
email: mbarton@xxxxxxx

>>> Marcos Sueiro Bal <mls2137@xxxxxxxxxxxx> 1/28/2008 10:40:58 AM >>>
Matt,

This is an interesting link, but as a scientific experiment it does not

seem very useful: What is the hypothesis? How are we quantifying it? A

statement such as "my fellow listeners appeared to be equally 
uncomfortable" does not seem conducive to analysis.

If the highs were perceived not to be "as silky smooth" --in other 
words, if the differentiating factor has been identified after just one

listen of a short passage--, should not the same listener be able to 
correctly identify such a difference in a blind test? Logic seems to 
indicate that he should, but perhaps the brain works in mysterious
ways.

Incidentally, it seems that not all ABX tests have concluded that 
listeners are less sensitive than we thought. I was told in school that

most average Joes can hear at most a difference of 1 dB, but a group of

5 listeners in an ABX test perceived differences of 0.4 dB 93% of the 
time (note: this is not a peer-reviewed paper, and this is from the ABX

web page, so it is not conclusive evidence).

http://www.provide.net/~djcarlst/abx_lvl.htm 

Cheers,

marcos

Matthew Barton wrote:
> Here's a link to an article from the October issue of Stereophile,
in
> which an interesting approach to blindfold testing is described:
>
> http://www.stereophile.com/asweseeit/1007awsi/index.html 
>
> This is not an analog vs. digital article, and I'm not endorsing the
> test or or its results, or any conclusions in the article, but I
think
> the approach is interesting. Instead of an A:B comparision, in which
> listeners first heard A, and then B, and were asked for opinions,
this
> engineer created a composite patchwork of different formats using a
> repetitive passage from a recent recording of Handel's Messiah. He
> didn't tell his audience that this is what they would be hearing:
>
> "It turned out that we'd been unwittingly involved in a blind
listening
> test. The DVD-A was a ringer. Philip had chosen a Handel chorus in
which
> the same music is heard four times. He had prepared four versions of
the
> chorus—the original 24-bit/88.2kHz data transcoded straight from
the
> DSD master; a version sample-rate–converted and decimated to
16/44.1
> CD data; an MP3 version at 320kbps; and, finally, an MP3 version at
> 192kbps—and spliced them together in that order. The last three
> versions had been subsequently upsampled back to 24/88.2 so that the
> DAC's performance would not be a variable. The peak and average
levels
> were the same for all four versions; the only difference we would
hear
> would be the reductions in bandwidth and resolution. "-- from
"Watching
> the Detectives," by John Atkinson, Stereophile, October, 2007.
>
> We can all argue about the specs here, but the most interesting
thing
> to me is that the changes in the audio unfolded over four iterations
of
> the same passage of music in the same recording. Listeners were not
> asked to use their memory of recording A to appraise recording B,or
vice
> versa. They heard (or did not hear) the changes as part of
continuous
> listening experience.
>
>
> Matthew Barton
> MBRS
> The Library of Congress
> 101 Independence Ave., SE
> Washington, DC 20540-4696
> 202-707-5508
> email: mbarton@xxxxxxx 
>