Re: [ARSCLIST] ABX for Dummies

["Richard L. Hess" <arclists@xxxxxxxxxxxxxxx>]

Hello, Bruce,

As I described in the one (and only) series of formal ABX tests that 
I ran in order to discover the visibility of an embedded data 
structure in video we gave the test subject the ABX box and s/he 
could switch as many times as s/he desired before voting and there 
was multiple voting on each clip and there were multiple clips. Each 
subject ended up voting about 150 times. They did not have to vote at 
any point and they could watch the clip repeat 10x or more before 
voting the first time. They could trigger the vertical interval 
switch at any point and most, if not all, of the clips were frame 
accurately matched in playback from the uncompressed server. All 
video interconnects were serial digital at 270 Mb/s.

If that did not skew the results in favor of showing a perceptible 
difference, I don't know what would.

I think those of us who suggest ABX testing feel that it is better 
than not using it for certain things. One does have to make decisions 
as to whether to deploy a specific technology or not and there are 
often factors more than visibility/audibility that affect the choice.

I generally send some of my good friends files to listen to without 
telling them which is which. One test I've done is the playback of 
two audio tapes on three different recorders. The ranking of the 
recorders has been at least 80% the same: Studer A80, Sony APR-5000, 
Studer A810. Now, doing fresh alignments and choosing different tapes 
to play might change that, but these tests are a LOT of work. It took 
me about six hours to prepare the six audio clips, and, even with all 
that, I don't think the test was perfect.

The vision tests I did professionally took two engineers, a 
statistician, a programmer, and a wiring person weeks to 
prepare...and there again, we did things that we'd redone if we could have.

I'm glad you're bringing a rigorous intellectual experimental 
psychology perspective to this, but I also think that calling your 
post "ABX for Dummies" was a bit condescending to those of us who 
have looked into it with a bit more depth -- with the help of 
professionals such as statisticians.

I, personally, attempt to discern quality differences without 
prejudice, but recently I thought I could hear some quality 
differences where a friend whose usually better than I at picking out 
these things did not hear the quality differences. Perhaps I knew too 
much about the tests or my ability to switch back and forth between 
tracks was faster and more sensitive than his. I was not in a 
position to create a blind test for myself that I could then easily 
go back and figure out what I had done, so perhaps my prejudices were 
getting in the way equally as much as my friend's slower switching method.

Cheers,

Richard



Richard L. Hess                   email: richard@xxxxxxxxxxxxxxx
Aurora, Ontario, Canada       (905) 713 6733     1-877-TAPE-FIX
Detailed contact information: http://www.richardhess.com/tape/contact.htm
Quality tape transfers -- even from hard-to-play tapes.