Request for an ABX and ABC/HR application

Kees de Visser · Post by **Kees de Visser** » Tue May 22, 2007 9:13 pm

Double blind listening tests are important to verify if presumed audible differences are really audible (in the current test setup).
PC users have several software tools to perform ABX tests (http://www.pcabx.com/ , http://ff123.net/abchr/abchr.html).
There doesn't seem to be a good ABX tool for the Mac. Stephen has proven that he can write great audio applications. Perhaps he is willing to give it a try?

If anyone else is interested in an ABX tool for the Mac, please post a message.

Post by **sbooth** » Wed May 23, 2007 1:36 pm

If I was going to do this I would need some more information- I'm not sure what the desired features for an ABX application are!

krmathis · Post by **krmathis** » Wed May 23, 2007 7:24 pm

An ABX application are used to performed blind comparison tests between two audio files.
* One file being the lossless source. PCM (WAV or AIFF), Apple Lossless, FLAC, etc.
* The other file being compressed by a lossy encoder. MP3, AAC, MusePack, etc.

It would be great if it had these features:
* Lossy audio support (MP3, AAC, Ogg Vorbis, Musepack, ...)
* Lossless/Uncompressed audio support (Apple Lossless, WavPack, FLAC, PCM (AIFF and WAVE), ...).
* Gain correction.
* Offset correction.
* Select playback range.
* Sessions.

Download Java ABC/HR and give it a try.
Or take a look at ABX Comparator. Source code available...

The applications would need to decode the loaded audio tracks to temporary PCM streams and correct gain. Then allow the user to select a playback range to use for the comparison.
The user should then be able to switch between the two simultaneously playing tracks. Bla, bla, bla...

I would be VERY pleased if you took a look into this!

Kees de Visser · Post by **Kees de Visser** » Wed May 23, 2007 9:42 pm

krmathis wrote:An ABX application are used to performed blind comparison tests between two audio files.
* One file being the lossless source. PCM (WAV or AIFF), Apple Lossless, FLAC, etc.
* The other file being compressed by a lossy encoder. MP3, AAC, MusePack, etc.

IMO it's not only about testing lossy encoders.
An (audio) ABX application is used to compare (play back) two audio files, A and B.
The operator can audition A and B as many times and as long as desired. The ABX application presents a third source, X, which is randomly A or B. The operator has to identify X as being A or B. This test should be repeated several times to obtain statistical significant results.

For practical reasons the application should be able to play various formats on the fly (no need to convert to pcm first), including the ones you mentioned.
As an audio professional I'm also interested in comparing high-resolution audio, like 24/96, 24/192 e.g.
With the Mac being one of the best audio platforms, it's probably not too hard to implement the audio part.
There are a lot of ABX experts on the Hydrogenaudio forum. With their help it should be possible to make an accurate wishlist for an ABX application.
Your list seems a good start. I'd like to add:
* statistical analysis of the test results.
* seemless switching between the audio files during playback.

krmathis · Post by **krmathis** » Thu May 24, 2007 5:30 pm

Kees de Visser wrote:IMO it's not only about testing lossy encoders.

Ok, then.

But it don't make sense to ABX two files with identical sample rate and size. PCM vs. 24/192 sort of makes sense though.

Kees de Visser · Post by **Kees de Visser** » Fri May 25, 2007 10:53 pm

krmathis wrote:But it don't make sense to ABX two files with identical sample rate and size.

Sure it does !
Examples:
- compare different dither versions, all at 16/44.1
- compare different cd-masterings, all at 16/44.1
- compare different AD converters, all at the same rate

I've sent an email to ff123 (ABC/HR developer) and he's willing to help out a bit when needed.
He recommends contacting schnofler for most of the basics. I'll contact him next.

Stephen, does it help to have a pc application like WinABX or java ABC/HR as an example, or do you want to start from scratch ?

Post by **sbooth** » Sat May 26, 2007 5:08 am

Kees de Visser wrote:Stephen, does it help to have a pc application like WinABX or java ABC/HR as an example, or do you want to start from scratch ?

I will start from scratch.

I honestly don't think getting a basic ABX application working would be extremely difficult, with all the audio code I've written that can be reused. The biggest challenge will probably be the statistics portion!

krmathis · Post by **krmathis** » Sat May 26, 2007 6:57 am

Let's hope 'ABX' become a reality!

Kees de Visser · Post by **Kees de Visser** » Thu Jun 21, 2007 10:42 pm

A few updates:

- ff123 (ABCHR for windows author) has offered some help.
- schnofler (java ABC/HR author) is also prepared to help out, when needed. He might also be prepared to improve his java application for the Mac platform.

How detailed should a list with specifications be?
Should the items be devided into basic/essential and advanced/optional ?

Post by **sbooth** » Sun Jun 24, 2007 2:11 am

Kees de Visser wrote:- ff123 (ABCHR for windows author) has offered some help.
- schnofler (java ABC/HR author) is also prepared to help out, when needed. He might also be prepared to improve his java application for the Mac platform.

Good news!

How detailed should a list with specifications be?
Should the items be devided into basic/essential and advanced/optional ?

As detailed as possible- I'm not very familiar with ABC/HR or ABX testing. Dividing the features into basic and advanced would also be very helpful.

Kees de Visser · Post by **Kees de Visser** » Tue Sep 25, 2007 7:20 pm

It's been a while since the last post. If sbooth is still willing to give it a try, we will have to assemble a list of requirements to get him started. Krmathis, are you still here to help ?
I've also asked for help at the chair of the AES Technical Committee on Perception and Subjective Evaluation of Audio Signals. They might be able to give some guidelines for an ABX application implementation. I'll keep you posted.

Kees de Visser

krmathis · Post by **krmathis** » Thu Sep 27, 2007 3:41 pm

I am still here.
..and I have already assembled a list of requirements to get him started (see above

).

Not much more to do until he Stephen, or someone else, start coding.

Post by **sbooth** » Thu Sep 27, 2007 4:33 pm

krmathis wrote:Not much more to do until he Stephen, or someone else, start coding.

I'm beyond swamped at the moment. I still am interested in writing an ABX application, but I just don't have the time right now. Hopefully in a month or so I might be able to get started.

krmathis · Post by **krmathis** » Thu Sep 27, 2007 6:25 pm

That would be awesome!
Cause you seem to be the "only" developer interested in writing audio related applications for OS X.

Kees de Visser · Post by **Kees de Visser** » Sat Dec 08, 2007 9:19 pm

sbooth wrote:
krmathis wrote:Not much more to do until he Stephen, or someone else, start coding.
I'm beyond swamped at the moment. I still am interested in writing an ABX application, but I just don't have the time right now. Hopefully in a month or so I might be able to get started.

Just a short message to let you know we're still very much interested

Did you have time to have a look at it yet ?

sbooth.org

Request for an ABX and ABC/HR application

Request for an ABX and ABC/HR application

Re: Request for an ABX and ABC/HR application

Re: Request for an ABX and ABC/HR application

Re: Request for an ABX and ABC/HR application

Re: Request for an ABX and ABC/HR application

Re: Request for an ABX and ABC/HR application