Confirming the audibility of lossyness (to my own satisfaction at least)

The problem I had with the common way of trying to determine which sound file is lossy and which isn't, is that the results I saw all showed that the chance of getting it right was - basically - chance. Being me I felt this proved nothing, and I had strong reservations about the method used. For the sake of statistical quality the test subjects have to sit through twelve - twelve - listens of the same song, which is bad enough but here's the kicker: they'd be played sequentially.

Now there's no doubting that audio compression codecs are advanced and do a very good job - too good, I'd wager, for someone to determine the difference between their output and the original between two different plays. Plus I'm an immediate kinda guy and I wanted to sort this out quicker.

I downloaded the Winamp Crossfader & Synchronization from here and configured winamp to run with multiple instances. I took two tracks which I had lossless (ALAC) copies of, and encoded one to 128kbps AAC and the other to 256kbps AAC. I decompressed both copies of each track into Wave files, made a copy of each, and renamed the copies using the following amazing method:

• Mash the keyboard to create a long, random filename like aulaervbaerhgviaeuklcunsbhvmnjrksrsrbsrtntralisd, and copy that filename.
• Without looking at the screen, hold down a cursor key for a quick count of two, and then mash the keyboard again.
• Rename the second file by pressing F2 and pasting the original filename.
• Again without looking, hold down the cursor key and mash the keyboard once more.

I know had two wave files whose filenames as displayed by Windows was the same: aulaervbaerhgvi... plus I don't know which one is going to be displayed first alphabetically.

I started up two instances of Winamp and loaded one wave file into each instance. Then I minimized the screen and started the crossfader plugin.

It's not that great - the tracks don't synch completely, so when the fader is centered you hear a noticeable echo, but you can quickly change between the two. It was easier to hear the difference between the 128kbps track and the original; more difficult and it took a while but you got there for the 256kbps track. Once I'd decided which side sounded best, all I had to do was do a file compare with the file being played and the original wave file I'd exported from the lossless copy, to see if I'd gotten it right.

I got two out of two with a high degree of certainty each time. It took between two and three minutes of listening to be sure which one was better for each track. People say things about 'fuller' and 'clearer' but what I also found was that at certain points, you could hear traces of that famous rumbling squeal that characterises low bit-rate encodes. That was enough for me - if the sound difference was at least noticeable by a direct compare, then there was definitely an audible difference between lossy and lossless and I at least have some ammunition to use against the doubters and justification for myself for loading lossless music onto my ipod. Reddit gets to read this because I don't have a blog.

tl;dr - I crossfaded between lossy and lossless tracks and heard the difference (allelujah).

Edit: Dogs by Pink Floyd and Conversation 16 by The National should anyone happen to be curious.