Converting to Apple Lossless - not 100% bit accurate?

[scott brown]

I've been messing around with converting files to Apple Lossless and have run into a problem: the Apple Lossless files don't convert back to the original wav files. The data is not 100% the same.

Example: I just fed this wav file to Max (output courtesy of shntool):

Code: Select all

Length:                       6:40.27
WAVE format:                  0x0001 (Microsoft PCM)
Channels:                     2
Bits/sample:                  16
Samples/sec:                  44100
Average bytes/sec:            176400
Rate (calculated):            176400
Block align:                  4
Header size:                  44 bytes
Data size:                    70623504 bytes
Chunk size:                   70623540 bytes
Total size (chunk size + 8):  70623548 bytes
Actual file size:             70623548
File is compressed:           no
Compression ratio:            1.0000
CD-quality properties:
  CD quality:                 yes
  Cut on sector boundary:     yes
  Sector misalignment:        0 bytes
  Long enough to be burned:   yes
WAVE properties:
  Non-canonical header:       no
  Extra RIFF chunks:          no
Possible problems:
  File contains ID3v2 tag:    no
  Data chunk block-aligned:   yes
  Inconsistent header:        no
  File probably truncated:    no
  Junk appended to file:      no
  Odd data size has pad byte: n/a

I had Max convert it to Apple Lossless. I then had Max convert that Apple Lossless file to WAV. Here is the resulting file:

Code: Select all

Length:                       6:40.24
WAVE format:                  0x0001 (Microsoft PCM)
Channels:                     2
Bits/sample:                  16
Samples/sec:                  44100
Average bytes/sec:            176400
Rate (calculated):            176400
Block align:                  4
Header size:                  44 bytes
Data size:                    70615584 bytes
Chunk size:                   70616759 bytes
Total size (chunk size + 8):  70616767 bytes
Actual file size:             70616768
File is compressed:           no
Compression ratio:            1.0000
CD-quality properties:
  CD quality:                 yes
  Cut on sector boundary:     no
  Sector misalignment:        1488 bytes
  Long enough to be burned:   yes
WAVE properties:
  Non-canonical header:       no
  Extra RIFF chunks:          yes (1139 bytes)
Possible problems:
  File contains ID3v2 tag:    no
  Data chunk block-aligned:   yes
  Inconsistent header:        no
  File probably truncated:    no
  Junk appended to file:      yes (1 byte)
  Odd data size has pad byte: n/a

As you can see, the 2 don't match. I've been struggling for a few weeks to figure out what's going on (including looking at the source code), but I haven't been able to figure it out yet. If I set the type to kAudioFileCAFType in code instead of kAudioFileM4AType, the files are totally fine. So maybe it's something with the M4A type?

Apple's afconvert utility has the same issue, and in trying to write a core audio utility to do it myself, I have the exact same issue. But XLD doesn't have the same issue.

So I'm posting here to give a heads up that the Apple Lossless conversion may not be 100% accurate. Maybe it's an Apple bug?

[Fuga]

Bump? Very worrying as I make great use of Max for ALAC.

[sbooth]

There is definitely something amiss, although I cannot determine exactly what is going on quite yet.

At first glance, it seems that M4A/Apple Lossless files created by Max contain all the data from the input file but are not handled properly on decompression. Core Audio reports an incorrect number of frames for these files.

Here is what I have tried:

Converting WAVE to M4A/ALAC and back to WAV with Max will NOT get the original wave back. There is a loss of frames at the end of the file that varies (684 in one of my tests, and it appears to be related to the alac remainder).

Converting WAV to M4A/ALAC with Max, then converting the M4A back to WAV with iTunes will get the original wave back.

[scott brown]

i feel dumb...I didn't even consider trying to convert back with iTunes. I just tried Max, XLD, afconvert and my own core audio test.

anyway, I knew I wasn't imagining this

[Fuga]

So, dumb question time ...

Files converted to ALAC are actually fine as ALAC files? That until otherwise, going *from* ALAC should use iTunes?

Thanks,

[sbooth]

Files converted to ALAC are actually fine as ALAC files? That until otherwise, going *from* ALAC should use iTunes

I think this is the case (say 80% certainty), but I'm not sure. As far as I can tell converting to ALAC is OK. I definitely would not use Max to convert from ALAC until we figure out the exact cause of the problem.

I peeked at the XLD source code, and for ALAC decoding tmkk completely ignores what CoreAudio says as far as the frame count, and reads it directly from the MP4 atoms. So that explains why XLD doesn't have this particular problem.

[scott brown]

Did you try converting to ALAC in Max and from ALAC in XLD? Was that OK?

I thought I tried that last week and it failed, but I may be wrong. I'll play around again tonight when I get away from this Windows machine at work...

[sbooth]

You're right- encoding to ALAC with Max and then back to WAV in XLD does not recover the original file either.

So it seems the problem might not be limited to decoding- Core Audio could be writing something funky to the files.

[Fuga]

What a kurfufle. Glad you two are on this.

[sbooth]

I've found the problem- Core Audio is writing incorrect iTunSMPB atoms to M4A/ALAC files. I think this is only an issue on 10.6, as 10.5 in the past did not write these atoms. I no longer have a 10.5 system, though, so I can't be sure.

I'll be releasing a new version of Max in the very near future which works around this problem.

In the meantime, if you have M4A/ALAC files encoded with Max on 10.6, you may want to strip out the iTunSMPB atoms. This is easily accomplished using AtomicParsley:

Code: Select all

AtomicParsley /path/to/file.m4a --manualAtomRemove moov.udta.meta.ilst.----.name:[iTunSMPB]

If you don't want to remove them, it is possible to fix them although it requires a bit more work.

[scott brown]

nice work!

[Fuga]

Am I right? Using Atomicparsley is a one file at a time process? If so that's not gonna happen.

But then that's not a real problem as I don't see me using those ALACs for anything but what they're doing now. (crosses fingers original CDs stay OK).

I'm glad to see a quick fix for future conversions coming.

[scott brown]

Am I right? Using Atomicparsley is a one file at a time process? If so that's not gonna happen.

i'm sure you can do it for a bunch of files at once (*.m4a), though if there's interest, I can quickly whip up a little droplet app - drop the files on the icon and it'll fix them

[Fuga]

Scott, see my edit above. OTOH, should something like that appear I sure would use it. My ALACs are mostly archival. But there are some that originated as friends' and family's CDs, files from which I might want to make a CD someday. So, count me as a definite maybe for someone wanting such a thing.

[sbooth]

Another solution would be to read the number of frames from one of the moov atoms and then rewrite the iTunSMPB block with the correct value. This would not be too difficult, but would likely be of dubious utility since I don't believe that iTunes needs the iTunSMPB block for ALAC gapless playback.