File Conversion - Maximum Files?

[BigJim]

I put in 3400 files and Max crashed. I put in 996 and it handled it.
(this was just during the loading phase, did not even try to convert yet)
Are you using a static or dynamic data structure?

Later, it crashed once, running 5 threads during convert of about 1000 files.
(from FLAC to ALAC). But I ran the same files in smaller chunks later,
and no crash.

[sbooth]

Everything is dynamic, but the likeliest place for bogging down or crashing is in the metadata reading. Unfortunately all the metadata is read in at once currently, which is not efficient for large numbers of files.

[BigJim]

I have about 20,000 files to convert from flac to m4a. It seems I have to do them in 40 batches of 500 files. Even then it sometimes crashes unexpectedly. I can't say this happens when adding them to the File Conversion window with only 500, but it has happened a few times during the conversion. I don't care about how efficient is it for the CPU, I care about how efficient it is for the end user. Would running with only 1 thread help? Am I correct that Max is not completely stable for what I'm trying to do? I will try re-booting with no overclocking, its possible its my fault. Is it a priority to fix it? Do you want a crash log (how do I gather this)? Don't get me wrong, I think Max is great overall, just trying to make it better, and you can imagine that I'm trying to avoid repetitive motion strain injury from having to do 40 drag and drops (plus re-do things when it crashes, wonder if I'll end up with lots of duplicates in iTunes) (plus sometimes it stops to show me an error message) where one should work. Is there a command line I could use just for this conversion, then I could try to script it myself or use find | xargs. OK found this http://ask.metafilter.com/107172/FLAC-ALAC Its a shame that Apple still hasn't given up on proprietary standards and refuses to support flac, forcing us to thier asinine alac format, but I guess that's what they teach Jobs' MBA's to do in B-school.

[BigJim]

Dragged and dropped a directory of 20,000 .flac files onto Max. Before even getting a chance to click the Convert button, here is the crash. Lemme know if you want the whole report. If I send it to Apple do they forward it to you?

Code: Select all

Thread 0 Crashed:
0   com.apple.CoreFoundation      	0x90967e9e __CFStrConvertBytesToUnicode + 62
1   com.apple.CoreFoundation      	0x9097fabb _resolveFileSystemPaths + 139
2   com.apple.CoreFoundation      	0x9098a01f CFURLCreateStringWithFileSystemPath + 431
3   com.apple.CoreFoundation      	0x9098a8fa CFURLGetFileSystemRepresentation + 58
4   com.apple.CoreFoundation      	0x909a1c63 -[CFXPreferencesPropertyListSource(CFXPreferencesSPIAdditions) _backingPlistIsNewer] + 51
5   com.apple.CoreFoundation      	0x909a1904 -[CFXPreferencesPropertyListSource synchronize] + 36
6   com.apple.CoreFoundation      	0x9099e66d -[CFXPreferencesPropertyListSource getValueForKey:] + 45
7   com.apple.CoreFoundation      	0x9099e17b -[CFXPreferencesSearchListSource getValueForKey:] + 107
8   com.apple.CoreFoundation      	0x909a3900 CFPreferencesCopyAppValue + 80
9   com.apple.Foundation          	0x90009524 -[NSUserDefaults(NSUserDefaults) objectForKey:] + 36
10  com.apple.Foundation          	0x9005523a -[NSUserDefaults(NSUserDefaults) stringForKey:] + 42
11  org.sbooth.Max                	0x00028ac4 +[AudioMetadata(TagMappings) customizeFLACTag:] + 90
12  org.sbooth.Max                	0x00025fea +[AudioMetadata(FileMetadata) metadataFromFLACFile:] + 1316
13  org.sbooth.Max                	0x00044f38 -[FileConversionController(Private) addOneFile:atIndex:] + 726
14  org.sbooth.Max                	0x000446a9 -[FileConversionController addFile:atIndex:] + 704
15  org.sbooth.Max                	0x000425de -[FileArrayController tableView:acceptDrop:row:dropOperation:] + 504
16  com.apple.AppKit              	0x94d8fe1d -[NSTableView performDragOperation:] + 314
17  com.apple.AppKit              	0x94adef15 NSCoreDragReceiveProc + 1411
18  com.apple.HIServices          	0x93ab455c DoDropMessage + 97
19  com.apple.HIServices          	0x93ab3fbd CoreDragMessageHandler + 1267
20  com.apple.CoreFoundation      	0x90915e21 __CFMessagePortPerform + 193
21  com.apple.CoreFoundation      	0x90937921 CFRunLoopRunSpecific + 3921
22  com.apple.CoreFoundation      	0x90937d18 CFRunLoopRunInMode + 88
23  com.apple.HIToolbox           	0x911b86a0 RunCurrentEventLoopInMode + 283
24  com.apple.HIToolbox           	0x911b84b9 ReceiveNextEventCommon + 374
25  com.apple.HIToolbox           	0x911b832d BlockUntilNextEventMatchingListInMode + 106
26  com.apple.AppKit              	0x9480b7d9 _DPSNextEvent + 657
27  com.apple.AppKit              	0x9480b08e -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 128
28  com.apple.AppKit              	0x948040c5 -[NSApplication run] + 795
29  com.apple.AppKit              	0x947d130a NSApplicationMain + 574
30  org.sbooth.Max                	0x0000291b _start + 209
31  org.sbooth.Max                	0x00002849 start + 41

I looked at a script or command line to invoke ffmpeg, but I read that ffmpeg does not preserve the tags. I guess Apple wants people to buy the music. Though, they don't sell lossless music do they? Maybe in a few years they're planning to re-sell us our entire music libraries again in higher quality... Well right now I'm really at a loss on how to handle these lossless FLAC files. Max preserves tags but crashes, ffmpeg doesn't preserve tags. Maybe I'll try spoon feeding the files a few hundred at a time to Max and only enable 1 thread. Maybe I'll look for tools that run on Windows under VMware.

EDIT: update, XLD seems to handle the job. It too crashes if I ask it to scan too many files. Limit may be around 1000. But so far it doesn't crash during the conversion phase.

EDIT: OK, my new theory is there is like some "killer folder" or "killer file" that crashes the scan. So it may not be a limit, just, the more you throw in the more chance you have of hitting a folder/file that won't scan. In my case I identfied "Bullitt soundtrack" as one such. Just to say it though, programs whose purpose is to process all kinds of random junk downloaded from the internet, should never trust their input data, and should detect and report errors rather than just crash.

[BigJim]

Sorry that nobody cares about this. I didn't come here to plug XLD, but in the end that is what got the heavy lifting done.

[rydenfan]

I cant seem to be able to encode more than about 10 albums at a time without MAX crashing. I would have expected it to handle a much larger amount.

[sbooth]

Sorry that nobody cares about this. I didn't come here to plug XLD, but in the end that is what got the heavy lifting done.

I care, I am just extremely busy with my "real" job and other commitments. I am the sole developer of Max, and I don't have as much time as I would like to devote to it.

[BigJim]

I would like to apologize for my unfair words.