View Full Version : Kernel resource limitations & phred/phrap & cap3

October 2nd, 2006, 11:05 PM
I run into the 'Argument list too long' problem (described in linuxjournal: http://www.linuxjournal.com/article/6060) a lot as I'm doing genome sequencing and assembly using phred/phrap and recently also cap3. So, I often have hundreds of thousands of chromatogram files to move or work on at a time.
I get around it with shell functions, which works fine most of the time, but when running an assembly with 1/2 million reads it's not possible to break up the input inbto smaller chunks.
The linxjournal article mentions rebuilding gthe kernel and editing MAX_ARG_PAGES in binfmts.h to allow more arguments passed to a single command. I tried this, but it didn't make any difference. Reading the thread on kernel.org it looks like MAX_ARG_PAGES is hard-coded elsewhere too. (I wonder if the author of the article was using gentoo or had some other way round this).
Question: has anybody managed to get round the 'Argument list too long' problem by recompiling the kernel?
Alternatively, anyone had success running cap3 on large numbers of reads?

July 7th, 2008, 09:24 AM
A way around this is to use the find command.

find . -type f -name "*.ztr" -print0 | xargs -0 cat > ../filenames.txt

September 4th, 2008, 10:28 PM
A way around this is to use the find command.

find . -type f -name "*.ztr" -print0 | xargs -0 cat > ../filenames.txtFine if all I needed was a list of filenames? I was trying to do an assembly on a very large numebr of sequencing reads and cap3 keeps crapping out because of too many files. I don't see how find will help there, can you clarify?