BASH: How to process files listed in text file?

[Valpskott]

I have 2 directories with png files. One directory is called "OLD" and the other one is called "NEW". Some of the files in these directory need to be processed, but only the ones that have been updated. For this I tried "diff --brief -r" combined with cat, grep and sed, this way I managed to get a list (outputed to a txt file) of the files that needs to be processed.

I also have successfully written the processing chain (this by entering the file-names manually).

How ever, I want this to be automated, so I need help so that the script processes the files that are mentioned in the TXT file.

[Vaphell]

while read -r f
do
  something with "$f"
done < file

or skipping the file stage entirely

while read -r f
do
  something with "$f"
done < <( cmd1 | cmd2 | cmd3 )

[Valpskott]

Thank you for the quick reply. I'm not quite sure how to implement this though.

Where in the code do I input the text file?
I guess the "proccessing code" should be where you wrote "something with "$f"" where the variable f is the information of each file mentioned in the txt file, correct?

Not sure what the "< file" or "< <( cmd1 | cmd2 | cmd3 )" is for after done. Does it suggest I do the diff grep sed thingie there, like after the loop? Then how will the loop know what files should be processed?

Please elaborate some on what this does so I get a better chance of understanding.

[SeijiSensei]

If the list is in a text file with one entry per line, I use this:

FILES=$(cat /path/to/the/file)

for f in $FILES
do
    stuff
done

Now for some caveats.

First, do the files in the list contain complete paths or just filenames? If the latter then you need to run the script from the directory in which the files are located, or have the script set the directory itself.

[Critics - please don't bother to complain about using "cat".]

[Valpskott]

after testing Vaphell's suggestion and inserting the text file after the "done <" stage, that part of the script seemed to work.

How ever, It seemed to have created a new problem with the processing code(which worked before) that I hope someone can help me with.

composite NEW/$f -blend 50x50 OLD/$f TEMP/$f.01.png
convert TEMP/$f.01.png -background "#808080" -flatten +matte \
-channel rgba -alpha set -fuzz 1% -fill none -opaque "#7f7f7f" \
-channel rgba -alpha set -fuzz 60% -fill black -opaque "#808080" \
-background "#ffffff" -flatten +matte \
-channel rgba -alpha set -fuzz 1% -fill none -opaque "#000000" TEMP/$f.02.png
composite -compose dst-out TEMP/$f.02.png NEW/$f -matte DIFF/$f

Which results in a bunch of errors.

convert: unable to open image `TEMP/armor/cloth_1.png.01.png':  @ error/blob.c/OpenBlob/2587.
convert: unable to open file `TEMP/armor/cloth_1.png.01.png' @ error/png.c/ReadPNGImage/3238.
convert: missing an image filename `TEMP/armor/cloth_1.png.02.png' @ error/convert.c/ConvertImageCommand/3011.
composite: unable to open image `TEMP/armor/cloth_1.png.02.png':  @ error/blob.c/OpenBlob/2587.
composite: unable to open file `TEMP/armor/cloth_1.png.02.png' @ error/png.c/ReadPNGImage/3238.
composite: missing an image filename `DIFF/armor/cloth_1.png' @ error/composite.c/CompositeImageCommand/1616.

I'm fairly confident it has something to do with the pathways, My guess is that imagemagick does not create the directories for me.

[SeijiSensei]

I'm fairly confident it has something to do with the pathways, My guess is that imagemagick does not create the directories for me.

No, it doesn't. You need to create them all in advance, or add lines to the script to do it for you before converting the files.

[Vaphell]

FILES=$(cat /path/to/the/file)

for f in $FILES
do
    stuff
done

ok, i'll bite
this will fail so hard with spaces... you need IFS= which suxx

then there is that cat... var=$( < file )

[SeijiSensei]

ok, i'll bite
this will fail so hard with spaces... you need IFS= which suxx

Yup, been there, done that.

Perhaps you can explain to me why

$(< filename)

is so much better than

$(cat filename)

other than what seems simple prejudice against using the cat command. In what circumstances would one alternative work and the other not?

[Valpskott]

After alot of tinkering I managed to make the script duplicate the directory structure. By cd:ing into the NEW directory, copying the files (cp --parents -r $f ../DIFF/$f) then removed the files (rm ../DIFF/$f).

I'm marking this thread as solved.

Thanks to both of you for helping me.

[Vaphell]

< is better because it is a standard functionality of shell and doesn't require spawning yet another process.
similar case is with touch - people create empty files with touch because they don't know they can use builtin > file for that (with nothing before >).

besides, while it doesn't make much difference here, there is plenty of difference between standard functionality that is intended for its purpose and pretty much foolproof, and handmade fugly hacks full of caveats, eg widespread use of for f in `ls` instead of for f in *