how to handle strings with spaces

[AmbiguousOutlier]

Hello, I'm attempting my first script. I'm trying to do a loop that picks up all text files in a directory including all it's subdirectories. However whenever there is a space in the one of the directories the "for in" statement treats this as two separate file names. Is there any way of treating them the same?

for i in $(find . -name *.txt); do #i think I need to put quotes around something??
	echo $i
done

alternatively I was thinking I could

count=$(find . -name *.txt | wc -l);
for i = 1 to $count; do
	echo find . -name *.txt | wc -l #obviously I need to somehow only select one at a time
done

Any thoughts?

[steeldriver]

Try

for i in "$(find . -name '*.txt')"; do 
    echo "$i"
done

or maybe better

while read -r -d $'\0' i; do 
    echo "$i"; 
done < <(find . -name '*.txt' -print0)

[LeDieu]

You can do the same without the find command:

#!/bin/bash

for file in *.txt
do
        echo $file
done

Hope this is what you where looking for.

[schragge]

You can do the same without the find command:

Nope.

I'm trying to do a loop that picks up all text files in a directory including all it's subdirectories.

In bash, you can make shell glob recurse through subdirectories, see the next post by steeldriver below.

Still, the while loop with read suggested by steeldriver is the most general solution. Quoting the find output inside for loop's in list won't work as it will turn the whole output of find into one argument. If the loop body only consists of one command then using find's action -exec may also be a viable alternative:

find -name \*.txt -exec command {} \;

And even if there are more than one command, sometimes -exec may be convenient:

find -name \*.txt -exec sh -c 'command1 "$0"; command2 "$0"' {} \;

[steeldriver]

... LeDieu's approach might work for subdirs if you enable the bash 'globstar' option

You can do the same without the find command:

#!/bin/bash

shopt -s globstar
shopt -s nullglob

for file in **/*.txt
do
        echo "$file"
done

Hope this is what you where looking for.

but the 'while loop' is how I'd do it, I think

[AmbiguousOutlier]

I could potentially be handling hundreds of files so if I understand the -exec command, this wouldn't work.

However I can't get the while loop working.

Try

while read -r -d $'\0' i; do 
    echo "$i"; 
done < <(find . -name '*.txt' -print0)

The above produces a redirection error.

while read -r -d $'\0' i; do 
    echo "$i"; 
done < $(find . -name '*.txt' -print0)

This still concatenates the filenames together. And looks like it's trying to execute the file.

I don't really understand what the above is doing so if I simplify it down, I get the below which still doesn't work;

while read i; do 
	echo $i; 
done < $(find . -name '*.jpg')

rhys@Orange:~/Documents/Scripts$ ./exists
./exists: 3: ./exists: cannot open ./flac_music/Evanescence/folder.jpg
./flac_music/Evanescence/fanart.jpg
./flac_music/Evanescence/Fallen/folder.jpg: No such file

But it got me thinking, I could output the result of the find into a text file and then read the text file in one by one.

find . -name '*.jpg' -print > ./file.txt

while read i
do           
	echo $i           
	echo "file processed"
done <file.txt

This has also given me the idea of producing a log file. What fun lies ahead.

I'm still interested to know what I'm doing wrong on the loops though.

[Vaphell]

I could potentially be handling hundreds of files so if I understand the -exec command, this wouldn't work.

depends how you are going to be processing these files. -exec simply executes the command with found name(s) pasted in. It might be unwieldy due to syntax, but not inherently impossible to do pretty much anything. That said, i do these in simple cases and go while read route for anything remotely complicated.


What do you run your script with? Your hashbang line should point to bash because some of mentioned tricks are bash features.

#!/bin/sh

vs

#!/bin/bash

?

test in terminal

$ sh    #switching to sh
$ while read f; do echo "$f"; done < <( find ) 
sh: Syntax error: redirection unexpected
$ bash # back to bash
$ while read f; do echo "$f"; done < <( find )
<plenty of paths>

Check if you can work with builtin globs, rock solid out of the bat. If you are forced to use find, in case of while read ; do... ; done < <(cmd) try to use \0 as often as possible. It's the only char that allows for unambiguous, rock solid solution for file processing.

builtin globs > while read -d $'\0' > lazy hacks

tl; dr: stick to #!/bin/bash and \0 as delimiter

[AmbiguousOutlier]

Yeah, I was using sh not bash. I've got it working now so I'll stick with it, what I've got works with both bash and sh. I guess that line means something.

I'm stuck again though, I have a file called;

/original/folder/subfolder/file.txt

I don't want to copy the file, but I do want to mirror the folder structure like below;

/mirror/folder/subfolder/

I then want to do some processing and move the new file to the new location keeping the original in tact.

I've used the sed command to substitute the directory however I need to chop off just the file name, so I can easily mkdir the string. But I'm not sure what to google.

Normally I would find the position of the last "/" then substring to this position. But google keeps falling back to sed. Any other commands I could use?

[AmbiguousOutlier]

#!/bin/bash

mypath=/mirror/folder/subfolder/file.txt

mkdir -p $(dirname "$mypath")

works

#!/bin/bash

mypath=/mirror/folder/sub folder/file.txt

mkdir -p $(dirname "$mypath")

doesn't work

It's these spaces in my file names, I'm struggling to handle them correctly.

[schragge]

Quote file names with spaces:

#!/bin/sh
file='/original/folder/sub folder/file.txt'
path=${file%/*}
newpath=/mirror/${path#/*/}
mkdir -p "$newpath"
cp "$file" "$newpath"

There are many ways to copy/move whole folders: rsync, cp -a, cpio, tar. See this post. It's about moving /home, but techniques described there are aplicable to copying/moving arbitrary directory subtrees.