Help needed with file sorting

[Merrattic]

Hmmmm, adding the -t seems to break things, sees the target file but wants it to be a target directory for moving ?

mv: target `dir1/dir2/my.pdf' is not a directory

when "pdf" should be the target directory (above dir1)

[steeldriver]

sorry I should have mentioned: the -t specifies the target, it's just an alternative syntax for the command i.e.

mv -t dir/ file

instead of

mv file dir/

If you just throw a -t in to the second form it will try to write dir to file - no what you want at all

[Merrattic]

This is agonisingly close

Here is my current line:

EXTS=("pdf"); for ext in "${EXTS[@]}"; do mkdir $ext; mv --backup=numbered **/*."$ext" "$ext"; done

To test, I have two files called my.pdf down in different sub directories, they both get moved but one gets overwritten.

This format does the same thing:

EXTS=("pdf"); for ext in "${EXTS[@]}"; do mkdir $ext; mv --backup=numbered **/*.${ext} $ext; done

[Vaphell]

add -v switch to make mv verbose and check what happens
you should get something like this:

`1/2/test.txt' -> `txt/test.txt'
`1/test.txt' -> `txt/test.txt' (backup: `txt/test.txt.~1~')
`test.txt' -> `txt/test.txt' (backup: `txt/test.txt.~2~')

i think that files ending with ~ may be hidden.

[Merrattic]

Thanks Vaphell, they were hidden, a quick CTRL-h revealed all. Everything worked as planned.

So how do I stop files with a ~ at the end being a hidden file? Is there something like chmod or do I need to do a rename on them?

[Vaphell]

renaming will be fine

$ rename -nv 's/(.+)[.]([^.]+)[.]~(.+)~/$1--$3.$2/' *~*~
test.txt.~1~ renamed as test--1.txt
test.txt.~2~ renamed as test--2.txt
test.txt.~3~ renamed as test--3.txt
test.txt.~4~ renamed as test--4.txt
test.txt.~5~ renamed as test--5.txt

-n is dry run, remove n to actually rename

[Merrattic]

Yes, seems no way of simply unhiding. Found a simple sed command to remove the last "~"

rename  's/~$//' *

just need to build that in

[Vaphell]

rename works with the same expressions (eg s/from/to/). In my example i cut main part of the name ($1), original extension ($2) and the number between ~ ~ ($3) and construct the new name off these parts.
Sed would require piping names one by one which makes the approach suck.

[Merrattic]

OK, ready to mark as solved Many thanks to everyone, especially Vaphell for guidance and support.

Here is my one liner, which needs to be placed in the top directory. One can add as many different file extensions as needed into the variable array EXTS. I have used txt and jpg as example extensions.

shopt -s globstar; EXTS=(txt jpg); for ext in "${EXTS[@]}"; do mkdir $ext; mv --backup=numbered **/*.${ext} $ext; rename 's/~$//' "./$ext/"*; done

If I break it down a bit with comments:

# allows sub directory matching

shopt -s globstar;


#sets the variable with extensions, from 1 to many

EXTS=(txt jpg);


#runs the code for each extension in the variable array EXTS

for ext in "${EXTS[@]}";


#do something

do


#make a directory with the same name as the extension

mkdir $ext;


#move files with the extension in all sub directories to the created top
#level directory of the same name.
# The **/* code which is enabled by the shopt -s globstar command
# allows matching with all sub directories
# The backup=numbered ensures any duplicates are renamed, so no files
#are overwritten 

mv --backup=numbered **/*.${ext} $ext;


# renames the duplicate files by removing the last character if it
# is a "~" in order to unhide the files. There are better ways of
# doing this ;)

rename 's/~$//' "./$ext/"*;


# all done after iterating through each extension type

done

[steeldriver]

Glad to hear you got it working

Just one nitpick - it's generally considered good shell programming practice to use lower case for your own variables - upper case (like EXTS) is reserved for system environment variables (if you use upper case there's a chance of overwriting a system variable - which can cause all kinds of hard-to-diagnose wierdness)