PDA

View Full Version : [SOLVED] tar + ssh: multiple files removing data structure



erotavlas
November 26th, 2013, 10:58 PM
Hi,

I have the following problem. I want to compress many files that are located in different paths i.e. /path1/file1, /path1/file2, /path2/file3 and so on on single file and I want to remove the folder structure. After this I want to put all in pipe with ssh command and then to decompress the archive in the server.
I'm using the following command


tar -jcf - /path1/file1 /path1/file2 /path2/file3 | ssh user@ip 'tar -C /path/ -jxf -'

it works only in part since it maintains the structure of the origin folder in the server.
How can I do?

Thank you

Bachstelze
November 26th, 2013, 11:25 PM
Try


tar -jcf - /path1/file1 /path1/file2 /path2/file3 | ssh user@ip 'tar --strip-components=1 -C /path/ -jxf -'

(The title is misleading, this has nothing to do with bash.)

erotavlas
December 1st, 2013, 06:09 PM
Ok, it works. Now if I want to reverse the direction of the operation. In the above example I can tar files and send them to a server. If I want to launch the command from my pc in the server and I want to copy file in my pc, how can I do? Is it possible to execute a remote tar and then pipe it with ssh and tar in my pc?
I can use scp with compression flag -C to do it but I don't know if I can do tar+ssh that is faster.


scp user@ip:/path_file1 user@ip:/path_file2 /home_path

Thank you for your help.

Lars Noodén
December 1st, 2013, 06:16 PM
You should be able to just reverse the situation. Output from ssh can get piped into the local tar.

erotavlas
December 1st, 2013, 09:56 PM
Ok, but how could I tar the files at origin in unique tar file and then send it via piping?

Lars Noodén
December 1st, 2013, 10:01 PM
You write to stdout as your file and then on the client side read from stdin.



ssh tar zcf - /some/path/ | ( cd /another/path; tar zxf -)


I should have mentioned that above, it's not obvious until you've seen it used in a few ways. Most programs will let you write to stdout or read from stdin using the shortcut - You can also sometimes use /dev/stdout or /dev/stdin. There are a few exceptions but most allow it.

erotavlas
December 1st, 2013, 10:20 PM
Ok, but how could I tar the files at origin in unique tar file and then send it via ssh?
Something like this


tar -jcf - /path1/file1 /path1/file2 /path2/file3 | ssh user@ip 'tar --strip-components=1 -C /path/ -jxf -'

but launched from my pc.

Lars Noodén
December 1st, 2013, 10:30 PM
In #6 above, you are tarring files on the remote machine and then extracting them on the local machine. Is that what you are trying to do? It's the opposite of what you have in #1.

erotavlas
December 2nd, 2013, 05:05 PM
Yes, I'm able to do it in one way as I specified in my first post and after the suggestion of Bachstelze I'm able to remove the data structure. Now I would like to know if it is possible to do this in the reverse way. That is by launching the command from a local machine to a remote server. The server should execute the tar of several file and pipes it via ssh to local machine that untar the output of ssh.

Lars Noodén
December 2nd, 2013, 06:11 PM
The server should execute the tar of several file and pipes it via ssh to local machine that untar the output of ssh.

That's what #6 above does.

You just move ssh in the formula:



ssh user@ip 'tar -jcf - /path1/file1 /path1/file2 /path2/file3' | tar --strip-components=1 -C /path/ -jxf -


Then the first tar is run on the remote machine and the second tar is run locally. The first makes the archive and sends it over stdout via the pipe to the second which reads it from stdin and extracts it.

erotavlas
December 3rd, 2013, 10:11 PM
Ok, it works. The --strip-components=n works well only if all the files have a path with the same deep. How can I do if the paths have different deep?

erotavlas
December 4th, 2013, 08:17 AM
That's what #6 above does.

You just move ssh in the formula:



ssh user@ip 'tar -jcf - /path1/file1 /path1/file2 /path2/file3' | tar --strip-components=1 -C /path/ -jxf -


Then the first tar is run on the remote machine and the second tar is run locally. The first makes the archive and sends it over stdout via the pipe to the second which reads it from stdin and extracts it.

Ok, it works. The --strip-components=n works well only if all the files have a path with the same deep. How can I do if the paths have different deep?

Lars Noodén
December 4th, 2013, 08:20 AM
I think for that you would need to run tar+ssh multiple times.

tar doesn't have conditional choices for its parameters so if you're wanting to have --strip-components strip different depths you'll need a separate instance of tar. And because the pipe is one dimensional, having only one target, you'll need another pipe to hit a different target. So all that means running tar+ssh or ssh+tar multiple times, once for each depth.

erotavlas
December 8th, 2013, 09:57 AM
Ok, thank you.