Results 1 to 7 of 7

Thread: How to convert .bz2 to .tar.bz2 using pipes? Create tar from standard input

  1. #1
    Join Date
    Feb 2007
    Location
    New York
    Beans
    894
    Distro
    Ubuntu 9.10 Karmic Koala

    How to convert .bz2 to .tar.bz2 using pipes? Create tar from standard input

    How do I convert a .bz2 archive of a single file into a .tar.bz2 archive? I can't decompress the entire thing and then recompress it, since the compression ratio is huge. I've tried every combination of "tar" "bunzip2" "|" ">" and "-" that I can think of, and it won't work.

    Also, how do I determine the uncompressed size of a .bz2 file?

    related

    tar manual only describes how to read a compressed file from standard input and decompress it to a real file.
    "Please remember to do things the Ubuntu way. There is always more than one solution to a problem, choose the one you think will be the easiest for the user. ... Try to think as a green user and choose the simplest solution." — Code of Conduct

  2. #2
    Join Date
    Apr 2006
    Location
    Timisoara, Romania
    Beans
    132
    Distro
    Ubuntu 11.10 Oneiric Ocelot

    Re: How to convert .bz2 to .tar.bz2 using pipes? Create tar from standard input

    Hi.
    Indeed, tar will not read from a pipe, while trying to create an archive (-c).

    This means that the following line will NOT work:
    bunzip2 -c archive.bz2 | tar -cf - | bzip2 > archive.tar.bz2
    I think this was what you wanted, but...
    Bogdan

  3. #3
    Join Date
    Feb 2007
    Location
    New York
    Beans
    894
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: How to convert .bz2 to .tar.bz2 using pipes? Create tar from standard input

    Pretty much, though tar will create a .tar.bz2 with the -j option, which simplifies it.

    The manual says "Starting with version 1.11.5, GNU tar uses standard input and standard output as the default device", but I don't quite understand.

    http://www.gnu.org/software/tar/manual/tar.html#SEC146
    "Please remember to do things the Ubuntu way. There is always more than one solution to a problem, choose the one you think will be the easiest for the user. ... Try to think as a green user and choose the simplest solution." — Code of Conduct

  4. #4
    Join Date
    Sep 2006
    Beans
    8,155
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: How to convert .bz2 to .tar.bz2 using pipes? Create tar from standard input

    I've tried a few permutations of bzip and tar as well.

    It looks like tar doesn't like to take a single file as data and you might be looking at a feature request for tar.

    The compression that tar does comes *after* making a tar file.
    tar has its own format so you may have to spend the time and uncompress the file and then recompress it.

  5. #5
    Join Date
    Feb 2007
    Location
    New York
    Beans
    894
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: How to convert .bz2 to .tar.bz2 using pipes? Create tar from standard input

    Quote Originally Posted by Lars Noodén View Post
    I've tried a few permutations of bzip and tar as well.

    It looks like tar doesn't like to take a single file as data and you might be looking at a feature request for tar.
    You can tar a single file. I definitely did that last night, though I don't remember which command does it.

    The compression that tar does comes *after* making a tar file.
    tar has its own format so you may have to spend the time and uncompress the file and then recompress it.
    Right. Tar doesn't do any compression at all, it's just a wrapper for (usually multiple) files and their metadata, which is why it wasn't used in this case, since the contents are a single file.

    It's not a matter of time; it's a matter of space. The compressed size is a few GB, but the uncompressed size is >100 GB, which I don't have space for.

    Since bzip acts on blocks, it should be possible to uncompress a block, pipe it to tar, pipe it to bzip, write to disk, read the next block, etc. If tar doesn't support compressing from standard input, maybe it can be tricked into it by using a named pipe or something?
    Last edited by Endolith; September 24th, 2009 at 04:49 PM.
    "Please remember to do things the Ubuntu way. There is always more than one solution to a problem, choose the one you think will be the easiest for the user. ... Try to think as a green user and choose the simplest solution." — Code of Conduct

  6. #6
    Join Date
    Sep 2006
    Beans
    8,155
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: How to convert .bz2 to .tar.bz2 using pipes? Create tar from standard input

    Quote Originally Posted by Endolith View Post
    You can tar a single file. I definitely did that last night, though I don't remember which command does it.

    Right. Tar doesn't do any compression at all, it's just a wrapper for (usually multiple) files and their metadata, which is why it wasn't used in this case, since the contents are a single file.

    It's not a matter of time; it's a matter of space. The compressed size is a few GB, but the uncompressed size is >100 GB, which I don't have space for.

    Since bzip acts on blocks, it should be possible to uncompress a block, pipe it to tar, pipe it to bzip, write to disk, read the next block, etc. If tar doesn't support compressing from standard input, maybe it can be tricked into it by using a named pipe or something?

    Yes you can tar a single file, but as everyone here notices not as a pipe.

    Named pipes where one of the things I tried in many permutations. No dice. There are other times, though, that named pipes can be useful.

  7. #7
    Join Date
    Feb 2007
    Location
    New York
    Beans
    894
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: How to convert .bz2 to .tar.bz2 using pipes? Create tar from standard input

    Is there a way to print the uncompressed size?
    "Please remember to do things the Ubuntu way. There is always more than one solution to a problem, choose the one you think will be the easiest for the user. ... Try to think as a green user and choose the simplest solution." — Code of Conduct

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •