Howto: Backup and restore your system!

[msrice]

My guess is you did something to your partition table while reinstalling... So the UUID got changed.

From the liveCD, check the UUID of your / partition and write it somewhere.
boot up like you would normally do, when grub pops up, press 'e' to edit the entry you want to boot and change the UUID that's in the boot command for the one you've got on paper, correct it accordingly and hit ctrl-x to boot.
It should boot fine to your restored installation.

If you're able to report back from your restored install, we will go to the next step. That is to make that change in UUID permanent by editing the grub files.

Hope that'll work...

Thanks, it did work. I edited the long strings as psusi suggested (Thanks to you, too.) I edited fstab also. I ran sudo update-grub, Now I get another entry "with Linux zero" which fails to boot.
What else do I need to do?
Thanks for your patience.

[El_Belgicano]

Thanks, it did work. I edited the long strings as psusi suggested (Thanks to you, too.) I edited fstab also. I ran sudo update-grub, Now I get another entry "with Linux zero" which fails to boot.
What else do I need to do?
Thanks for your patience.

If you have a working entry and that extra one, you may like Grub-Customizer (http://ubuntuforums.org/showthread.php?t=1664134). To remove annoying grub entries of all kind, setting the names you want the others to have and of course all the usual warnings apply: have it all backed up before starting tweaking (but that seems to be done already) and do not change until you know what it does. ;-)

[spicysomtam]

I wanted to downgrade from Ubuntu 11.04 back to 10.10 (11.04 gives me desktop freezes and 10.10 is very stable). I will upgrade again in a month or two when the bods at Ubuntu have fixed issues :)

Anyway I had used a backup script I wrote many, many years ago to back it up to my USB hard disk. This uses tar. Nice and simple. I don't use LVM or bother with multiple filesystems; I just stick everything (apart from data) on /. This makes backup and recovery simple. I like things simple.

Restore did not work. I boot off a Ubuntu 10.10 live cd and restore that way. Here are my steps:


# mkfs.ext4 /dev/sda3
# mount /dev/sda3 /mnt
# cd /mnt
# tar -xzf /media/USB\ 500GB/Backups/Linux/ub1010/Laptop9/04\ May\ 2011\ 14-08/root.tgz
# blkid /dev/sda3
# vi etc/fstab # update uuid for /
# vi boot/grub/grub.cfg # Do a global search and replace for the uuid
# grub-install --root-directory=/mnt /dev/sda
# cd -
# umount /mnt
# umount /media/USB\ 500GB
# reboot


The procedure is fine; seems like the filesystem I backed up has a corruption. I have two issues:


Could not update ICEauthority file /var/lib/gdm/.ICEauthority

There is a problem with the configuration server. /usr/lib/libgconf2-4/gconf-sanity-check-2


Firstly I discover gdm files are owned by hplip. Seems like the uids got swapped around in /etc/passwd. I swap them around and this fixes the issues.

Then pulseaudio and wifi are not starting up in session startup. I find file /lib/dbus-1.0/dbus-daemon-launch-helper is group owned by lpadmin. Dbus authentication is broken. Another similar issue with gids; lpadmin and messagebus gids are swapped around in /etc/group. I swap them back, and also update the gid for messagebus in /etc/group.

Its worth repeating the restore and doing the uid/guid fixes before first boot so that no odd files end up on the fs.

Now I am back to normal operation.

This corruption seems to have occurred Ubuntu 10.04 to 10.10 upgrade time. I recon something on the printing system does not upgrade correctly (something from HP?). I recon the shadow files were correct but the text files were wrong and somehow the system still worked correctly (until I did a backup/restore).

Hope this will be useful to someone; hence posting and sharing.

It goes to prove that the backup you make can be of a corrupt fs, even though your system works fine. There may be additional work on restore!!!

It also occurs to me that backups should be taken when hardly anything is running. In the old days you could do this by changing to single user mode, but in ubuntu, you still have a desktop. Thus I suggest booting into recovery and spinning up the root ascii shell and doing backups this way. Might be worth checking not much is running before doing the backup. IMHO you cannot get a good backup while alot of things are running.

[El_Belgicano]

<snip>
# mkfs.ext4 /dev/sda3
# mount /dev/sda3 /mnt
# cd /mnt
# tar -xzf /media/USB\ 500GB/Backups/Linux/ub1010/Laptop9/04\ May\ 2011\ 14-08/root.tgz
# blkid /dev/sda3
# vi etc/fstab # update uuid for /
# vi boot/grub/grub.cfg # Do a global search and replace for the uuid
# grub-install --root-directory=/mnt /dev/sda
# cd -
# umount /mnt
# umount /media/USB\ 500GB
# reboot
<snip>

I'd say you should add a "-p" flag to your extraction command to preserve the permissions. (MAN TAR (http://unixhelp.ed.ac.uk/CGI/man-cgi?tar))
So the command would be:
tar -xpzf /media/USB\ 500GB/Backups/Linux/ub1010/Laptop9/04\ May\ 2011\ 14-08/root.tgz


Hope that helps your future restores...

[psusi]

I'd say you should add a "-p" flag to your extraction command to preserve the permissions.

That is enabled by default when you run it as root.

[dwlamb]

I used the method that starts this thread to back-up my system. To limit output to derive simply any error/warning messages, output was piped to '>/dev/null'. Below is a list of messages I received. Does any of these pose a problem in the future to restoring the system?

tar: Removing leading `/' from member names
tar: Removing leading `/' from hard link targets
tar: /tmp/ssh-KIeEtB1374/agent.1374: socket ignored
tar: /tmp/orbit-gdm/linc-48d-0-3f26bf054e439: socket ignored
tar: /tmp/.ICE-unix/1374: socket ignored
tar: /tmp/orbit-daniel/linc-5d1-0-19630c20689f6: socket ignoredA whole slew of files under the path stated above with linc-<alphanumeric file names>
tar: /tmp/.X11-unix/X0: socket ignored
tar: /tmp/pulse-MWTNYr3tBJaJ/native: socket ignored
tar: /tmp/keyring-IJHByM/pkcs11: socket ignored
tar: /tmp/keyring-IJHByM/ssh: socket ignored
tar: /tmp/keyring-IJHByM/control: socket ignored
tar: /tmp/.esd-1000/socket: socket ignored
tar: /tmp/.winbindd/pipe: socket ignored
tar: /home/daniel/.gvfs: Cannot stat: Permission denied
tar: /var/run/cups/cups.sock: socket ignored
tar: /var/run/mysqld/mysqld.sock: socket ignored
tar: /var/run/acpid.socket: socket ignored
tar: /var/run/postgresql/.s.PGSQL.5432: socket ignored
tar: /var/run/avahi-daemon/socket: socket ignored
tar: /var/run/dbus/system_bus_socket: socket ignored
tar: /var/run/samba/winbindd_privileged/pipe: socket ignored
tar: /dev/log: socket ignored

Thanks for such a simple straight-forward tutorial.

[psusi]

You shouldn't be backing up /tmp or /dev. Either add them to the --exclude list, or use --one-file-system.

[j.jansen]

this has worked for me a longer time.
TILL I started with ubuntu 64 bits.
I used TAR in my scripts and everything was good.
Now with 64 bit ubuntu I get an error when I start the restored system
error has to do with ; /var/lib/gdm/..

Alternative I connected both harddisks and used cp.
cp -rvdbP /source/* /target -> same problems.

Analyzing (booting a third partition) I found:
/source/var/lib/gdm belongs to statd in the group admin with rights 755
the restored:
/target/var/lib/gdm belongs to root in the group root with rights 750
- also the files below gdm have changed attributes.
- even after changing the attributes the restored system will not run correct, so there is more effected as only this directory, but have not found which ones).
- booting from source gives:/var/lib/gdm belongs to gdm in the group gdm



Question: What can be a reason that source and target (backup/restore with TAR) have changed attributes.
What could I do to get this working?
-which command can compare /source and /target - not only the file itself but its attributes ?

[psusi]

Everything in /var/lib/gdm should be owned by gdm:gdm. If the gdm user does not have the same user ID that you had before, then the file will be owned by whoever has the old ID.

[Deviant06]

I think this is what I need but I want to double check.
This is my sis's pc. I installed XP Pro for her and then used wubi to install the 11.04 cd I had burned.
I updated to Gnome 3 and modified it and I don't want to go through al that again, not to mention remembering where to find everything I did on the web. I need to upgrade her to Win 7 because unfortunately some of the apps she needs for school require it. And since I installed Ubuntu kind of like an application from windows, I don't think there is an actual ext 3 or 4 partition, am I right?
I don't know if this actually matters with your approach since there were no drive lettering involved. But I just wanted to make sure that if I backup using your commands, reformat, install Win 7 ( it does need to be first, at least on her pc)
then install ubuntu, then run your commands, it will return Ubuntu to my previous version?

Please let me know if this is correct, and if not, how should i go about doing this.
Thanks for all the help.

(all os's I'm using are x86.
and I can't just upgrade xp to 7 unfortunately)

[psusi]

You can back up the WUBI install but can not restore it as a WUBI install. When restoring, you will need to create the actual ext3/4 partition and extract to there.

[kleovoulinos]

Excellent Guide there, helped me a lot!

[Venus2]

This was very informative but I am only interested in restoring my default graphics drivers, not everything. After I upgraded to 10.04 during the configuration process I opted not to use proprietary drivers and to use open source drivers instead. Well, I have changed my mind. But the proprietary drivers will no longer populate in the System > Admin > Hardware Drivers. If I had known opting for an open source driver would somehow eliminate the proprietary drivers on my system, obviously I would not have made that choice. Now I want to use the proprietary drivers and now they are no longer available. I have searched these forums for some simple restore default commands, but I'm so new I really cannot tell if the advice given is generic or situation and spec specific.

For what its worth, mine is an HP Pavillion dv 6000 with an Intel GMA 950 graphics card, and I want to use the Intel hardware I had. I can simply just go back to disc and start at 8.04 and build from the beginning, but would rather have a simple command in the terminal which would restore the default hardware and ONLY the default graphics driver hardware more specifically. I'm still in the process of learning and understanding the commands and so far I've found nothing in the forums I'm willing to actually try for fear it will really mess things up even worse. I will continue to search cheat sheets and so forth until I can find a very easy to understand explanation of any commands related to restoring the default drivers.

For the daring, kind, advanced user, I'm looking for an actual breakdown of the meaning. For example, if the command is sudo resdefault Intel - driver, I can pretty well understand that I'm commanding it to restore default Intel drivers. Right now the only thing I really understand is apt-get update so I am a very new newby.

[Deviant06]

You can back up the WUBI install but can not restore it as a WUBI install. When restoring, you will need to create the actual ext3/4 partition and extract to there.

thanks psusi but I have a few questions..

When I tried running the commands (in ubuntu) on the first page, it even backed up my Windows folders. Which of course I don't need it to do. Is there any guides or any more information on exactly how I need to go about this? would using a backup program be easier?
Or do I just go into my C:\Ubuntu folder in windows and save the entire folder? or just the install folder with the fuze hidden file?
and how do I go about reloading it?


Thanks in advance

[geazzy]

thanks for this tutorial :)

[psusi]

thanks psusi but I have a few questions..

When I tried running the commands (in ubuntu) on the first page, it even backed up my Windows folders. Which of course I don't need it to do. Is there any guides or any more information on exactly how I need to go about this? would using a backup program be easier?
Or do I just go into my C:\Ubuntu folder in windows and save the entire folder? or just the install folder with the fuze hidden file?
and how do I go about reloading it?


Thanks in advance

You either need to use the --one-file-system switch that I advocate, or make sure that /host is in the list of --excludes.

[Deviant06]

When searching google a bit more, i found two other ways.
The actual Wubi wiki says to simply backup root.disk,
then i can reformat, instal win 7, instal 11.04 again,
and when all is said and done, copy the root.disk back.
I only have it on a 20gb so its doable for me.

Also---


I read http://ozansafi.wordpress.com/2010/04/11/how-to-backuprestore-a-wubi-installation/

which says to make a backup of the /ubuntu folder on your harddisk along with wubildr and wubildr.mbr files in the root folder, copy down the boot config data in my boot.ini.
and the only possible error I'll get is the drive uid, which I can update manually in menu.lst.

Then theres the commands in this post as you said just make sure /host is in the excludes.

Obviously the directions on the Wubi Wiki seem easier, but not as efficient.

what do you think about the 2 other ways I found, and will they work?
One better than the other?

(I'm trying to keep in mind I'm reformating this XP/ wubi 11.04 installation and reinstalling win7 with wubi ubuntu 11.04, which might have issues just copying a boot.ini file or something, so I'm leaning to the backup of root.disk)


(thx psusi, and sry bout all the questions, just want to be comfortable about doing this, and I've only started using Linux 6 months ago)

[psusi]

When searching google a bit more, i found two other ways.
The actual Wubi wiki says to simply backup root.disk,
then i can reformat, instal win 7, instal 11.04 again,
and when all is said and done, copy the root.disk back.
I only have it on a 20gb so its doable for me.


The down side to that is that it requires windows to backup and restore, and you are also backing up free all of the free space in the Ubuntu filesystem. Then you end up having to reinstall Ubuntu anyway in order to restore.

Personally I think that WUBI is nothing but trouble and should be avoided. I suggest that you just tar everything up and restore it on its own partition and forget about WUBI.

[Deviant06]

The down side to that is that it requires windows to backup and restore, and you are also backing up free all of the free space in the Ubuntu filesystem. Then you end up having to reinstall Ubuntu anyway in order to restore.

Personally I think that WUBI is nothing but trouble and should be avoided. I suggest that you just tar everything up and restore it on its own partition and forget about WUBI.

I'm sorry but your first paragraph didn't make sense..? requires windows to backup and restore what..? and backing up free all of the free space...? I'm sorry I don't understand what you're tryin to say.

The fact is that I am going to reformat and WILL be using wubi to install ubuntu.
This is my little sister's pc and the benefits of being able to uninstal Ubuntu at any time without technical knowledge and not having to mess with partitions is needed for her. The whole point of having linux on here is for her to broaden her computer skills, and to give her a head start. She will be getting OSX soon and the similarities between linux and osx will help her jump in easier when she actually does get her new laptop.
So I WILL be using Wubi for her.
(on MY pc, I have separate partitions, just as you advocate, but this is not mine)

So could you please clarify what you were saying.. and betwen the other options... which one is best..
Can I simply backup the root.disk file. format the entire pc, instal win7, wubi ubuntu, and then copy and overwrite the root.disk file back into my windows ubuntu folder?
Is that how I woulld go about doing it?
or do I need to actually copy the entire ubuntu folder as well as the wubildr.mbr & wubildr files as outline in that link I posted..?


(Remember I HAVE to do a complete format of my drive to instal Windows 7 on it, it will not allow me to upgrade, but I have an external to backup the files I need)


I need to do this asap, so I'm really just waiting on what you say.

I appreciate your help psusi

[psusi]

I'm sorry but your first paragraph didn't make sense..? requires windows to backup and restore what..? and backing up free all of the free space...? I'm sorry I don't understand what you're tryin to say.

Since you are backing up and restoring the file inside Windows, you obviously must have Windows. The image file is the same size, no matter if you are only using 1% or 100% of it, so you will end up backing up a lot of unused free space assuming your Ubuntu filesystem is not 100% full.

Can I simply backup the root.disk file. format the entire pc, instal win7, wubi ubuntu, and then copy and overwrite the root.disk file back into my windows ubuntu folder?
Is that how I woulld go about doing it?
or do I need to actually copy the entire ubuntu folder as well as the wubildr.mbr & wubildr files as outline in that link I posted..?

You probably will also need to restore the original wubi config files.

[Deviant06]

Even if I backup the entire 20gb file with unused space inside.. I will be able to use it after the restore correct? itll just take longer to backup..
Thats fine if thats what you mean.

And was I right about how to restore it?

After I've reinstalled windows... run wubi and instal ubuntu.. when all thats done..
just load windows.. and copy my files over the current ones? then reboot into ubuntu?
that should give me the exact same ubuntu setup I currently have?

and arent the wubi config files the wubildr.mbr & wubildr file in C:\ubuntu?

[psusi]

Even if I backup the entire 20gb file with unused space inside.. I will be able to use it after the restore correct? itll just take longer to backup..
Thats fine if thats what you mean.

Yes.

And was I right about how to restore it?

I think so.

After I've reinstalled windows... run wubi and instal ubuntu.. when all thats done..
just load windows.. and copy my files over the current ones? then reboot into ubuntu?
that should give me the exact same ubuntu setup I currently have?

and arent the wubi config files the wubildr.mbr & wubildr file in C:\ubuntu?

I think so.

[Deviant06]

Thats good enough for me then. Thanks man.

[psgill]

I'm trying to learn how to image or create a backup server.
So I decided to use the "tar" method for backup.

I have succesfully backed up all the directories from "/" except the ones that need to be excluded such as sys proc etc. size approx is 500MB.

I have the .tgz file on another server so I guess I can just use wget to download it on the new system using LIVE CD.

So I boot up the new server with Live CD and become the root user.

The new system has a 80GB hdd. Currently, there's a hdd with 80GB of unallocated space so it has no partitions whatsoever.

I have tried to follow many different steps and guides to get the backup server up but each time I end up with different errors. Sometimes it says kernel is not loaded. At other times it says Device UID doesn't exist; ofcourse the hardware is different.

Can someone please help me with a short step by step guide which tells me the commands to execute from the LIVE CD terminal which help me setup the partitions required and the GRUB set-up so that the back-up server can boot up.

I will really appreciate your help.

[Deviant06]

I'm trying to learn how to image or create a backup server.
So I decided to use the "tar" method for backup.

I have succesfully backed up all the directories from "/" except the ones that need to be excluded such as sys proc etc. size approx is 500MB.

I have the .tgz file on another server so I guess I can just use wget to download it on the new system using LIVE CD.

So I boot up the new server with Live CD and become the root user.

The new system has a 80GB hdd. Currently, there's a hdd with 80GB of unallocated space so it has no partitions whatsoever.

I have tried to follow many different steps and guides to get the backup server up but each time I end up with different errors. Sometimes it says kernel is not loaded. At other times it says Device UID doesn't exist; ofcourse the hardware is different.

Can someone please help me with a short step by step guide which tells me the commands to execute from the LIVE CD terminal which help me setup the partitions required and the GRUB set-up so that the back-up server can boot up.

I will really appreciate your help.


I don't know if this will help you, but as far as the Device UID goes, its talking specifically about the HD name. goto this site and checkout his instructions. It's pretty simple to rename the UID string to see the new HDD.
http://ozansafi.wordpress.com/2010/04/11/how-to-backuprestore-a-wubi-installation/
goodluck

[psgill]

Thanks.

[ricardus1867]

Thanks for the guide! I'm going to use this to backup my server. :-)

My two cents: If you use 7zip (or any other program with LZMA compression), you'll save an important amount of space.

To install 7zip, run

apt-get install p7zip
It's not natively supported by tar, so you'd have to 7zip it separately:

tar cvpf backup.tar --exclude=/proc --exclude=/lost+found --exclude=/backup.tar --exclude=/mnt --exclude=/sys /
7za a backup.tar.7z backup.tar
rm -f backup.tar

Results:

Uncompressed tarball: 729 MiB
Compressed with gzip: 298 MiB
Compressed with bzip2: 278 MiB
Compressed with 7zip: 229 MiB

That's only three quarters of the gzip compressed tarball!

[rsperson]

Tutorials & Tips (http://ubuntuforums.org/forumdisplay.php?f=100)I searched this thread for netbook - no joy

I would love to do this for a netbook to a little linux box. I have a partially hacked D-link DNS-323 to do more then serve for files. I might have ftp running - but will probably get it done first. My Netbook is a HP Mini 1000.
I have spent a long time getting it just right. I have ONE lingering issue that is out of my control (Moonlight for Silverlight). The app is not under my control and neither is the Moonlight project.

I want to try a Hackintosh OS X 10.5.x because it SHOULD be able to run Silverlight. If this is a bust - I want to restore the full netbook on Ubunto without re-doing the weeks of hacking I needed to get it just right.

Is there any hope?

Thanks.

[psusi]

What?

[rsperson]

Lets try again.

I have a netbook - HP Mini 1000
This has NO CD or DVD drive.

Thus no local restore drive.

I have a limited Linux machine - A D-link DNS-323 NAS that runs a very hacked linux to get more functions. FTP server eventually, and Rsync - soon.

Can I create an image or full copy of my Netbook system on my Dlink that I can restore - when I don't have a CD/DVD drive.

Tutorials & Tips (http://ubuntuforums.org/forumdisplay.php?f=100)I searched this thread for netbook - no joy

I would love to do this for a netbook to a little linux box. I have a partially hacked D-link DNS-323 to do more then serve for files. I might have ftp running - but will probably get it done first. My Netbook is a HP Mini 1000.
I have spent a long time getting it just right. I have ONE lingering issue that is out of my control (Moonlight for Silverlight). The app is not under my control and neither is the Moonlight project.

I want to try a Hackintosh OS X 10.5.x because it SHOULD be able to run Silverlight. If this is a bust - I want to restore the full netbook on Ubunto without re-doing the weeks of hacking I needed to get it just right.

Is there any hope?

Thanks.

[jal]

@rsperson


If your problem is limited disk space, then there are ways to send the tar output over the network to another machine.

http://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/

[Techjacker]

I've created a bash script that uses rsync to backup on to an internal hard drive. The script is set to run when the computer shuts down at night so it is 100% automated.

The script does the following:

1:1 backup of your media drive
incremental backup of system drive
backup of all packages installed
optionally a backup of your VPS (although this requires some additional set up)


I've posted the script on gitorious - feel free to contribute if you think you can improve it:
https://gitorious.org/backup-forget#more

Instructions on how to use it are also in the repo or can be viewed here:
http://andrewgriffithsonline.com/software/bash-backup-script/

[Jamesi]

Thank you for helping i needed to back up my ubuntu!

[Jonny87]

Love this guide. have been using this method for quite some time now. I have a new question though. If one is to backup "/" then restore it to a new HDD/partition, I'm guessing it will come up with conflict errors with the likes of the UUID's. In fact I think that may have been the cause for an error I had a while ago when restoring a backup. My question is, is there a way to reset this or some get around this? I realise that one could manually edit the fstab after the restore, but is that all that would need to be fixed?

[psusi]

I realise that one could manually edit the fstab after the restore, but is that all that would need to be fixed?

That and reinstalling grub:

sudo dpkg-reconfigure grub-pc

[Jonny87]

That and reinstalling grub:

sudo dpkg-reconfigure grub-pc

Thanks, would have realised that during the reinstall. If a system isn't bootable would that method of reinstalling grub work from a live cd by chroot to the actual installed / partition?

[psusi]

Thanks, would have realised that during the reinstall. If a system isn't bootable would that method of reinstalling grub work from a live cd by chroot to the actual installed / partition?

Yes, but you have to bind /sys, /proc, and /dev into the chroot:

for f in sys proc dev ; do mount --bind /$f /mnt/$f ; done
chroot /mnt

[Panties]

Hi, and welcome to the Heliode guide to successful backing-up and restoring of a Linux system!

Most of you have probably used Windows before you started using Ubuntu. During that time you might have needed to backup and restore your system. For Windows you would need proprietary software for which you would have to reboot your machine and boot into a special environment in which you could perform the backing-up/restoring (programs like Norton Ghost).
During that time you might have wondered why it wasn't possible to just add the whole c:\ to a big zip-file. This is impossible because in Windows, there are lots of files you can't copy or overwrite while they are being used, and therefore you needed specialized software to handle this.

Well, I'm here to tell you that those things, just like rebooting (http://www.ubuntuforums.org/showthread.php?t=34629), are Windows CrazyThings (tm). There's no need to use programs like Ghost to create backups of your Ubuntu system (or any Linux system, for that matter). In fact; using Ghost might be a very bad idea if you are using anything but ext2. Ext3, the default Ubuntu partition, is seen by Ghost as a damaged ext2 partition and does a very good job at screwing up your data.

1: Backing-up

"What should I use to backup my system then?" might you ask. Easy; the same thing you use to backup/compress everything else; TAR. Unlike Windows, Linux doesn't restrict root access to anything, so you can just throw every single file on a partition in a TAR file!

To do this, become root with

sudo su
and go to the root of your filesystem (we use this in our example, but you can go anywhere you want your backup to end up, including remote or removable drives.)

cd /
Now, below is the full command I would use to make a backup of my system:


tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /Now, lets explain this a little bit.
The 'tar' part is, obviously, the program we're going to use.

'cvpfz' are the options we give to tar, like 'create archive' (obviously),
'preserve permissions'(to keep the same permissions on everything the same), and 'gzip' to keep the size down.

Next, the name the archive is going to get. backup.tgz in our example.

Next comes the root of the directory we want to backup. Since we want to backup everything; /

Now come the directories we want to exclude. We don't want to backup everything since some dirs aren't very useful to include. Also make sure you don't include the file itself, or else you'll get weird results.
You might also not want to include the /mnt folder if you have other partitions mounted there or you'll end up backing those up too. Also make sure you don't have anything mounted in /media (i.e. don't have any cd's or removable media mounted). Either that or exclude /media.

EDIT : kvidell suggests below we also exclude the /dev directory. I have other evidence that says it is very unwise to do so though.

Well, if the command agrees with you, hit enter (or return, whatever) and sit back&relax. This might take a while.

Afterwards you'll have a file called backup.tgz in the root of your filessytem, which is probably pretty large. Now you can burn it to DVD or move it to another machine, whatever you like!

EDIT2:
At the end of the process you might get a message along the lines of 'tar: Error exit delayed from previous errors' or something, but in most cases you can just ignore that.

Alternatively, you can use Bzip2 to compress your backup. This means higher compression but lower speed. If compression is important to you, just substitute
the 'z' in the command with 'j', and give the backup the right extension.
That would make the command look like this:


tar cvpjf backup.tar.bz2 --exclude=/proc --exclude=/lost+found --exclude=/backup.tar.bz2 --exclude=/mnt --exclude=/sys /
2: Restoring

Warning: Please, for goodness sake, be careful here. If you don't understand what you are doing here you might end up overwriting stuff that is important to you, so please take care!

Well, we'll just continue with our example from the previous chapter; the file backup.tgz in the root of the partition.

Once again, make sure you are root and that you and the backup file are in the root of the filesystem.

One of the beautiful things of Linux is that This'll work even on a running system; no need to screw around with boot-cd's or anything. Of course, if you've rendered your system unbootable you might have no choice but to use a live-cd, but the results are the same. You can even remove every single file of a Linux system while it is running with one command. I'm not giving you that command though! ;-)

Well, back on-topic.
This is the command that I would use:


tar xvpfz backup.tgz -C /
Or if you used bz2;


tar xvpfj backup.tar.bz2 -C /
WARNING: this will overwrite every single file on your partition with the one in the archive!

Just hit enter/return/your brother/whatever and watch the fireworks. Again, this might take a while. When it is done, you have a fully restored Ubuntu system! Just make sure that, before you do anything else, you re-create the directories you excluded:

mkdir proc
mkdir lost+found
mkdir mnt
mkdir sys
etc...
And when you reboot, everything should be the way it was when you made the backup!

2.1: GRUB restore
Now, if you want to move your system to a new harddisk or if you did something nasty to your GRUB (like, say, install Windows), You'll also need to reinstall GRUB.
There are several very good howto's on how to do that here on this forum, so i'm not going to reinvent the wheel. Instead, take a look here:

http://www.ubuntuforums.org/showthread.php?t=24113&highlight=grub+restore

There are a couple of methods proposed. I personally recommend the second one, posted by remmelt, since that has always worked for me.


Well that's it! I hope it was helpful!
As always, any feedback is appreciated!

About this... Let's assumed... I'm running Ubuntu 5.10 Server.

that I have run command below, to backup:-
tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /


Now, I created a folder say... "Testing" in /root & /home. I also created a "deleteme.txt" file in /boot.
I also upgraded to the latest Kernel, Running Ubuntu8.10.

And now, time to restore back to Ubuntu 5.10 Server

tar xvpfz backup.tgz -C /


So, My concerns are:
a. Does this method work?
b. What will happen to "deleteme.txt" and "Testing" folder that I created? Will it be removed? or stay?
c. Is this thing, really similar to the MAC's Time Machine? or Window's 3rd party application called "http://www.comodo.com/home/data-storage-encryption/data-recovery.php" Comodo Time Machine? (because I used both of it before)
d. Any drawback/feedback about this method, that I should know?

[chinamike]

This looked like a good thing to try, although I suppose I messed up on one of the procedures. When you gave examples of exclude, I took it for face value and did it as shown. I dropped: in tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys --exclude=/Downloads --exclude=/Videos --exclude=/Pictures --exclude=/Music /

BUT...I did not REALIZE I had to add the path /home/[my user name]/ followed by Downloads, or Music, or Pictures! it went ahead and COPIED all that stuff....arrrrrggggghhhh!

I'm now looking for the "kill process" command within a terminal to start over...I have WAY TOO many .avi movie files in Videos to let this keep running! :P

I'll repost when I do it right!

All the best and thank you for nice description and code

China Mike

[chinamike]

OK! Back! it worked well THIS time, after adding the required /home/[username]/Video and other related ones I wanted to exclude. The tgz came in at 1.3 GB. I had recently reinstalled my system making sure my home folder was on a separate partition. I had it that way on another PC, which will make the reinstall nice. This way I'll have all those nice little added proggies instead of just a clean Lucid install. Thanks again!

China

[witya]

will this work if I boot from live usb and restore (untar) everything to the new hdd?

I mean, will I be able to boot exactly the same OS, with same config and users?

[davecraford]

Excellent guide this very informative really it helps

[Jonny87]

will this work if I boot from live usb and restore (untar) everything to the new hdd?

I mean, will I be able to boot exactly the same OS, with same config and users?

Yes it will but as mentioned a bit earlier in the thread, it will require some adjustments such as editing the fstab and updating grub so that it reads the new harddrive as being the root partition. Same applies if you restore to the same HDD but have reformatted the HDD or the partition.

[xx58]

:rolleyes:If you have single operating system installed like Ubuntu 11.04, then Remastersys will do job for you. Remastersys have full back up or Dist back up, where Full back up are customer back up for your computer and Dist back up for any computer.
Clonezilla - good for dual boot back up, exellent for separetly for Ubuntu or Windows operating back up.
Using Remastersys and Clonezilla and they both are exellent.
Good luck

[Spitfire1900]

I did a test of this in a VM enviroment simulating a failed drive.

Created a tar.bz2 image on a share using "tar -cvp --exclude=/proc --exclude=/lost+found --exclude=/home/eot-it/BackupPortal --exclude=/mnt --exclude=/sys / | bzip2 > /home/eot-it/BackupPortal/Backup.bz2

Tars it all up on the network share successfully.

Create new VM with new hard drive, installed Ubuntu 10.04 Server, logged in, mounted share, create root passwd, login as root, and run "tar -xjvpf /home/eot-it/BackupPortal/Backup.bz2 -C /" takes a long time and finishes with an error, but appear to have worked otherwise.

Reboot system to get this "Gave up waiting for boot device."

Any fixes?

[Spitfire1900]

Yes it will but as mentioned a bit earlier in the thread, it will require some adjustments such as editing the fstab and updating grub so that it reads the new harddrive as being the root partition. Same applies if you restore to the same HDD but have reformatted the HDD or the partition.
where is this?

[Spitfire1900]

I did a test of this in a VM enviroment simulating a failed drive.

Created a tar.bz2 image on a share using "tar -cvp --exclude=/proc --exclude=/lost+found --exclude=/home/eot-it/BackupPortal --exclude=/mnt --exclude=/sys / | bzip2 > /home/eot-it/BackupPortal/Backup.bz2

Tars it all up on the network share successfully.

Create new VM with new hard drive, installed Ubuntu 10.04 Server, logged in, mounted share, create root passwd, login as root, and run "tar -xjvpf /home/eot-it/BackupPortal/Backup.bz2 -C /" takes a long time and finishes with an error, but appear to have worked otherwise.

Reboot system to get this "Gave up waiting for boot device."

Any fixes?
I think I might have fixed my own problem, but it will take awhile to determine. When tarring initially make sure you include --exclude=/dev and --exclude=/etc/fstab. I'll come back with results as soon as possible.

[Spitfire1900]

The question is, if I intend on moving to a new hard drive, what am I not supposed to copy?

[pjmlmas]

offline backup?

MNT_BACKUP_PART=/media/somemountedparttobackup;
tar -cvpzf /media/exthd/tar-bu-test/backup.tar.gz --directory=$MNT_BACKUP_PART . --exclude=$MNT_BACKUP_PART/backup.tar.gz --exclude=$MNT_BACKUP_PART/proc --exclude=$MNT_BACKUP_PART/lost+found --exclude=$MNT_BACKUP_PART/sys --exclude=$MNT_BACKUP_PART/mnt --exclude=$MNT_BACKUP_PART/media --exclude=$MNT_BACKUP_PART/dev


i am hoping to use same script offline. but getting the right directory path in (or maybe s/b out) seems elusive.

Above is latest attempt during tarring, perhaps some trick during untaring?

[j*tech]

Will tar backup/ restore to another separate system? I would like to "transfer" my current 11.04 setup to my standalone system (with fresh install of 11.04).

[newb85]

@j*tech,

I'm not sure what you mean by wanting to transfer your current setup on a fresh install. Aside from the physical machine, what do you hope will be different? Do you want to keep all the programs you have installed? All your settings?

If you know what it is you want to keep, then yes, it is possible to do it with tar backup/restore. You simply (or perhaps not so simply) have to instruct it to tar that and only that. Then, once the new machine has 11.04 loaded, untar what you have there.

[dcsoldschool53]

Thanks. This tutorial works fine, except I had to change the command around to send it to another drive, because I had received a warning message that my hard drive was full. Other than that, it had worked perfectly.

[jvance492]

A great guide and a very successful way to backup server ubuntu without imaging (which is bad)

I wanted to put in my experience with this:

I used a different machine with different hardware and of course the hard drives GUID's didnt match up. so upon restoring my files, I broke the install of server ubuntu every time. Even going through restoring grub was not successful.

So I took the new machine, did a fresh install of server ubuntu which generated the correct GUID of the hardware. I then used a LIVE CD to get the /boot/grub/grub.cfg file and the /etc/fstab file

I ran my restore commands for the backup.tgz which of course broke my system.

I loaded up a LIVE CD again, replaced those files and the sytem booted finally! its been 3 days of installing server but i finally got it.

I also want to menchion that I used the same exclude command when restoring it to skip the boot folder.

I wouldn't have had any of these problems if I was using the actual server with the same hard ware, but in case you are making a backup machine the method to restore grub didn't work for me.

Also since im using 11.04 its grub2 i believe.

Anyway THANK YOU for this guide. I have a PDF of the guide because its so useful.

Cheers

[LuigiAntoniol]

When it comes to backing up my computer, I'm not too comfortable with the command line, but that's just me.

After trying so many different backup systems/apps, I finally settled on Lucky Backup.

It has all the functionality I need for backing up my system to an external USB backup drive; settings, documents, the works.

[painguin]

Guys I've screwed up my Ubuntu 11.04 (x64) install and cannot login to the desktop. I can login under the shell but in the desktop I am getting an error at the top right of the screen stating "The Gnome Power Management Console has not been configured. Please contact your Network Administrator". If I try to login it loops me back to the login screen.

If I use this method of backing things up and if I restore in a fresh Ubuntu installation will my problem be solved or will the backup keep my "bad settings" and carry them over to the new installation?

I have a thread posted with the problems I'm having. It is here: http://ubuntuforums.org/showthread.php?t=1850416

Thanks in advance!

PAinguIN

[newb85]

Guys I've screwed up my Ubuntu 11.04 (x64) install and cannot login to the desktop. I can login under the shell but in the desktop I am getting an error at the top right of the screen stating "The Gnome Power Management Console has not been configured. Please contact your Network Administrator". If I try to login it loops me back to the login screen.

If I use this method of backing things up and if I restore in a fresh Ubuntu installation will my problem be solved or will the backup keep my "bad settings" and carry them over to the new installation?

I have a thread posted with the problems I'm having. It is here: http://ubuntuforums.org/showthread.php?t=1850416

Thanks in advance!

PAinguIN

That depends on whether your last backup was before or after these "bad settings" first appeared on your system (which may have been earlier than their symptoms).

If restoring does not solve it, and you know what specific files contain the "bad settings", you could try removing those files from your .tar file before restoring. My guess, though fairly uneducated, is that your specific error message is because a Power Management settings file is already missing from your system. If that's the case, restoring over a fresh install will probably fix the problem.

[mikeygstl]

Out of curiosity, why doesn't anyone use rsync? Much faster, easier to keep up to date, and if compression is the goal, one can always mount a compressed filesystem (check out fusecompress). For instance, the command I use to back up my entire box is:

sudo rsync -av --exclude=/proc --exclude=/dev --exclude=/sys --exclude=/mnt --exclude=lost+found --exclude=/tmp / /mnt/backup/system/

Nothing fancy - just fast. First backup takes a bit, after that rsync automatically only updates files that have changed since the last backup. Makes backups over network connections a lot nicer on the pipe.

Oh, another nice thing... that backup, just add the /dev, /proc, /sys, and /tmp mount points, and you can boot straight into it (if your backup is on another device, as mine is). Just add a listing in grub for it.

[piymon]

ok first thing is i was not able to go thru the entire post with 131 pages and may be my question is already answered so sorry for that.

i wanted to know where is the softwares setup files are stored which i can backup so that i dont have to go thru synaptics to redownload them..

thanks in advance

[jchw]

I am not sure if anyone has mentioned Deja Dup in this thread because there are so many pages of responses to read.

I am relatively new to Ubuntu and I am not sure I would be able to follow the tutorial and then could I actually trust my backup working in the event of a major crash. This is due to my lack of skills not the tutorial methods. I have also installed Ubuntu on computers used by my wife, daughter and sister and it is next to impossible to get them to do anything if it is more complex than a click.

I have used Deja Dup several times to restore my data after a crash and re-installation and Deja Dup worked very well every time. Admitedly this is a two step approach where I reinstalled 10.10 and then restored the data. The restore set up all my settings even the connection details for Thunderbird. I have to go into Synaptic to load some of the special prograns like Fslint, Skype and wicd but that is very quick to do.

Deja Dup is set to backup every day to a USB memory stick automatically. All I have to do is plug it in and then disconnect it after the backup is completed. We each have our own USB stick and this has solved the backup problem for us.

[ziphem]

I have a question on "restoring" the system backed up with rsync.

I cloned my primary hard drive to a second, backup hard drive (the clone, cloned with Easeus). Let's say that in my laptop's second hard drive bay, I put an empty (not previously cloned) hard drive, and do a system backup as discussed in this thread - the rsync -av with the necessary exclusions. If, weeks later and after multiple file and system changes captured on the non-Easeus cloned hard drive with rsync, I were to then replace my primary hard drive with the cloned hard drive, and then 'restore' (also using rsync -av) back *from* the non-clone *to* the clone, would that work to update the clone? If theoretically yes, would I need to do it when the clone is not being booted off as a primary drive - i.e., booting up off live CD?

Day 1: Primary-->Cloned (via Easeus)
Days 2-30: Primary-->Empty HD (using rsync -av, daily)
Day 31: Empty HD-->Clone (using rsync -av, one time)

I hope this isn't too muddled!!

Thanks!!!

[mafi127]

Hi all,

Here is a shell I have put together from this post. I am not a guru on shell scripts this was put together with the help of google search :P.
Feel free to comment and/or change it.
I saved it as backupAndRestore.sh and you can run it as follows
sudo ./backupAndRestore.sh
enjoy

#!/bin/bash
S1='b'
S2='y'
echo "Please enter b for backup or r for restore:"
read action

if [ $action = $S1 ]; then

echo "These Directories will be excluded:"
echo -e "\033[1m\033[32m /proc /lost+found */backup.tgz /mnt /sys /dev"
echo -e "\033[0m"
echo "To chance these values edit this shell"
echo "continue (y/n):"

read action
if [ $action = $S2 ]; then
echo "Backing up PC ~ timestamp " ;date
tar cpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=*/backup.tgz --exclude=/mnt --exclude=/sys /
fi
else
echo -e "\033[1m\033[31mWARNING: this will overwrite every single file on your partition with the one in the archive!"
echo -e "\033[0m"
echo "Please enter name and location of the archive to restore:"
read fileName
echo "You are about to restore $fileName do you want to continue?(y/n)"
read action
echo -e "\033[1m\033[47mJust sit back and watch the fireworks.This might take a while. When it is done, you have a fully restored Ubuntu system!"
sleep 5
echo "start"
if [ $action = $S2 ]; then
tar xvpfz $fileName -C /
echo "Creating excluded directories"
#Just to make sure that all excluded directories are re-created
mkdir /proc
mkdir /lost+found
mkdir /mnt
mkdir /sys

fi
fi
echo -e "\033[0m"

Thank you for this one (Y)

-EDIT-

I got somekindow error whit this. How to fix it?

tamps@tamps-ThinkPad-R500:~/Desktop$ sudo bash backupAndRestore.sh
[sudo] password for tamps:
Please enter b for backup or r for restore:
b
These Directories will be excluded:
/proc /lost+found */backup.tgz /mnt /sys /dev

To chance these values edit this shell
continue (y/n):
y
Backing up PC ~ timestamp
P okt 2 11:48:16 EEST 2011
tar: Removing leading `/' from member names
tar: /home/tamps/.purple: file changed as we read it
tar: /home/tamps/.config/deluge/ipc/deluge-gtk: socket ignored
tar: /home/tamps/.gvfs: Cannot stat: Permission denied
tar: /dev/log: socket ignored
tar: Removing leading `/' from hard link targets
tar: /var/log/syslog: file changed as we read it
tar: /var/run/cups/cups.sock: socket ignored
tar: /var/run/pcscd/pcscd.comm: socket ignored
tar: /var/run/sdp: socket ignored
tar: /var/run/acpid.socket: socket ignored
tar: /var/run/avahi-daemon/socket: socket ignored
tar: /var/run/dbus/system_bus_socket: socket ignored
tar: /var/run/samba/winbindd_privileged/pipe: socket ignored
tar: /tmp/.X11-unix/X0: socket ignored
tar: /tmp/.winbindd/pipe: socket ignored
tar: /tmp/.ICE-unix/1356: socket ignored
tar: /tmp/pulse-B1jQAgh8jq5s/native: socket ignored
tar: /tmp/ssh-atuIircd1356/agent.1356: socket ignored
tar: /tmp/orbit-tamps/linc-2222-0-753816f517661: socket ignored
tar: /tmp/orbit-tamps/linc-643-0-6603bd0d786: socket ignored
tar: /tmp/orbit-tamps/linc-7e2-0-70ecb6a61acab: socket ignored
tar: /tmp/orbit-tamps/linc-647-0-db085babd03: socket ignored
tar: /tmp/orbit-tamps/linc-63c-0-3202c5d85d362: socket ignored
tar: /tmp/orbit-tamps/linc-1ee9-0-70027a34cab98: socket ignored
tar: /tmp/orbit-tamps/linc-5f6a-0-45e240b4df882: socket ignored
tar: /tmp/orbit-tamps/linc-74a-0-7ac39f4a1efca: socket ignored
tar: /tmp/orbit-tamps/linc-750-0-504812681be83: socket ignored
tar: /tmp/orbit-tamps/linc-f78-0-127dbe71dd163: socket ignored
tar: /tmp/orbit-tamps/linc-6ba-0-6c64fabab5e73: socket ignored
tar: /tmp/orbit-tamps/linc-5f9-0-2dee455ab42a: socket ignored
tar: /tmp/orbit-tamps/linc-54c-0-1911e3d8248d2: socket ignored
tar: /tmp/orbit-tamps/linc-784-0-3903ebf221e1f: socket ignored
tar: /tmp/orbit-tamps/linc-1f09-0-3153a849c3b13: socket ignored
tar: /tmp/orbit-tamps/linc-63b-0-a038f3952fea: socket ignored
tar: /tmp/orbit-tamps/linc-7eb-0-7434c9f2c17e: socket ignored
tar: /tmp/orbit-tamps/linc-897-0-494dcc7a6023e: socket ignored
tar: /tmp/orbit-tamps/linc-788-0-2af8d36325a77: socket ignored
tar: /tmp/orbit-tamps/linc-6e5-0-94caffaf1701: socket ignored
tar: /tmp/orbit-tamps/linc-700-0-5c8bb9124d44c: socket ignored
tar: /tmp/orbit-tamps/linc-a95-0-10314c4dcf97: socket ignored
tar: /tmp/orbit-tamps/linc-b56-0-4da3eb9cdeaa5: socket ignored
tar: /tmp/orbit-tamps/linc-576-0-527a21de1b2e4: socket ignored
tar: /tmp/orbit-tamps/linc-6c7-0-31c946a7d0530: socket ignored
tar: /tmp/orbit-tamps/linc-598-0-9dab47a7bbeb: socket ignored
tar: /tmp/orbit-tamps/linc-7e9-0-75080b581ae87: socket ignored
tar: /tmp/.esd-1000/socket: socket ignored
tar: /tmp/keyring-upfkrF/pkcs11: socket ignored
tar: /tmp/keyring-upfkrF/control: socket ignored
tar: /tmp/gedit.tamps.3047023951: socket ignored
tar: /tmp: file changed as we read it
tar: Exiting with failure status due to previous errors

[PayPaul]

OKay. I love the first part of this, creating the backup file. However I have what may be a true noobie question. Could I just install a new copy of ubuntu from a live CD into an existing partition on my main drive then copy the backup file back into the newly created root of the filesystem and restore my settings, installed packages etc..?

[khurtsiya]

this thread big one!

is first post correct for Ubuntu 11.04?

[AureiAnimus]

OKay. I love the first part of this, creating the backup file. However I have what may be a true noobie question. Could I just install a new copy of ubuntu from a live CD into an existing partition on my main drive then copy the backup file back into the newly created root of the filesystem and restore my settings, installed packages etc..?

I don't think so. As I understand it, uncompressing the .tgz overwrites everything. I wanted to ask a similar question:

Can I use this method to make a backup of my 11.04 install, then install 11.10 and 'just' extract the backup to revert back to 11.04 in case something doesn't work?

this thread big one!

is first post correct for Ubuntu 11.04?

Yes.

[ziphem]

I actually updated my rsync arguments from just


rsync -av --delete (plus specifics)
to:


rsync -av -Hl -X --delete --ignore-errors --exclude-from=/home/user/Backup_jobs/backup.lst / /media/Backup/
...thinking that -Hl and -X would be good to include to get closer having a 'mirror image' transfer.


My backup.lst btw is:


# Include
+ /dev/console
+ /dev/initctl
+ /dev/null
+ /dev/zero

# Exclude
- /dev/*
- /proc/*
- /sys/*
- /tmp/*
- lost+found/
- /media/SW_Preload/*
- /media/Backup/*
- /mnt/*
To expound on what I did, I bought a HD identical to the one I currently use as my primary HD. I used Easeus to make a full disk-copy, so it would be mirror image. I then put that drive somewhere safe.

I then took another drive that I have lying around, and put that in my computer as the secondary (backup) hard drive. Unfortunately, this HD is smaller, so I can't rsync everything onto it. So if you look at my backup.lst, I exclude /media/SW_Preload, which is my Windows 7 system. I exclude that because I use Linux 99% of the time, and there likely won't be any changes. That, plus the fact that I don't have size on my (smaller) backup drive to fit it. Of note, all my mounts, ie FAT32 etc partitions etc., are under /media/, not /mnt/.

There are two reasons for not rsyncing to the equal size, ghosted drive. First, it's a matter of security. If I lose my laptop, then both drives are gone and I have no backup. The second reason is because I've had problems with my primary drive reading the secondary drive's filesystem (I'm using LVM!).

At some point, let's say once every month or two, I'll swap out my primary drive with my ghosted secondary drive. I'll then restore from the backup drive to the ghosted drive by just switching the paths in my rsync backup file (and I guess that's the test then to see if it really works!).

[saurabh pandey]

so thanks for share these post its a amazing.

[kleverbear]

Hi all, great thread, need some help... I got some "Permission denied" while backing up. This is my first backup and i want to make sure i'm not getting paranoid and these xx_cache.tdb files are not critical.

Here's my backup command:
tar -cvpz --exclude=/backup.tar.gz --exclude=/proc --exclude=/lost+found --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/dev --exclude=/home --exclude=/opt / | ssh 10.x.x.x "( cat > sm1_backup.tar.gz )"

The command works great, during the output i was able to see a lot of these errors and caught a sample:
tar: /var/cache/samba/winbindd_cache.tdb: Cannot open: Permission denied
tar: /var/cache/samba/netsamlogon_cache.tdb: Cannot open: Permission denied

This was on several directories during the backup. Should i be concerned, also i noticed it stops at
/var/www/index.php
/webmin-setup.out
tar: Exiting with failure status due to previous errors
kb23@sm1:/home/media$

Is this normal? or did i do something wrong?

[beer-in-box]

What happens if I have more than one partition to backup and recover later?

Now I have one partition for / point and one for /home point (And I don't know why I did that). At this very moment, it backups my system. I don't expect any problems here. When it finishes, it will create one big bzip2 file.
Then when I mess up my system and can't get it in working condition without reinstalling it, I will want to recover it. But will the backup know where to copy stuff?

[scottbomb]

I did a backup of both computers the other day and it worked great. But this time, I'm getting the following error "tar: Removing leading '/' from member names" .... and it just stops and does nothing until I kill it with a CTRL-Z.

Any idea why it's doing this?

[piymon]

I wanted to format my whole computer and install windows 7 and ubuntu 11.04 on it.
well i wanted to know if i intall windows after ubuntu will the grub generate any error and if the dual booting will work fine.
secondly, i took the whole backup as mentioned in the first page of this post using tar so do i need to reinstall ubuntu and then restore my files or i just need to extract files in a new partition and add necessary folders to that partition and ubuntu will work.

one more thing in dis backup method does all the applications which i downloaded thru synaptics will be preserved or destroyed..

[adoggy]

Awesome! The only problem is that the terminal always says something like this once I am done entering everything in:
"Cowardly refused to start 'tar'. " :confused: Please reply.

[adoggy]

hey piymon. if u install ubuntu AFTER Windows then you will be able to choose which operating system to use at startup. u can go here to get the installer once you have windows.http://www.ubuntu.com/download/ubuntu/windows-installer

[adoggy]

Awesome! The only problem is that the terminal always says something like this once I am done entering everything in:
"Cowardly refused to start 'tar'. " :confused: Please reply.

[newb85]

I wanted to format my whole computer and install windows 7 and ubuntu 11.04 on it.
well i wanted to know if i intall windows after ubuntu will the grub generate any error and if the dual booting will work fine.
secondly, i took the whole backup as mentioned in the first page of this post using tar so do i need to reinstall ubuntu and then restore my files or i just need to extract files in a new partition and add necessary folders to that partition and ubuntu will work.

one more thing in dis backup method does all the applications which i downloaded thru synaptics will be preserved or destroyed..

All programs that were installed when you created the backup should be in the backup (unless, of course, you excluded them). So they should be there when you restore.

[newb85]

I've seen a few people ask about whether they can do a fresh install and then restore their system using the method in this tutorial. Here's the long answer:

If you're restoring your system because since creating the backup you've done something you don't like and don't know how else to put it back, then yes, that's how I would go about it.

If something in your system isn't quite right, but you want to keep your current programs and settings, creating a backup now, fresh installing, and restoring will not accomplish anything. Restoring will restore your problem along with your programs and settings. (Unless you exclude from the backup the file that's creating the problem. But then, if you knew that, there are probably quicker ways to fix the problem.)

[atey1]

I tried using this method to backup linux on one hard drive and restore it to another hard drive. However, when I restored and tried to login, I get errors lik e"could not update ICEAuthority in var/lib/gdm" and "The disk drive with UUID="..." is not ready yet or not present."

It gets to the login screen, but there are no users and it won't shut down or restart (it clicks but doesn't do anything). I can drop to a terminal and see all my files, but can't login.

I tried changing the drive mout information in /etc/fstab, but that didn't work, it just went straight to the login screen I mentioned.

I'm thinking this has to do with the fact that in my old drive root was on sda3 and swap was on sda5, while on this drive, the root should be at sda1 (the old drive was a triple booted mac/win/linux).

Any ideas?

[youoneah]

Awesome! The only problem is that the terminal always says something like this once I am done entering everything in:
"Cowardly refused to start 'tar'. " :confused: Please reply.

This hasn't been answered and I just got the same message, so here's what caused it for me. I was following an arsgeek tutorial (http://www.arsgeek.com/2006/10/18/how-to-back-up-and-restore-your-ubuntu-machine/) using the same approach as the original post. When entering the command, I left off the pathname, or starting point, which in this case is root "/". So when I enteredsudo tar cvpzf myfilename.tgz --exclude=”/proc/*” --exclude=”/lost+found/*” --exclude=”/dev/*” --exclude=”/mnt/*” --exclude=”/media/*” --exclude=”/sys/*” --exclude=”/tmp/*” --exclude “/var/cache/apt/*”
tar replied "Cowardly refusing to create an empty archive".

By entering insteadsudo tar cvpzf myfilename.tgz --exclude=”/proc/*” --exclude=”/lost+found/*” --exclude=”/dev/*” --exclude=”/mnt/*” --exclude=”/media/*” --exclude=”/sys/*” --exclude=”/tmp/*” --exclude “/var/cache/apt/*” /

the system backup was underway.

This is not the same error message as "Cowardly refused to start 'tar'"; however, a google doesn't turn up that exact error phrase. I'm sure if you post the exact command you used an answer would be forthcoming.

[Barney-R]

Useful thread, but how can anyone work their way through 133 pages? I've managed about 60 so far.

I did find it easy to create the backup file though, so thanks for that. It took about ten minutes on a fast machine (i7 processor, 6GB RAM etc).

Apologies if this is covered elsewhere in the thread, but I'm a comparative newbie and would welcome some clarification before committing myself (and possibly trashing my system).

Ubuntu 10.10 (Maverick). Two hard drives, one for the OS, one for data. Ubuntu only, not dual-boot.

Here's the code I used to create the backup in a folder named Backup on a partition (on the second hard drive) named General.

sudo su

cd /media/General/Backup

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /

I'm about to replace a failing hard drive, but the old drive is 300GB and the replacement is 750GB. Will the size difference cause a problem when I come to restore from the backup file?

The failing drive isn't partitioned, and neither will the replacement be prior to installation. It's a completely blank (new) drive.

(I'm planning to partition it later rather than give the whole 750GB to the operating system.)

Working from a live CD, I wouldn't know how to identify the new drive, so I'm planning to install Ubuntu first, and then restore the present system over the top. That way the location of / will be established before I start playing about in the terminal.

Am I right in thinking I can simply open the terminal and type the following line of code to restore the system from the backup file?

sudo tar xvpfz /media/General/Backup/backup.tgz -C /

The old drive's failing fast, and I already have the new one, so I'd appreciate any advice while there's still time. Thank you.

[Barney-R]

Please?

Anyone?

Just over a week ago I had 133 bad sectors. Today it's 181. I don't know how many more sectors I can afford to lose before it starts affecting my data, settings, performance.

I need to get the new drive in as soon as possible, so I'd welcome clarification of the above points if there's anyone here who can advise me.

[ziphem]

Thought I'd follow-up; I finally got around to "restoring," or rsync-ing back from my intermediary device to my backup HD, which I had ghosted from my primary hard drive 6 months ago. So, there were about 6 months worth of differences between my primary hard drive and the ghosted HD.

So I popped in the ghosted HD, with the intermediary HD also in the PC. I copied the file rsync-backup.sh from the intermediary HD to the gohsted hard drive, and renamed the file rsync-restore.sh. I edited rsync-restore.sh to switch the source and the target, as follows:


rsync -av -Hl -X --delete --ignore-errors --exclude-from=/home/user/Backup_jobs/backup.lst /media/Backup/ /
and it sure did work. My formerly out-of-date ghosted hard drive is now functionally a copy of my primary hard drive. Very nice indeed. Anyways, I thought that I'd follow-up with the hopes that my setup proves useful to someone else.

I actually updated my rsync arguments from just


rsync -av --delete (plus specifics)
to:


rsync -av -Hl -X --delete --ignore-errors --exclude-from=/home/user/Backup_jobs/backup.lst / /media/Backup/
...thinking that -Hl and -X would be good to include to get closer having a 'mirror image' transfer.


My backup.lst btw is:


# Include
+ /dev/console
+ /dev/initctl
+ /dev/null
+ /dev/zero

# Exclude
- /dev/*
- /proc/*
- /sys/*
- /tmp/*
- lost+found/
- /media/SW_Preload/*
- /media/Backup/*
- /mnt/*
To expound on what I did, I bought a HD identical to the one I currently use as my primary HD. I used Easeus to make a full disk-copy, so it would be mirror image. I then put that drive somewhere safe.

I then took another drive that I have lying around, and put that in my computer as the secondary (backup) hard drive. Unfortunately, this HD is smaller, so I can't rsync everything onto it. So if you look at my backup.lst, I exclude /media/SW_Preload, which is my Windows 7 system. I exclude that because I use Linux 99% of the time, and there likely won't be any changes. That, plus the fact that I don't have size on my (smaller) backup drive to fit it. Of note, all my mounts, ie FAT32 etc partitions etc., are under /media/, not /mnt/.

There are two reasons for not rsyncing to the equal size, ghosted drive. First, it's a matter of security. If I lose my laptop, then both drives are gone and I have no backup. The second reason is because I've had problems with my primary drive reading the secondary drive's filesystem (I'm using LVM!).

At some point, let's say once every month or two, I'll swap out my primary drive with my ghosted secondary drive. I'll then restore from the backup drive to the ghosted drive by just switching the paths in my rsync backup file (and I guess that's the test then to see if it really works!).

[ziphem]

.....and I decided to exclude from my Windows partition exclusion (- /media/SW_Preload/*) several directories that *do* get updated once in a while and I want to ensure get backed up! One such directory is:

C:\Digital Camera Pics

I couldn't quite get the escape character to work, so I used the wildcard *. The above Digital Camera Pics directory sits in my Linux filesystem under /media/SW_Preload/Digital Camera Pics. So, to get this directory with spaces to rsync successfully, I added the following line in the "# Include" section:

+ /media/SW_Preload/Digital*Camera*Pics

This worked.

I thought I should post this in case anyone else has issues getting files or directories with spaces to successfully rsync, as I did.

Thought I'd follow-up; I finally got around to "restoring," or rsync-ing back from my intermediary device to my backup HD, which I had ghosted from my primary hard drive 6 months ago. So, there were about 6 months worth of differences between my primary hard drive and the ghosted HD.

So I popped in the ghosted HD, with the intermediary HD also in the PC. I copied the file rsync-backup.sh from the intermediary HD to the gohsted hard drive, and renamed the file rsync-restore.sh. I edited rsync-restore.sh to switch the source and the target, as follows:


rsync -av -Hl -X --delete --ignore-errors --exclude-from=/home/user/Backup_jobs/backup.lst /media/Backup/ /
and it sure did work. My formerly out-of-date ghosted hard drive is now functionally a copy of my primary hard drive. Very nice indeed. Anyways, I thought that I'd follow-up with the hopes that my setup proves useful to someone else.

[badinoff]

Hi,

Ubuntu/Linux noob here. I am running dual boot and have to re-install Windows. I want to use this opportunity to re-install everything, clean up partitions, and all that good stuff. I know I can have my fun with Windows without having to touch Ubuntu, but I want to format the hard drive completely.

So I decided to backup my Ubuntu and restore it once I reinstall it. I have two questions:

1. I don't imagine backup.tgz can't be used during new system installation, but I won't be surprised if I imagine wrong. Can I somehow work it into the OS installation process?

2. If the answer to #1 is no, does it matter which version of Ubuntu is installed originally, and which version the backup file was created in? I'm guessing that I would have to at least be on the same major release (11.10), but do system updates have to match what was backed up?

Thanks in advance!
B

[newb85]

The answers to both questions depend on how much you included in backup.tar.gz.

If you included the entire system, then in theory, you could use the live CD to create the partitions exactly as they were before and then unpack the tar file to exactly the same location as before. Without reinstalling or updating, Ubuntu should be exactly as before. (You may need to correct the UUIDs in fstab, as the partitions you created will probably have different UUIDs than their predecessors.)

However, the sordid truth is that if you use that method, it won't really "clean up" your Ubuntu partition at all. If you want to do that, you will need to install and update the system, and then unpack the portions of the tar file that you want to keep (such as your home folder). Unfortunately, this will also mean you will have to reinstall your programs and adjust the settings to your liking. I understand that a program like Deja Dup would make this process easier, but I have no experience there.

[crazybear]

I think I get the basic ideas of how to tell the system what to back up and what not to, but I have two main questions.
1] In the event of a disk crash [as opposed to messing up a file or several] will this backup allow me to rebuild an copy of my original disk [up to date of backup] if I mix it with a new but the same OS? Or would a cloned drive be the better/easier way?
2] How do I tell it WHERE to place the backup [I want it to be on another partition on another drive, for safety.

Thanks

[chickenPie4breakfast]

my freeking God, this thread is far too long for a sane person to try and wade through!!!

[newb85]

I think I get the basic ideas of how to tell the system what to back up and what not to, but I have two main questions.
1] In the event of a disk crash [as opposed to messing up a file or several] will this backup allow me to rebuild an copy of my original disk [up to date of backup] if I mix it with a new but the same OS? Or would a cloned drive be the better/easier way?
2] How do I tell it WHERE to place the backup [I want it to be on another partition on another drive, for safety.

Thanks

I'll answer in reverse order. In heliode's original example,

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

the backup is in the new file "backup.tgz". Heliode did not specify a path to the file, so the file will be placed in the current directory. (The command "cd /" changed the current directory from the user's home directory to the root directory.) However, if the command had been

tar cvpzf backups/backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backups --exclude=/mnt --exclude=/sys /

The .tar file would have been placed in a the directory "backups" within the current directory. (In the exclusions, I excluded the entire "backups" directory because presumably, that directory would contain multiple backups, and one would not want to back up any of these backups.)

If one wanted to place the backup in a different partition or other external storage, then the path should include the mount location of that partition/drive. For example, on my system, I have a Windows partition "TI106151W0F".

tar cvpzf /media/TI106151W0F/backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/media/TI106151W0F/backup.tgz --exclude=/mnt --exclude=/sys /

Warning! In that example, I excluded the backup file, but did not exclude my Windows partition. This is a very bad idea, as tar is not an effective tool for backing up and restoring Windows, although it can be an effective tool for botching Windows. A few years ago, I had a dual-boot system, and in that version of Ubuntu, the Windows partition was placed instead in the "/mnt" directory, which we have excluded. (I don't believe the "/media" directory had yet been conceived.) So, to update the command to reflect where Ubuntu now mounts my Windows partition, exclude the entire media folder.

tar cvpzf /media/TI106151W0F/backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /



Now for your first question. I'm not sure what you mean by "cloned drive". If you mean simply extracting the backup to an empty partition/drive, then I would recommend that. The backup contains your entire system, so there's no need to install the OS and then overwrite the entire OS. (Of course, this will mean extracting the backup while running a liveCD or USB.)

If you install Ubuntu first and then extract, the only issue is removed files. Heliode's command for extracting adds all the files to your system that are in the backup, but it does not remove files that are not in the backup but not in the system. Therefore, any native files that were removed from the system before creating the backup will still be there after extracting the backup.

Before doing either, however, I suggest you take a look at the file system table, or fstab (location: /etc/fstab). If the partition/drive you're putting your system isn't the same one it was on before, the fstab may need to be changed to reflect the new partition/drive.

[claire14]

Hi, and welcome to the Heliode guide to successful backing-up and restoring of a Linux system!

Most of you have probably used Windows before you started using Ubuntu. During that time you might have needed to backup and restore your system. For Windows you would need proprietary software for which you would have to reboot your machine and boot into a special environment in which you could perform the backing-up/restoring (programs like Norton Ghost).
During that time you might have wondered why it wasn't possible to just add the whole c:\ to a big zip-file. This is impossible because in Windows, there are lots of files you can't copy or overwrite while they are being used, and therefore you needed specialized software to handle this.

Well, I'm here to tell you that those things, just like rebooting (http://www.ubuntuforums.org/showthread.php?t=34629), are Windows CrazyThings (tm). There's no need to use programs like Ghost to create backups of your Ubuntu system (or any Linux system, for that matter). In fact; using Ghost might be a very bad idea if you are using anything but ext2. Ext3, the default Ubuntu partition, is seen by Ghost as a damaged ext2 partition and does a very good job at screwing up your data.

1: Backing-up

"What should I use to backup my system then?" might you ask. Easy; the same thing you use to backup/compress everything else; TAR. Unlike Windows, Linux doesn't restrict root access to anything, so you can just throw every single file on a partition in a TAR file!

To do this, become root with

sudo su

and go to the root of your filesystem (we use this in our example, but you can go anywhere you want your backup to end up, including remote or removable drives.)

cd /


Now, below is the full command I would use to make a backup of my system:


tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

Now, lets explain this a little bit.
The 'tar' part is, obviously, the program we're going to use.

'cvpfz' are the options we give to tar, like 'create archive' (obviously),
'preserve permissions'(to keep the same permissions on everything the same), and 'gzip' to keep the size down.

Next, the name the archive is going to get. backup.tgz in our example.

Next comes the root of the directory we want to backup. Since we want to backup everything; /

Now come the directories we want to exclude. We don't want to backup everything since some dirs aren't very useful to include. Also make sure you don't include the file itself, or else you'll get weird results.
You might also not want to include the /mnt folder if you have other partitions mounted there or you'll end up backing those up too. Also make sure you don't have anything mounted in /media (i.e. don't have any cd's or removable media mounted). Either that or exclude /media.

EDIT : kvidell suggests below we also exclude the /dev directory. I have other evidence that says it is very unwise to do so though.

Well, if the command agrees with you, hit enter (or return, whatever) and sit back&relax. This might take a while.

Afterwards you'll have a file called backup.tgz in the root of your filessytem, which is probably pretty large. Now you can burn it to DVD or move it to another machine, whatever you like!

EDIT2:
At the end of the process you might get a message along the lines of 'tar: Error exit delayed from previous errors' or something, but in most cases you can just ignore that.

Alternatively, you can use Bzip2 to compress your backup. This means higher compression but lower speed. If compression is important to you, just substitute
the 'z' in the command with 'j', and give the backup the right extension.
That would make the command look like this:


tar cvpjf backup.tar.bz2 --exclude=/proc --exclude=/lost+found --exclude=/backup.tar.bz2 --exclude=/mnt --exclude=/sys /


2: Restoring

Warning: Please, for goodness sake, be careful here. If you don't understand what you are doing here you might end up overwriting stuff that is important to you, so please take care!

Well, we'll just continue with our example from the previous chapter; the file backup.tgz in the root of the partition.

Once again, make sure you are root and that you and the backup file are in the root of the filesystem.

One of the beautiful things of Linux is that This'll work even on a running system; no need to screw around with boot-cd's or anything. Of course, if you've rendered your system unbootable you might have no choice but to use a live-cd, but the results are the same. You can even remove every single file of a Linux system while it is running with one command. I'm not giving you that command though! ;-)

Well, back on-topic.
This is the command that I would use:


tar xvpfz backup.tgz -C /


Or if you used bz2;


tar xvpfj backup.tar.bz2 -C /


WARNING: this will overwrite every single file on your partition with the one in the archive!

Just hit enter/return/your brother/whatever and watch the fireworks. Again, this might take a while. When it is done, you have a fully restored Ubuntu system! Just make sure that, before you do anything else, you re-create the directories you excluded:

mkdir proc
mkdir lost+found
mkdir mnt
mkdir sys
etc...


And when you reboot, everything should be the way it was when you made the backup!

2.1: GRUB restore
Now, if you want to move your system to a new harddisk or if you did something nasty to your GRUB (like, say, install Windows), You'll also need to reinstall GRUB.
There are several very good howto's on how to do that here on this forum, so i'm not going to reinvent the wheel. Instead, take a look here:

http://www.ubuntuforums.org/showthread.php?t=24113&highlight=grub+restore

There are a couple of methods proposed. I personally recommend the second one, posted by remmelt, since that has always worked for me.


Well that's it! I hope it was helpful!
As always, any feedback is appreciated!

hi how do i send the back up to a external hard drive , or can i do this to get a full back up to an external hard drive = https://help.ubuntu.com/10.10/keeping-safe/C/backup-files.html

[dave2001]

If your backing up a system, and not just personal data. I have found the best option is just to make a hard drive image. For this task, backup options in Ubuntu all seem to be lacking in one aspect or another. Clonezilla is OK, but makes large backup files and hasn't always worked correctly for me.

If you can borrow the use of someone's Windows system for a few minutes, download the free Paragon Backup and Recovery: http://www.paragon-software.com/home/br-free/download.html

Then install and it and create a recovery CD.

The recovery CD provides a pre-boot enviroment which allows you to backup a disk-image of your HDD. There are lots of options available and it creates pretty small files. It's compatible even with ext4 files systems.

[claire14]

hi i can get to windows after i have done a back up and got back into main partition with a new linux pendrive i have to make as the pendrive i had isn't been recognised

[janisgeiger]

Does this way completely clone the system as it is, without anything broken?

eg. drivers, applications, their configurations, launchers, panels, their content, default zoom levels inside specific folders (using nautilus), position of symbols on the desktop, ETC

Is this, a clone of absolute 1:1, possible and if, is it possible using this method?

[crazybear]

If your backing up a system, and not just personal data. I have found the best option is just to make a hard drive image. For this task, backup options in Ubuntu all seem to be lacking in one aspect or another. Clonezilla is OK, but makes large backup files and hasn't always worked correctly for me.

If you can borrow the use of someone's Windows system for a few minutes, download the free Paragon Backup and Recovery: http://www.paragon-software.com/home/br-free/download.html

Then install and it and create a recovery CD.

The recovery CD provides a pre-boot enviroment which allows you to backup a disk-image of your HDD. There are lots of options available and it creates pretty small files. It's compatible even with ext4 files systems.

Yes, I want to back up the entire system and information - My disks crash regularly and can't be recovered [newer drives are less-recoverable - everyone beware!]...so, Maybe this is the solution. I have Windows and have Paragon....but not sure about the set up, as Windows can't see Linux disks...will try and see what happens in the pre-boot environment. Thanks.

NB - you see to imply that one should install the Pargon on the Ubuntu system...but a windows-based program won't. I do NOT dual boot. I keep my different OS on different disks entirely! Did you mean other?

[janisgeiger]

hey. I'm pretty much surprised about how this linux thing is working....

I was sceptical about this tar method so I used dd_rescue to make an image of my drive, erased the drive, mounted the image and used sudo cp to copy it to the new empty drive. Did Grub restore etc (before) et voila, it's back as if nothing happened... No driver issues and so on. complete duplicate of the system. how is this possible?

[dave2001]

you see to imply that one should install the Pargon on the Ubuntu system...but a windows-based program won't. I do NOT dual boot. I keep my different OS on different disks entirely! Did you mean other?

Sorry if I wasn't clear Crazybear, You do indeed need a windows based system to install Paragon on.

However, once you use the Windows-based Paragon application to create a "rescue and recovery" CD, your good to go.

The resulting CD is bootable, and works as a stand-alone solution for backup and recovery, with NTFS, FAT, and Ext2, Ext3, Ext4 filesystems. I don't know why Paragon doesn't just offer a download of the "rescue and recovery" CD as an iso.

[muppet317]

Just a quick note.

The first time I ran the script in the original post I got the following error:

tar: /: file changed as we read it
root@ubuntu-desktop:/#

I checked the thread and it didnt seem clearly resolved, the clearest advice was

You probably were executing a program and it was writing actively to a file who was being read for tar'ing.

I'd say: back up again (just for safety) but try to have a minimal number of programs running. (I was getting this when tar'ing my home folder with firefox running...)So I ran it again with every application other than terminal emulator closed and hit the same problem.

I resolved this and it worked fine simply by adding --exclude=/dev to the original code. The backup then ran fine without the error (and I noticed that when it had previously stopped with the earlier it hadn't completed a full backup).

I haven't yet tried unpacking, but thought I'd post this up here in case other people come across the same problem.

[muppet317]

And thank you for amazingly useful how to, if this saves me from trying to remember all the tweaks and packages I struggle to find whenever i wreck my system this will be an absolute godsend.

[ziphem]

Thought I'd do another followup from previous posts (http://ubuntuforums.org/showpost.php?p=11321204&postcount=1315)

Using rsync to backup my system to an internal 2nd HD (the backup) is fantastic - but it creates a major problem if the backup hard drive is not mounted (my drives/partitions don't mount automatically, but I mount them manually every boot up).

If the backup is mounted, which is at /media/Backup, then the system will rysnc appropriately, and the world is good. However, what if I forgot to mount one partition manually upon booting up? If the 2nd hard drive is not mounted, my rsync script will create the directory /media/Backup on the PRIMARY drive, so...: you guessed it, my primary drive runs out of HD room. What happens next is a system that ultimately ceases to work, and which will not reboot back into my linux distro. This is fixed by deleting the [primary HD] /media/Backup directory right away, if possible. If not possible, then I have to reboot into a rescue disk/USB, and delete /media/Backup that way. Quite a pain.

Below is a solution that solves this problem. I hope this helps others, as it is quite a good help to me. I don't know why I didn't think of writing this when I started using rsync to backup my system! Also, I guess I could do something fancier instead of echo "Not all devices mounted!!", but it serves my purposes. Eventually I'll see that not all drives are mounted ;)


#!/bin/bash

if [ -d /media/SHARED ] && [ -d /media/Backup ] && [ -d /media/SW_Preload ]
then
rsync -av -Hl -X --delete --stats --ignore-errors --exclude-from=/home/user/Backup_jobs/backup.lst / /media/Backup/
else
echo "Not all devices mounted!!"
fi

[Robert Lock]

Dude, Heliode, this is some darn good advice! I love the break down of the command, demonstrating you not only know the command, but what it actually means. I also really appreciate the sense of humor you bring to the table (i.e., Windows CrazyThings (tm), now that's entertainment!) I'm in my first six months of making the switch, and I couldn't be more pleased with Ubuntu or with Linux for that matter. Like you and many others, I was once a WinKy user, and this backup method beats the daylights out of anything coming out of the MS world.

Thanks for your time of posting this in the forums!

P.S.-- I think I know the command you're talking about when you said:

"You can even remove every single file of a Linux system while it is running with one command. I'm not giving you that command though!"

-That's probably a good idea - but don't we all feel like typing sudo s**** / every now and then when our computers aren't doing what we want?

[hoveman]

@Heliode: Thanks for your nice tutorial.

I would also suggest to exclude the /media folder, since external devices or some dvd's shouldn't be backuped? Or what would you say?

best regards...

[nepalnt21]

so if i use this guide to create a backup, and then with some messing around screw up my system completely, i should be able to use the backup to restore back completely, including gui settings, etc.?

[Robert Lock]

@Heliode: Thanks for your nice tutorial.

I would also suggest to exclude the /media folder, since external devices or some dvd's shouldn't be backuped? Or what would you say?

best regards...

hoveman - you can exclude the /media folder, or simply unmount any partitions you have mounted there, however, if you want them as part of your backup, then you should leave them. I think heliode is giving this tutorial as a system backup, but you can use it for a full backup, if you wish. As far as audio CD (if one is in the drive) you'll want to exclude /home/<user>/.gvfs - or, of course, just eject the CD, but this should give you some more options.

[ziphem]

hoveman - you can exclude the /media folder, or simply unmount any partitions you have mounted there, however, if you want them as part of your backup, then you should leave them. I think heliode is giving this tutorial as a system backup, but you can use it for a full backup, if you wish. As far as audio CD (if one is in the drive) you'll want to exclude /home/<user>/.gvfs - or, of course, just eject the CD, but this should give you some more options.

And make an exclusion to avoid backing up your backup...

[Robert Lock]

And make an exclusion to avoid backing up your backup...

LOL - yeah, of course, I was thinking about that when Heliode mentioned it in the guide - I can see how that could be a hairy proposal in a never-ending loop.

[ecolom1269]

I need to set up a backup but first have to get Ubunto 12.04 to stop reconize my baracuda IDE in enclosure as a mouse or keyboard. I can see the enclosure isted as "cypress" the manufacture but my dell laptop m5030 does zip when its plugged in.

I tinkered around with dmesg, and lsusb, fdisk and got some information but was unable to mount it manually. I believe its because of the way this particular usb enclosure case is acknowledged by ubunto. I really don't want to go out and purchase another case. :( Im a bit poor at the moment. ;)

lsusb list the device (with HIDdriver)
fdisk doesn't show the device
dmesg says its connected.

I want to use this baracuda for backup

[ecolom1269]

oh obviously i tried the mount command with the info from other commands.

sorry...didn't mention that.

[crazybear]

Only a newbee [as I] could make such a horrible mistake. I tried to boot in and got a screen that said 'no OS on this disk'...so I mistakenly reinstalled 11.04. [it seemed to indicate it would save all of my files and drivers - it only saved MOST [not all] of my programs! I thought it would only put in the OS files - and not remove all the: history, configuration files etc. [they were done by an expert over many weeks - and he is now no longer alive]....so, I have a system that won't configure my 3 screens; won't configure the ATI ccc driver, won't print and several other long-fought configurations are gone - as is the history, which would allow me to re-create his fixes. However, I think all is not lost....I made a copy of the system and it is on another disk [now seen as /dev/sdb1]. On it there is a file /boot/initrd.img-2.6.38-13-generic [and all the other versions I had had installed] that I THINK I need to copy/move/unpack? to my main system and all will [?] be OK?!?!?!?!? BUT - I: 1] don't know how and 2] if I should...though it seems I should. What else might I have to move [and how to do so]?

Please tell me step by step and ask anything / tell anything that could make the above more of a problem than a solution. Many thanks! The only difference between the 'old' copied system and the new is the addition of Ubuntu studio [which I think will not make a difference -but don't know].

NB - I think trying to configure the ccc again Ditto making printer [Cannon MP 540] and several other devices. It seems now only the configurations for these devices are gone....]I made a LITTLE progress with the ATI ccc driver, but it is NOT working as it did and NOT working correctly

NBB - and while I'm at it...during some update 2.6.38-14 was installed, but it says 'virtual' after it and it doesn't boot...what is that?! I move down to the 13 and it boots..or did...now it only boots in low video mode [or seomething like that - I'm in Windoz now]
-------------------------------
Looking further, in the copied [about a month old] Grub files are:
abi-kernal#
config-kernal#
initrd.img-kernal#
System.map-kernal#
vmcoreinfo-kernal#
vmlimuz-kernal#
files.....
Question, How and which [all?] of these do I move back to my main device/OS - and what dangers precautions need be taken into account? Thanks much!

--------------------
With more hunting I found the copied [month old, but all needed changes were before then!] history file!....but am not very confident about how to move it into my system in sda1 or if it will cause problems with what is there...nor how to execute the commands where they are - if possible. I have printed it out [30 pages]. There is also a .bashrc file that from its size seems important. Now what do I do?! This is my trial by fire.....I'll be intermediate after this!

Even more hunting [have edited this many times over several hours!] I find that when I was asked for my user name I made a slight punctuation/spacing difference and some [not all] of the old files that used to be in my old account are still PARTLY there?!?!?...now it is even more confusing [to me] how to reconstruct things...but maybe not....don't know. Would REALLY appreciate the help!

And, as if I didn't have enough problems, now this morning [I've logged on many times already] it won't let me log in - as if the password was changed. I did NOT change it and there is ONLY ONE account! Boy, did I mess up and not back-up correctly nor know what I was doing.....learning the hard way....

sorry if I sound a bit panicked - I am!

[kiwimenace]

Is it possible for me to just manually copy all the files(incl hidden file) from "file system" partition to a safe place and then just copy them back again later under a live disk to restore my system?

I have a separate /home partition.

Cheers

[crazybear]

Is it possible for me to just manually copy all the files(incl hidden file) from "file system" partition to a safe place and then just copy them back again later under a live disk to restore my system?

I have a separate /home partition.

Cheers

What I don't know [and fear] is that a wholesale movement/overwriting of ALL files might [I'd think LIKELY will] cause problems - NO? My system looked quite different before/after. Wouldn't it be better to move only the configuration files to start with - those with the old setups. I don't really know what is stored where. I also don't understand the new entry for a virtual kernel; nor not being able to log in with password except in failsafe graphics mode [but if the old configuration files were there, that would likely go away. Will older files overwrite newer ones? Many thanks.

[newb85]

@ crazybear & kiwimenace,

Simply copying all files back to the partition using a live disc should only be a problem if you've created/deleted, moved, resized, etc. a partition or partitions (i.e., if any of the UUIDs have changed). In that case, it would still work, but you would have to edit your fstab file to reflect the changes.

The default for tar -x is not to remove files from the destination that are not in the backup, so if you're taking the wholesale approach, delete all contents of the destination partition first (start with a blank partition).

@ crazybear

The trouble with only moving the old configuration files is exactly that you don't know where they all are. With a little research, though, you could probably find most of them. What exactly did you mean when you said this:

My system looked quite different before/after.

How did it look different?

[crazybear]

@ crazybear & kiwimenace,

Simply copying all files back to the partition using a live disc should only be a problem if you've created/deleted, moved, resized, etc. a partition or partitions (i.e., if any of the UUIDs have changed). In that case, it would still work, but you would have to edit your fstab file to reflect the changes.

No, I have not moved, resized the partition. I re-installed over and it said it would save my files.....it apparently did, but all the drivers and general set-up is different! The drivers for the Printer, Video Card and several other things were GONE...these my Guru had worked long and hard on.

The default for tar -x is not to remove files from the destination that are not in the backup, so if you're taking the wholesale approach, delete all contents of the destination partition first (start with a blank partition).

I didn't use tar to backup. I simply made a copy using dd, I believe. The Linux Guru who used to help me [and is now apparently dead] also made a file within the main device called 'Ubuntucopy' which also has a copy of the originals, it seems. Lastly, as when I re-installed I used a slight variation of the original user name, there seem to be the old and new users.

@ crazybear

The trouble with only moving the old configuration files is exactly that you don't know where they all are. With a little research, though, you could probably find most of them. What exactly did you mean when you said this:



How did it look different?

I especially don't know which are important and what they contain.....which are the old [I guess I can try looking at dates on them] and which might have been modified with the new install. More importantly: 1] I don't know what problems could be caused if there were two sets of a file; or mixed and matched files that need to coordinate with one another, and 2] Using Live CD some [quite a few] are locked to me and I can't open [and likely can't copy and overwrite without permission - though I'm the 'Adminstrator' - except by terminal, I would think.

As to looking different, this is hard to fully explain. Remember too [all seem to ignore this], that NOW I can NOT log in except [I discovered] in default graphics mode under the repair scenario...for reasons unknown [I could log in for a while after the change the 'normal' way]. Additionally, there are two new entries in the list of kernels. [14, I believe] and they say (virtual)...I don't know why nor what they are and I never set up knowingly anything like a VM scenario. Also, worth mentioning 13, 12 and earlier kernels can NOT be booted into. ONLY 14 and ONLY in failsafe graphics mode...although this was not the case until I started to [?!] set up the ATI ccc driver....which [I thought] was working with one small exception of one LCD screen of three being the wrong color and it would allow me to only change that one - none of the others...seemed minor...maybe unrelated.

I do NOT have another OS on this disk. Lastly, just before the system went 'weird' I suddenly had the new look of the Ubuntu Studio - unexpected, but not bad...but strange as i had NOT just installed it...I believe I had months and months ago and it suddenly took over the look of the desktop. That was BEFORE I got the screen that there was 'NO OS on this disk', which caused me to mistakenly panic and re-install...I should have tried other things first...as there WAS an OS.

So, I have copies of the old configurations and files...if I can determine which I need and which they are....please tell me someone how in terminal I can move them back and not be blocked with permissions. I think the system doesn't know who the Administrator is anymore.

Hope that answers your questions and asks mine in a manner that is comprehensible. My old Guru would have fixed this in a flash....alas....I'm on my own now.:(

Thanks so much for the help....I'm VERY hesitant to attempt moving the files back until I
1] can open and see all the files
2] know which ones would be different and should be moved [and which might cause a problem if moved - if any]
3] how to move them in the terminal, so I can have the proper permissions
4] if the old settings for the video card, printer et al. should all be returned or if some hybrid will be made in the process [a Frankenstein]?
5] any other warnings a novice like me should know about BEFORE attempting this.

[newb85]

First, some general forum etiquette: You're posting your questions in the wrong thread. Despite its vague name, this thread is about backing up and restoring using the tar command and the method prescribed by Heliode. If you choose instead to use DejaDup, more power to you, but your questions should be posted in a thread about DejaDup. If no appropriate thread exists, create your own. In addition to adding off-topic posts to an already unwieldy thread, your questions have confused those who have tried to help you. (E.g., I assumed you had backed up your system according to the howto at the beginning of the thread, so some of the answers I gave you were not appropriate for your situation.)

Second, some general backup basics: There are a plethora of backup and restore options, and part of the freedom of Ubuntu is your ability to choose which one you want to use. However, you need to be consistent: if you created your backup with DejaDup, you need to restore your system using the recommended DejaDup method. (I am not at all familiar with DejaDup, so I can't be any help here.) Some backup methods copy everything in your system, while others track changes you've made and still others use different paradigms. Mixing and matching can have undesirable results, as you've found.

Third, a point on administrator and superuser: In Linux, being an administrator does not automatically give you access to all files. It enables you to gain access to all files, that is, to act as superuser. If you try, from a terminal, to open a file that your user doesn't have permission to open, the system won't let you. However, if you precede the command with "sudo" and enter your password when prompted, the system will allow you to open it as superuser. The default file browser in Ubuntu is Nautilus. If, in the terminal, you enter sudo nautilus, it will open a browser window where you will have access to all files. There are also commands you can execute as superuser to change ownership or permissions of particular files.

Lastly, you probably shouldn't post something like I'm on my own now. on a forum like this one. You're not on your own. You have access to a community through this forum. You just have to learn a few things along the way.

[crazybear]

Thanks. Sorry, if this is on an inappropriate thread. I had started my own...waited several days...but had NO responses and NO access to Ubuntu [now it is several weeks]...so looked around for another thread I though was on the same topic. Sorry, if that was not true. Are there not moderators to move things around. I'll not post here further...but go back to my own thread and wait. I'll try out what you suggest. I hope it works. Hard to believe that with an install CD one can get in and change any Ubuntu system. in this case, however, it is most welcome. Again sorry and again thanks.

- NB, yes, I know there is a community here and I have used it a few times before. Sometimes with success and sometimes without. As I said, I had a local older man who had spent his entire life programming Linux who had often walked the few blocks over to assist me. Sadly, he has died, and sadly he was SO far advanced and so quick at the terminal [he did everything in the terminal] he often did things that I couldn't understand or even keep apace with. Now I'll cut and past some of this back to my thread and hope it works out. ):P

[crazybear]

deleted

[crazybear]

deleted

[UsernameIsInUse]

Usefull, thanks!