Howto: Backup and restore your system! [Archive] - Page 4

afd

November 27th, 2008, 12:39 PM

Scorp123 - Thanks for setting me straight on a few things.

Perhaps you can help as well as explain. The 'drivers' (as termed on the wiki page you pointed me towards) or 'modules' as you describe them, are saved somewhere within the OS filesystem.

Why would the backup not include these files? I ensured I didn't exclude anything that the hardware would rely on when restoring.

Is it down to the fact that I backed up and restored while running the OS? In this walkthrough the author states that the backup/restore can happen while running the OS:
One of the beautiful things of Linux is that This'll work even on a running system; no need to screw around with boot-cd's or anything.

So, if you're right in saying that Wifi has little or nothing to do with restoring the OS I really don't see the point in doing backup/restores of the OS as opposed to just home folders. If I backup a computer correctly, format said computer and then restore the computer to it's original condition before the format I would expect everything to be as it was... if any module or driver was excluded I would see that as a failed backup/restore.

I'm sure I made a mistake in either the backup or restore and not that the procedure is flawed.

So, who knows where my previously working modules went? Is there a way to inspect the backup tar for fwcutter/STA wireless driver files?

cygnis1

November 27th, 2008, 12:52 PM

Perhaps it is not the backup that is causing the problem. I know that just about everything I do does not work like it does for most users. If there is something inherently wrong with your system and when you restore it you still have the same problem.

F W Adams

November 27th, 2008, 12:57 PM

Not backing up the contents of directories you don't want:

Just did an experiment, you can backup a directory without backing up its contents, this saves you having to re-create the empty directory manually with mkdir after restoring. Perhaps more important, you don't need to remember to do it. The restore will give you the empty directory.

For example, use:

--exclude=/mnt/*

rather than

--exclude=/mnt

mnt will then be restored as an empty directory

scorp123

November 27th, 2008, 01:40 PM

Why would the backup not include these files? Unless you excluded /lib/modules from your backups they are being backed up. Again: Whatever is in the backup or isn't cannot help you if you have flaky hardware that decides to "not to be there" when it feels like it. "Now I am here, ... now I am not ... Now I am here ... now I am not ... Now I am here ... now I am not ... " <== has nothing to do with the driver or the backup.

In this walkthrough the author states that the backup/restore can happen while running the OS: I think this is being misunderstood. You can restore most parts while the OS is running, but there are a few critical areas which better be not touched while the OS is active. A complete restore of stuff inside /etc, /usr, /var/cache or /lib can go awfully wrong when any process in there was still running. Other areas like your own /home, /opt, /var/tmp and so on are far less sensitive.

Rule of thumb: If you only need to restore single files (e.g. maybe you accidentally deleted something?) or minor things (e.g. single config files) you are most likely safe and do it while the OS is running. As soon as you need to overwrite entire disk structures (e.g. a complete wipe and restore of /usr !) you better be on a live CD because it's safer and has a far higher chance of not going wrong.

If I backup a computer correctly, format said computer and then restore the computer to it's original condition before the format I would expect everything to be as it was... Under the condition that all hardware too will work correctly: Yes. That's the purpose of a backup. I do that all the time. But my backup can't help if if between the time I took the backup and the time I decided to restore a hardware component decided to fry itself ... thus rendering the computer unable to load any driver because it can't see said piece of hardware anymore. See a few postings further up ... my previous computer which worked tip top for the past six years all of a sudden decided that it had no USB ports ... Obviously somewhere around Monday evening something got fried and the component that controlled the USB ports in my PC was gone ... No more USB sticks, no more USB keyboards, no more USB mice ..... But is this the fault of my backup? Of course not. My backup can't help it if hardware dies.

In your case it would be better to open a new thread and have that WiFi adapter of your's troubleshooted, backup or not.

bigrev

November 28th, 2008, 01:43 PM

Hi

Right now I am backing up my home partition (and my documents from my Windows partition) daily with the help of Simple Backup Config. With so many different options that were written in this thread, I am a bit confused regarding which is the best option to backup my entire system, so that in case of a catastrophe I can easily restore everything like it was. Can someone help me?:)

nufros

November 29th, 2008, 12:28 PM

I feel the same way as bigrev... with 76 pages full of different opinions on what works and what doesn't, as a basic user I'm really confused and worried about trying anything in this How-to...

What I'd like to know is if it is possible (and if so, how) to easily create a backup CD or DVD (by means of commands or specific programs) that I could insert on startup to restore to system to it's former glory in the even of a catastrophic failure (or just to roll back system changes I may not like)

kyleryner

November 29th, 2008, 09:30 PM

I feel the same way as bigrev... with 76 pages full of different opinions on what works and what doesn't, as a basic user I'm really confused and worried about trying anything in this How-to...

i echo this sentiment as well. Ive only been a linux user for a week :)

So far i had to reinstall once when Ubuntu failed to boot after a crash.. not really a bid deal since i didnt have any new data or anything. And for now, i have no plans on saving important data in its root.. all douments etc still to be accessed/saved in a windows drive (mounted).

But still, the prospect of re-installing ubuntu and re-downloading loads of programs in case of a catastrophic crash is a pain.

But i dont think im looking for a complete HD or root backup (i dont even do that with windows).

At the very least, is it possible to save the downloaded programs as an install package (ie., like a windows installer.exe file) to serve as a backup/installer for faster restoration of a system?

El_Belgicano

November 30th, 2008, 06:06 AM

What has worked for me so far:

On an idle system: (running, but nothing more than the WM/DE and little tasks)

1 ) Backup "root":
sudo tar cvpzf /media/Backup/bkp-root.tgz --exclude=/home --exclude=/etc/fstab --exclude=/tmp --exclude=/boot --exclude=/proc --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /

2 ) Backup "home":
sudo tar cvpzf /media/Backup/bkp-home.tgz /home/

3 ) Move backups somewhere safe.

4 ) System damage !!!!

5 ) Text install with the minimal Gutsy CD (enough to have "tar" installed again)

6 ) Move Backups from storage location to "/" and change terminal location to "/".

7 ) Simply untar both archives.

8 ) Type "sudo reboot", your password

9 ) Enjoy your new old system...8)

lametike

November 30th, 2008, 06:09 AM

is there a software i can use for back up or ghost-ing?(clone)

scorp123

November 30th, 2008, 10:43 AM

But still, the prospect of re-installing ubuntu and re-downloading loads of programs in case of a catastrophic crash is a pain. Google "AptOnCD". It's in the repos too and you'd install it via: sudo apt-get install aptoncd ... the program should then be available via the "System > Administration" menu. What it does: It burns all the patches and updates you downloaded onto a CD or a DVD. When you have to reinstall a system again, you don't download all the stuff again but instead just download "aptoncd" again and then tell it to restore the packages from the DVD. If you have a slow internet connection and/or quota restrictions on your internet subscription then "aptoncd" is ideal.

At the very least, is it possible to save the downloaded programs as an install package Bingo, see above ;)

scorp123

November 30th, 2008, 10:52 AM

is there a software i can use for back up or ghost-ing?(clone) I personally am not a fan of "ghosting". If you do a complete harddisk image you not only copy the sectors of the HD that are really used but you will also waste tons of disk space for empty sectors.

I personally use this method (which is similar to what's posted on the first page of this thread):
http://www.linuxmint.com/forum/viewtopic.php?f=42&t=3969&start=0&st=0&sk=t&sd=a

I work as system administrator since 1998 and above method is tested and tried (at work I'd use magnetic tapes though and not USB harddisks); I always get my machines completely back even if nothing is left ("bare metal" recovery). As far as "cloning" goes: In that guide there is a section where I explain how to save the list of all installed packages and how to restore that list again. So you can tell a new system to automagically download and install the same packages a previous system had, so both systems end up having the same software installed. Copy over any config files you'd need and voila: 100% identical clones. And it's fast too.

But if you really insist on "ghosting" ... well, there is partimage and there is clonezilla ..... I never used those. The reason is simple: When your installation goes to Nirvana those programs won't be around anymore ... so how are you going to restore your backup?

That's why I only trust in command line methods (so the programs will work, GUI or not) and in tools which are most definitely around on any Live CD, regardless if it's Ubuntu, OpenSUSE, Gentoo, OpenSolaris ... "tar" and "gzip" are always there, I don't need to hunt for extra-software to get my data back. ;)

But if you really want "ghosting" ... OK, you'd have to search the forums here for how to use above programs. And then there is "remastersys" which supposedly can burn a backup of your installation onto a bootable DVD ..... Nice idea, but I haven't tried that (I have way more data than could ever fit onto a DVD), but maybe you'd want to give that a try too?

SmartWish

December 7th, 2008, 06:06 AM

Awesome guide! very easy to understand you too :D
I give this guide a strong 6.

b2m

December 10th, 2008, 09:55 PM

would this work with a windows volume ? so say if u issued the command in your windows volume from ubuntu ? could u extract it back on a different drive and have the same results ?

scorp123

December 11th, 2008, 04:27 AM

would this work with a windows volume ? Windows uses NTFS as filesystem. Writing to NTFS (which is regarded as being a "trade secret" by Microsoft) is a bit tricky on Linux. Current implementations use a mechanism called "ntfs-3g" which has to pass through "user space" instead of passing through the kernel (like Linux's native filesystem drivers do) making reading and writing large chunks of data really really slow. Just see for yourself ... a few postings further up someone had troubles with NTFS. It was too slow and it took days to backup the data ....

For making backups of your Windows partitions from Linux you'd be better off by using programs such as partimage ... You should find instructions on how to use that one when you search this forum. "partimage" does not read or write from or to the filesystem, it instead takes the entire partition (regardless what filesystem is on it) and puts it into a compressed image. As far as NTFS goes I'd say that this is the safest method from a Linux perspective.

b2m

December 12th, 2008, 12:47 AM

ah ok cool. ill give that a shot. im having a lot of trouble restoring my backup. after i extract the file back on to the linux partition i try to reboot and the system hangs at the ubuntu boot screen and after quite a while i just get a command promt. im unsure how to proceed. has anyone else had this issue ? i get this error message "check root=bootarg cat /proc/cmdline or missing modules, devices: cat /proc/module is /dev. ALERT!" any ideas whats causing this ?

xarte

December 12th, 2008, 01:53 AM

um. why would anyone want to buy a program when a couple of lines in the CLI will do the exact same thing? ... I don't get it....

Great how-to, thanks so much!

scorp123

December 12th, 2008, 08:56 AM

why would anyone want to buy a program when a couple of lines in the CLI will do the exact same thing? "buy a program" ??? You lost me there.

Who's buying a program???

scorp123

December 12th, 2008, 08:57 AM

i get this error message "check root=bootarg cat /proc/cmdline or missing modules, devices: cat /proc/module is /dev. ALERT!" any ideas whats causing this ? Can you please post a photo showing the exact message?

xarte

December 12th, 2008, 05:03 PM

"buy a program" ??? You lost me there.

Who's buying a program???

oh, it's just that someone was talking about using Norton Ghost and it costs a hundred and fifty bucks.... totally irrelevant anyway, I shouldn't post when I'm half asleep....

now if only Santa would swing for a 1TB drive....

BassKozz

December 16th, 2008, 03:33 PM

Has anyone ever tried QuickStart (http://www.ubuntugeek.com/quickstart-back-up-restore-and-set-up-ubuntu-quickly-and-easily.html) ?
Is QuickStart better/worse then the method described in this thread?

scorp123

December 16th, 2008, 06:47 PM

Is QuickStart better/worse then the method described in this thread? Let's assume the following:

1.) - you used QuickStart for your backups. And because you relied on this "point and clicky" thing you have no clue how to restore your files manually. Why should you? You "pointed and clicked". Why should you know anything about manually restoring archives? So you will have to use QuickStart again for your restores.

2.) - your computer dies

3.) - for various stupid reasons your Internet died too. You can't get to the Internet, you can't ask for help in the forums, you can't download anything.

Because of 2.) you need 1.). But 3.) is a real bummer and means you're screwed and won't be able get your backups back if it happens ....

Are you really sure you want to rely on a tool that most likely won't be around once a disaster happens?

Wouldn't it be better to learn a few basic commands, gain some confidence and a little bit of know-how with your new OS, and therefore be able to restore any backup from any Live CD if something bad happens?

12 years of Linux usage (I started in 1996) have taught me never ever to trust a backup tool that won't be there anymore when a disaster happens. You can choose to believe me. You can choose not to believe me and go through the same hells I have already been to :D

Fact is that the few basic "tar" commands shown here in this thread are not so hard as some people make them to be. Once you get the syntax they are easy. And the most important thing: "tar" is always there. Name a Live CD. No matter which one. I bet it has "tar" already installed! It's therefore easy to get the files back. Heck, even stupid WinZIP can open *.tar.gz files and get your files out if needed. :)

QuickStart looks nice, yes .... but what when your data are gone? And what if you don't have access to the Internet anymore and therefore can't download it again ... ?

Think of it. ;)

gychang

December 16th, 2008, 11:51 PM

Let's assume the following:

1.) - you used QuickStart for your backups. And because you relied on this "point and clicky" thing you have no clue how to restore your files manually. Why should you? You "pointed and clicked". Why should you know anything about manually restoring archives? So you will have to use QuickStart again for your restores.

2.) - your computer dies

3.) - for various stupid reasons your Internet died too. You can't get to the Internet, you can't ask for help in the forums, you can't download anything.

Because of 2.) you need 1.). But 3.) is a real bummer and means you're screwed and won't be able get your backups back if it happens ....

Are you really sure you want to rely on a tool that most likely won't be around once a disaster happens?

Wouldn't it be better to learn a few basic commands, gain some confidence and a little bit of know-how with your new OS, and therefore be able to restore any backup from any Live CD if something bad happens?

12 years of Linux usage (I started in 1996) have taught me never ever to trust a backup tool that won't be there anymore when a disaster happens. You can choose to believe me. You can choose not to believe me and go through the same hells I have already been to :D

Fact is that the few basic "tar" commands shown here in this thread are not so hard as some people make them to be. Once you get the syntax they are easy. And the most important thing: "tar" is always there. Name a Live CD. No matter which one. I bet it has "tar" already installed! It's therefore easy to get the files back. Heck, even stupid WinZIP can open *.tar.gz files and get your files out if needed. :)

QuickStart looks nice, yes .... but what when your data are gone? And what if you don't have access to the Internet anymore and therefore can't download it again ... ?

Think of it. ;)

This is great, I have been in a similar situation with windows XP.

Can anyone volunteer way to restore when a disaster happens with few commands for newbie like me can follow?

thanks,

gychang

scorp123

December 17th, 2008, 04:46 AM

Can anyone volunteer way to restore when a disaster happens with few commands for newbie like me can follow?
How about this very thread right here? :D
http://ubuntuforums.org/showthread.php?t=35087
.
.
.
.

gychang

December 17th, 2008, 08:24 AM

How about this very thread right here? :D
http://ubuntuforums.org/showthread.php?t=35087
.
.
.
.

should've read the first part of this thread, thanks,

gychang

vanadium

December 17th, 2008, 08:41 AM

Originally Posted by BassKozz View Post
Is QuickStart better/worse then the method described in this thread?
...<snip>...
Are you really sure you want to rely on a tool that most likely won't be around once a disaster happens?

I want to bring to the attention that Quickstart is not Free software (no open source) so you cannot inspect on beforehand what havoc it is going to create on your system. Another very good reason to stay away from it. Unfortunately, ubuntugeek is giving the tool publicity right now.

raynard

December 17th, 2008, 02:46 PM

This tutorial says it is unwise to backup the other mounted drives as well.
By default, I have a cdrom and another hard drive mounted in the folder 'media'.
Could I just exclude the media-folder, or would that give problems when I restore the system?

scorp123

December 17th, 2008, 06:44 PM

This tutorial says it is unwise to backup the other mounted drives as well. Yes. For obvious reasons. You don't want your backup to get bigger than really necessary.

Could I just exclude the media-folder Yes.

or would that give problems when I restore the system? No, it shouldn't. Unless you do weird things with those folders? I once had a case where someone mounted an external HD into a weird non-standard location and moved half his Linux installation there because he was running out of disk space ... But because he was using symbolic links (which he had not mentioned before) his system was still working ... somehow. But because he did not think of his "special design" the backup of course went horribly wrong ...

So if you just happen to have a CD and other stuff inside your /media folder because you use that stuff a lot but other than that it has no connection and relation whatsoever to the operating system .... then there should be no problem. You restore the system, and then mount your stuff there again when everything is done.

raynard

December 18th, 2008, 05:50 AM

So if you just happen to have a CD and other stuff inside your /media folder because you use that stuff a lot but other than that it has no connection and relation whatsoever to the operating system .... then there should be no problem. You restore the system, and then mount your stuff there again when everything is done.

Yes, but, the cdrom and external hd are automatically mounted when I boot up the pc, and only contain data which isn't necessary to run the system. I'm worried that, if I exclude the /media folder, they won't be mounted by default anymore, and I have absolutely no clue how to mount them myself.

scorp123

December 18th, 2008, 05:20 PM

I'm worried that, if I exclude the /media folder Syntax ... it matters! Are you excluding ...

/media
- OR -
/media/*

... ? Nope, the two do not mean the same!! If you exclude "/media" then the folder and everything underneath it is excluded. So when you restore the folder will be missing. ... No big deal if it happens, you can simply recreate it again if needed (e.g. via this command "sudo mkdir /media" ), but that's how it is. But if you exclude "/media/*" then you only exclude the stuff underneath that folder, but not the folder itself. When you restore your backup you would then get an empty folder "/media" again ... and I guess this is your intention?

they won't be mounted by default anymore Your assumption is most likely correct. The system would likely complain about a missing mount point :)

and I have absolutely no clue how to mount them myself. No big deal and easy to fix. The fix would be to simply create that folder again ... from there on Ubuntu would do all it's magic again. Something like this is easily fixed. :)
.
.
.

raynard

December 19th, 2008, 04:33 AM

OK, so if I exclude /media/* ,and when restoring just recreate a folder named 'cdrom0' and 'hdb1', everything should work like before ? I thought these folders were dynamic links to other files ?

scorp123

December 19th, 2008, 04:54 AM

and when restoring just recreate a folder named 'cdrom0' and 'hdb1' Ubuntu should do that automagically.

If you really want to be on the safe side, make a copy of this file: /etc/fstab

And write down the output this command spits out (copy & paste please) now that everything is working as it should: mount

With the two infos combined it's easy to get disks mounted again correctly just in case they won't mount anymore automatically.

scorp123

December 19th, 2008, 05:12 AM

BTW, mounting stuff manually isn't too hard. Take a look at the manual (yes, most if not all Linux distros ship with a detailed user manual!):
man mount (... use cursor keys to navigate, use "Q" to quit).

The procedure to mount an (external) harddisk partition could be e.g.
sudo fdisk -l ... this should spit out a list of all partitions that were detected. If I do that here I get this as reply:
> sudo fdisk -l
[sudo] password for sysadm:

Disk /dev/sda: 640.1 GB, 640135028736 bytes
255 heads, 63 sectors/track, 77825 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x1549f232

Device Boot Start End Blocks Id System
/dev/sda1 * 1 20727 166488472 7 HPFS/NTFS
/dev/sda2 22523 22539 136552+ 83 Linux
/dev/sda3 22540 77825 444084795 5 Extended
/dev/sda4 20728 22522 14418337+ 7 HPFS/NTFS
/dev/sda5 22540 22788 2000061 83 Linux
/dev/sda6 22789 23784 8000338+ 83 Linux
/dev/sda7 23785 24282 4000153+ 83 Linux
/dev/sda8 24283 25278 8000338+ 83 Linux
/dev/sda9 25279 26274 8000338+ 83 Linux
/dev/sda10 26275 76825 406050876 83 Linux
/dev/sda11 76826 77825 8032468+ 82 Linux swap / Solaris

Partition table entries are not in disk order

Disk /dev/sdb: 640.1 GB, 640135028736 bytes
255 heads, 63 sectors/track, 77825 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x9d8754c4

Device Boot Start End Blocks Id System
/dev/sdb1 1 77825 625129281 83 Linux

Disk /dev/sdc: 400.0 GB, 400088457216 bytes
255 heads, 63 sectors/track, 48641 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xf2427359

Device Boot Start End Blocks Id System
/dev/sdc1 1 48641 390708801 83 Linux

Disk /dev/sdd: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xa60fad0b

Device Boot Start End Blocks Id System
/dev/sdd1 1 60801 488384001 83 Linux

So that's the partitions that the system knows about. Now let's check which of them is mouted already -- so I'd type "mount" without arguments:
> mount

/dev/sda5 on / type xfs (rw,relatime)
/proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
varrun on /var/run type tmpfs (rw,nosuid,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /proc/bus/usb type usbfs (rw,devgid=46,devmode=664)
/dev/sda2 on /boot type ext3 (rw,relatime)
/dev/sdb1 on /data type xfs (rw,relatime)
/dev/sda10 on /home type xfs (rw,relatime)
/dev/sda8 on /opt type xfs (rw,relatime)
/dev/sda9 on /tmp type xfs (rw,relatime)
/dev/sda6 on /usr type xfs (rw,relatime)
/dev/sda7 on /var type xfs (rw,relatime)
securityfs on /sys/kernel/security type securityfs (rw)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)

If you find that output too confusing (I sure do!), you could also try a command such as "df -h" ... The actual purpose of that program is to print out the remaining free disk space on all mounted partitions. But it just so happens to also produce a nice list of all currently mounted partitions: > df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda5 1.9G 240M 1.7G 13% /
tmpfs 4.0G 0 4.0G 0% /lib/init/rw
varrun 4.0G 120K 4.0G 1% /var/run
varlock 4.0G 0 4.0G 0% /var/lock
udev 4.0G 3.0M 4.0G 1% /dev
tmpfs 4.0G 184K 4.0G 1% /dev/shm
/dev/sda2 130M 30M 94M 24% /boot
/dev/sdb1 597G 472G 125G 80% /data
/dev/sda10 388G 60G 329G 16% /home
/dev/sda8 7.7G 4.2M 7.7G 1% /opt
/dev/sda9 7.7G 4.3M 7.7G 1% /tmp
/dev/sda6 7.7G 4.4G 3.4G 57% /usr
/dev/sda7 3.9G 904M 3.0G 24% /var

And now, when we compare the output of "fdisk -l" against the one from "mount" or "df -h" we can easily spot the partitions that were not mounted, for whatever reason (in my case: I simply don't want them to be mounted :) )

So we'd pick one not on the list of the mounted partitions (the system would complain otherwise) and then mount it manually wherever we want:
sudo mount /dev/sdd1 /path/to/my/mountpoint ... mount should be able to auto-detect the correct filesystem type these days.

The same syntax can be used against CD's or DVD's. Usually the device name for the first CD or DVD drive is /dev/scd0 (that name is a leftover from the "old days" when CD burners had to be SCSI or had to use SCSI emulation or else writing CD's would not work ... : SCSI CD drive #0):
sudo mount /dev/scd0 /path/to/the/mountpoint

In rare circumstances it might happen that "mount" is unable to guess the correct filesystem type (corrupt disk maybe? Or badly formatted?) so you have to supply the type you think is correct as argument:
sudo mount -t ext3 /dev/sdd1 /path/to/mount/point

If you want to know all the filesystem types that are supported on Linux, take a look at the manual:man mount ... there is a large list of filesystems Linux can read from and in most cases also write to.

halovivek

December 19th, 2008, 06:18 AM

Thanks for posting. i will try to do backup and i will get back to you.

raynard

December 19th, 2008, 08:44 AM

BTW, mounting stuff manually isn't too hard. Take a look at the manual (yes, most if not all Linux distros ship with a detailed user manual!)
...

Thanks, that really helped me out! :D

mamboze

December 21st, 2008, 03:31 AM

I checked out this link

I personally use this method (which is similar to what's posted on the first page of this thread):
http://www.linuxmint.com/forum/viewtopic.php?f=42&t=3969&start=0&st=0&sk=t&sd=a

Thanks scorp123, this is exactly what I want to do, no GUI dependence, bare metal restore if necessary. But the fstab file on my Hardy box has this:

# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
# /dev/sda1
UUID=59c42850-e5a7-4699-bce5-6344aca2b9c3 / ext3 relatime,errors=remount-ro 0 1
# /dev/sda3
UUID=90ac031c-d3cd-4d24-8e2b-479f045b7a97 /boot ext3 relatime 0 2
# /dev/sda8
UUID=352d6f9d-bf8e-4dfd-8952-2b591231c453 /home ext3 relatime 0 2
# /dev/sda5
UUID=63930592-076c-4ec6-a667-a07973227514 /opt ext3 relatime 0 2
# /dev/sda7
UUID=63f1c7de-ee1a-4f01-b97e-e8d49dd514b2 /tmp ext3 relatime 0 2
# /dev/sda6
UUID=a4bc4004-6769-446a-9580-c373b348aab1 /var ext3 relatime 0 2
# /dev/sda2
UUID=290741df-79da-4f79-b710-d922a1b22612 none swap sw 0 0
/dev/scd1 /media/cdrom0 udf,iso9660 user,noauto,exec,utf8 0 0
/dev/scd0 /media/cdrom1 udf,iso9660 user,noauto,exec,utf8 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto,exec,utf8 0 0

How do I replace the UUIDs with device names, as suggested in the above link. Hardy has been installed for about 6 months. I'm still a relative noob on Ubuntu, any help would be welcome.

scorp123

December 21st, 2008, 01:03 PM

... But the fstab file on my Hardy box has this:

# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
# /dev/sda1
UUID=59c42850-e5a7-4699-bce5-6344aca2b9c3 / ext3 relatime,errors=remount-ro 0 1

How do I replace the UUIDs with device names .... Switch the red parts above with the blue parts, but make sure you keep the green parts "as is". And make sure you do this for each line that starts with "UUID=....-something". Ubuntu -nice as it is- wrote the old-style device names above each such line, so it's easy to find out. Another way of finding out would be commands such as "mount" (without argument) or "df -h" ... these commands will list currently mounted partitions using the device names (df's actual purpose is to list the free remaining disk space ... but it does the other job too :) )

sarang

December 21st, 2008, 01:09 PM

But why would you ever want to replace UUID's with names like /dev/sda1? UUID's have many advantages and if you want to add entries, you should use the

sudo vol_id -u /dev/sda1

to find out the device UUID. Even if you add UUID's to fstab, you can still mount the device by specifying its device name, like mount /dev/sda1 /media/sda1 .

sarang

December 21st, 2008, 01:14 PM

And yes, take a look at:

HowTo: Automatically launch a backup when a particular external drive is connected (http://ubuntuforums.org/showthread.php?t=984671)

I think it is very stable but it would be awesome if more people tested it.

scorp123

December 21st, 2008, 03:55 PM

But why would you ever want to replace UUID's with names like /dev/sda1? If you had read the link above you would know, it's all explained there.

UUID's suck if you don't know how to handle them. And newbies usually don't and then get panicky and confused about why their restore did not work. I have seen it too many times already. Or just go a few pages back where someone had the same problem.

scorp123

December 21st, 2008, 04:05 PM

... Even if you add UUID's to fstab ... Your harddisk just went to Nirvana. 500 GB of data lost. Your heartbeat is as high as if you were participant in an Olympic Marathon contest. You're sweating like mad. All you can think of is the harddisk, the harddisk, the harddisk, the harddisk ....

And for some stupid reason the live CD you wanted to use does not work, all you have is this Mini-Linux (e.g. old Gentoo install disk maybe?) which drops you right into a shell. No mouse. No GUI. Just you and "bash" and the command line ....

"Even if you add UUID's to fstab ..." <== Imagine yourself in the situation above: harddisk gone, you're sweating like mad, you're panicked ... and you will see that your statement is not realistic.

Chances are that an average "Joe User" who just lost his harddisk will _NOT_ know how to get the UUID's from the command line back into /etc/fstab again .... And he will most likely totally forget that he has to edit /boot/grub/menu.lst too and then execute "sudo update-grub" or else the boot process will fail too ..... :D

Hence my suggestion to remove UUID's ..... As far as backups and restores are concerned they suck big time and create a lot of hassle for the average user. The guide I once wrote was written with that in mind.

If you think you can handle UUID's ... please, by all means use them. If you can handle them ... fine.

My personal opinion is they create far too much trouble and offer no real benefit.

sarang

December 21st, 2008, 10:07 PM

If you think you can handle UUID's ... please, by all means use them. If you can handle them ... fine.

My personal opinion is they create far too much trouble and offer no real benefit.

Well, I have often derived great utility from UUID's for automating disk-related tasks so I disagree about the 'no real benefit' part. I have also found them to be very useful in dealing with raid configuration after juggling drives around. As far as backups are concerned, here is what I think: I don't believe in having to back-up by hand - I get too lazy and it does not happen as often as it should. Also, the article you cite is mistaken: for new disks, you do not need to change fstab or grub.conf/menu.lst entries - you just set the UUID of the new disk to that of the old one using tune2fs. Thus if one is going to use scripts any way, it does not take much additional effort to code in the tune2fs part in the scripts. But once done, the scripts remain virtually failure-proof. The assignment of device names does not involve any contract which in my opinion makes them unsuitable for any automation. (I have had device names change because of BIOS updates and on one occasion, after a kernel update. I have also seen machines whose BIOS is quirky and drives get different device names at each reboot. Using device names in all these cases would have caused massive breakage.) I'd rather spend more time learning about UUID stuff than have to rely on luck that device names will remain the same. Especially if my scripts are going to have things like dd in them.

This is just my opinion, and of course you are free to disagree with it :). I just wish to point out that there was a reason most fstabs switched to UUID's and I feel it is well worth it to make yourself comfortable with them.

scorp123

December 22nd, 2008, 03:45 AM

I have also found them to be very useful in dealing with raid configuration after juggling drives around. Volume labels can do that too, e.g. you assign a volume label to your drive and then have them mounted via "label=TheirVolumeNameHere" in fstab. This stuff has been there for ages and it works. No need to mess around with those hexadecimal strings.

I don't believe in having to back-up by hand It's the only method that works reliably. Period.

Also, the article you cite is mistaken: for new disks, you do not need to change fstab or grub.conf/menu.lst entries - you just set the UUID of the new disk to that of the old one using tune2fs. "tune2fs" is for Ext2/Ext3 ... too bad if you used something else such as ReiserFS, XFS, JFS. So you would have to lookup the appropriate utility for your filesystem _before_ something bad happens (XFS: "xfs_admin") ... but which user does that? It's already a miracle if users care to make backups of their stuff. So that will not work. And again: you're not realistic, IMHO. If the harddisk really fails having to manually mess with UUID's is way too complicated and will be infinitely frustrating for the end-user, especially if they are new to Linux. With the device names you simply replace the dead disk with the new one, voila done. It will just work. Or if you use volume labels, that's fine too: Just give the new disk the same volume label your old disk had, e.g. "RootDisk" ... that's easy, and you don't have to mess with that hexadecimal UUID gibberish.

it does not take much additional effort to code in the tune2fs part in the scripts. And what if I use XFS? What if someone uses ReiserFS? Now it gets complicated again .... :D

But once done, the scripts remain virtually failure-proof. Please feel free to write a tutorial of your own with your suggestions of optimisation included. As Linus Torvalds uses to say: "Talk is cheap. Show me the code!" ):P

:D

sarang

December 22nd, 2008, 02:22 PM

Volume labels can do that too, e.g. you assign a volume label to your drive and then have them mounted via "label=TheirVolumeNameHere" in fstab. This stuff has been there for ages and it works. No need to mess around with those hexadecimal strings.

True, I agree labels are a good compromise. Redhat seems to use them very effectively and they are good enough for me. UUID's are a more extreme version with much lesser chance of namespace collisions..

It's the only method that works reliably. Period.

I don't think so. I know lots of servers that are not backed up by hand.

"tune2fs" is for Ext2/Ext3 ... too bad if you used something else such as ReiserFS, XFS, JFS. So you would have to lookup the appropriate utility for your filesystem _before_ something bad happens (XFS: "xfs_admin") ... but which user does that? It's already a miracle if users care to make backups of their stuff. So that will not work. And again: you're not realistic, IMHO. If the harddisk really fails having to manually mess with UUID's is way too complicated and will be infinitely frustrating for the end-user, especially if they are new to Linux. With the device names you simply replace the dead disk with the new one, voila done. It will just work. Or if you use volume labels, that's fine too: Just give the new disk the same volume label your old disk had, e.g. "RootDisk" ... that's easy, and you don't have to mess with that hexadecimal UUID gibberish.

As I said, labels are a good compromise in most cases, but if I am going to use something like dd in a script, I would rather use UUID's to further minimize name collisions. Also, if you are going to use labels, I think using UUID's is equivalent. You agreed that you can give the new disk a label with insignificant effort. I do not see how assigning a UUID is significantly more difficult - for me at least it is a matter of copy and paste from the old fstab. About the hexadecimal gibberish: when you download the iso for an iso etc, do you not check the MD5 / SHA2 sums to verify that you have the right ISO? UUID strings have similar length and involve a similar level of gibberishness.

And what if I use XFS? What if someone uses ReiserFS? Now it gets complicated again .... :D

Your argument against UUID's was primarily that they are difficult for newbies to use. I don't think many newbies use XFS or ReiserFS. And if a non-noob wants to use those filesystems, it would be wise for him/her to familiarize with fs utilities that are equivalent to tune2fs.

Please feel free to write a tutorial of your own with your suggestions of optimisation included. As Linus Torvalds uses to say: "Talk is cheap. Show me the code!" ):P
:D

See: http://ubuntuforums.org/showthread.php?t=984671
(http://ubuntuforums.org/showthread.php?t=984671)

Again: I agree that UUID's are more difficult to deal with, but I do not agree with the way you totally debunk them as useless.

scorp123

December 22nd, 2008, 05:31 PM

True, I agree labels are a good compromise. Redhat seems to use them very effectively and they are good enough for me. UUID's are a more extreme version with much lesser chance of namespace collisions.. OpenSUSE and newer releases of SLES use the "Disk by ID" method, so their "fstab" entries will default to something like ... /dev/disk/by-id/scsi-1ATA_WDC_WD5000BEVT-22ZAT0_WD-WXNX08LPA864-part2 ... which too may make sense or not. But at least both OpenSUSE and SLES let you select for each partition how you want it mounted, e.g. by device name, by device path, by volume name, by UUID, etc.

If you know what you do you can mix and match the methods. Fine. But my "backup HowTo" was written with newbies in mind and chances are they get confused. So for the sake of "keep it simple and stupid" I prefer device names.

I don't think so. I know lots of servers that are not backed up by hand. Define "backed up by hand" please? I don't think we mean the same. What I meant was: "reliable method that will get the system back, GUI or not, bare metal or not". I did not mean that I sit next to a server and watch how it performs a backup ... heck, noooo. That's what SSH, "cron" and scripting is good for. If anything went wrong I will get an e-mail from cron telling me so. But if anything goes horribly wrong I am sure that I will get my data back. "tar" is always there, there are no dependencies with other libraries or GUI tools which you'd have to download first. So for as long as I can get into at least a rescue shell I am sure I can get my data back, be that from external USB drives (e.g. at home) or from my LTO tapes (e.g. at work).

All GUI backup solutions I looked at so far have one major problem: chances are that when disaster strikes they are not around anymore ... so having to use them for a restore is maybe even impossible; hence a backup done with such tools is basically worthless.

The same is true for any almost obscure non-standard tools that sometimes get suggested, e.g. "pax", "star", and some others ... None of those tools is in wide use, and chances are that you won't have them ready and available when disaster strikes. So same as above: any backups done with those tools is potentially worthless if you don't have the tools anymore to perform the restore ....

Hence my insistence on using "tar" for backups. Chances are very high that you will find "tar" no matter what distro you use. So you will be able to get the data back. That's ultimately what defines if a backup was successful or not: a successful restore! :D

I do not see how assigning a UUID is significantly more difficult Imagine being stuck on a GUI-less shell. There is just you, bash and good old "vi". Maybe not even "vim" but something old-style like "vim-tiny". So how exactly is a newbie supposed to get that frustratingly long UUID back into the right places again? There is no GUI, no easy "copy & paste" aside from what "vi" offers ... Volume labels are way easier to remember. You can simply and easily assign them when you format your new harddisk, you just have to make sure it's the same volume label as before, and that's it.

for me at least it is a matter of copy and paste from the old fstab. And now please imagine worst case scenario: You're stuck on a minimum shell. There is no GUI here. There is no copy & paste. And voila, that's where the problems start with those UUID's.

About the hexadecimal gibberish .... That's how it looks to newbies.

when you download the iso for an iso etc, do you not check the MD5 / SHA2 sums to verify that you have the right ISO? Only when I have to download stuff from Novell.com .... their servers are sometimes slooooow and act weird ... :lolflag:

I don't think many newbies use XFS or ReiserFS. You're mistaken here. Go and take a look in the forums. There are enough discussions where newbies find Ext3's speeds insufficient and they then come and ask what other filesystem they should try .. and often enough and depending on what they have on their disks they then use ReiserFS or XFS. You can't simply assume that everybody is using Ext3, even if they are newcomers.

See: http://ubuntuforums.org/showthread.php?t=984671
(http://ubuntuforums.org/showthread.php?t=984671) OK, I am sure that there are people here who might find this interesting and useful. Thanks for posting that.

I agree that UUID's are more difficult to deal with .... That's my point exactly \\:D/

;)

bgs100

December 24th, 2008, 12:39 AM

Hey, based on this guide, I have made a GUI zenity shell script that does what this guide describes. It is called shBackup and it supports both tar (tgz) and bzip2 compression
Hope you enjoy!
Deb package:
101347

Launchpad page: https://launchpad.net/shellbackup

Edit:
Update.
"the variable $HOME is different between normal user and root" Bug Fixed.
New terminal options added
"Delete old backup files option" bug fixed

IB1Dance

December 24th, 2008, 05:27 PM

Just unzip and allow running as executable. a .deb coming soon!

nice & simple thX

backing up now in process :-)

and because of the previous posts I have some idea how shBackup is actually working ... nuice one dudes N dude'es's

bgs100

December 24th, 2008, 06:58 PM

nice & simple thX

backing up now in process :-)

and because of the previous posts I have some idea how shBackup is actually working ... nuice one dudes N dude'es's

Hey your welcome!!! Now there's a .deb package too. Just check back at the post where you got the tar.gz from

cigtoxdoc

December 24th, 2008, 07:41 PM

bgs100

December 25th, 2008, 02:15 PM

Very timely as I need to do a totally backup of my "home" PC and reinstall it on my "vacation" PC and as I trashed the latter during Thanksgiving. I have about 4.7 GB on hard drive on "home" system. I want to write all of that to a DVD (never did under Ubuntu). I want to put DVD in a case and drive to vacation home, fire-up "vacation" PC, put DVD in player and boot the system from DVD and restore. Nothing to save on "vacation" PC's hard drive. Vacation PC will not have an Internet connection until I do the restore and use my USBConnect881 air card.

How do I do the above in terms of commands I need to use?

John

Well, to do that in the way you described you would have to make a customized ubuntu cd with your personal data. But, you could just install a normal ubuntu (without personal data) on "vacation" PC and then put backup file on "vacation" PC and restore on the "vacation" PC

cigtoxdoc

December 26th, 2008, 12:02 AM

Maybe another approach would be to make a DVD with all the deb files I need (everthing that would normally load with 8.10 plus all networking and dial-up. How do I do this and get everything to have trhe right dependencies? As I understand the Ubuntu, I could boot PC with 8.10 CD and load OS. I would then set sources.list to point towards the DVD drive. I could then install software from the DEB files on the DVD. Is this practical? Only other alternative is to get work with my external Zoom V92 modem.

Is there a better way that gets around having to have an internet connection?

John

PoPpiLLs

December 26th, 2008, 04:26 AM

cigtoxdoc take a look at APTonCD http://aptoncd.sourceforge.net/

Haluci

December 26th, 2008, 03:59 PM

Thanks for the howto! I just have a couple questions about it:

1. I'd like to create the .tgz file on an external hard drive (which I know how to mount). How would I type that in the tar command?

2. I have two partitions on my hard drive I'd like to backup: an unbuntu partition and a windows partition. Would the procedure be the same for both (I'll exclude the... exclude options on the windows partitions)?

Thanks for the help!

bgs100

December 26th, 2008, 06:08 PM

Thirtysixway

December 26th, 2008, 09:21 PM

Will this work if moving your system to a new, larger harddrive?

bgs100

December 27th, 2008, 10:02 AM

bgs100

December 29th, 2008, 11:40 PM

New version of shbackup available.

BassKozz

December 31st, 2008, 12:04 AM

New version of shbackup available.

shbackup?

thomasr

January 1st, 2009, 02:13 PM

Another nice method is Remastersys

http://www.remastersys.klikit-linux.com/

pshootr

January 1st, 2009, 07:46 PM

Thanks for the tut. :) I did a backup and I got this message at the end. (tar: /: file changed as we read it) Is this normal? Thanks

Haluci

January 2nd, 2009, 02:04 PM

tar cvpzf {Insert External Hard Drive Location Here (example: /media/Maxtor/ or maybe /mnt/Maxtor/) and include slash at end}backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

Thanks! So, if I'm backing up a partition from a live cd, would I have /dev/sda2 (for example) at the end or would I be better off navigating to the root of both partitions?

El_Belgicano

January 2nd, 2009, 04:29 PM

Thanks for the tut. :) I did a backup and I got this message at the end. (tar: /: file changed as we read it) Is this normal? Thanks

You probably were executing a program and it was writing actively to a file who was being read for tar'ing.

I'd say: back up again (just for safety) but try to have a minimal number of programs running. (I was getting this when tar'ing my home folder with firefox running...)

If you still get the warning, try from a LiveCD.

Thomas

pshootr

January 2nd, 2009, 05:55 PM

You probably were executing a program and it was writing actively to a file who was being read for tar'ing.

I'd say: back up again (just for safety) but try to have a minimal number of programs running. (I was getting this when tar'ing my home folder with firefox running...)

If you still get the warning, try from a LiveCD.

Thomas

Thank you. I will take your advise and back up again. I have samba running and ssh but that is about it. This is a server install so no firefox or any gui.

pshootr

January 2nd, 2009, 07:37 PM

scorp123

January 2nd, 2009, 07:48 PM

I did a backup of my servers 4 gig drive. If I pull out that drive, and put in three 40 gig drives to create an array with. Will I be able to use the backup on my 4 gig drive to move my server setup to the new raid array? Yes, but pay attention when you restore! /etc/fstab and /boot/grub/menu.lst must not be overwritten with the versions in your backup, you have to keep the new versions which reflect the new disk setup or you won't be able to boot afterwards.

pshootr

January 2nd, 2009, 08:09 PM

Yes, but pay attention when you restore! /etc/fstab and /boot/grub/menu.lst must not be overwritten with the versions in your backup, you have to keep the new versions which reflect the new disk setup or you won't be able to boot afterwards.

Ok cool. Thanks alot for the heads up.

any.linux12

January 5th, 2009, 08:15 AM

Hi!

The howto is good but there are somethings that I would like to know.

1- Is it possible to backup just the data that have been changed?

2- Is it possible to backup to a FTP server somewhere in the world (by mounting it in a folder)?

3- Do it everyday at night

thanks in advance

any.linux12

January 5th, 2009, 08:45 AM

And what means tar: /: file changed as we read it
tar: Error exit delayed from previous errors

El_Belgicano

January 5th, 2009, 04:41 PM

And what means

See:

You probably were executing a program and it was writing actively to a file who was being read for tar'ing.

I'd say: back up again (just for safety) but try to have a minimal number of programs running. (I was getting this when tar'ing my home folder with firefox running...)

If you still get the warning, try from a LiveCD.

Thomas

the second warning is something else... probably too much sockets or read errors...

cglasier

January 7th, 2009, 11:20 AM

Please excuse me if this has been asked already but 82 pages is a lot to go through and search didn't find anything. Can the /etc/privoxy directory be excluded as well. I keep getting a “file changed as we read it” error on /etc/privoxy/user.action and a “Error exit delayed from previous errors” error on /etc/privoxy/config~.

Thanks for the help

El_Belgicano

January 7th, 2009, 04:01 PM

bgs100

January 7th, 2009, 08:20 PM

shbackup?
See my post here: http://ubuntuforums.org/showthread.php?p=6427725#post6427725

any.linux12

January 8th, 2009, 05:57 AM

Hi!

I want to know if is possible to check the diference between 2 folders and save the file that have been modified to a third one.

Like if I have a Monday folder (with a full backup of my system) and on tuesday night I want a script that compare the content of my data souce (for ex. /fyle-server/*) and save (just the diferences) on the folder Tuesday.

Should be very nice to do this. Please comment this idea!

KC1117

January 10th, 2009, 10:07 PM

I use Ubuntu 8.04. My problems might not be related to the attempt at a backup, but they started right after. I am new to linux so probably did something wrong, but don't know how to pinpoint my problem. Here are my symptoms: 1)firefox settings (bookmarks, extras) gone, 2)keep getting a message OpenOffice is still open and can't be used when I log in, 3) when I open OpenOffice I get a message about not having enough space in a hidden openoffice.org backup file, 4) when I hit the main logout symbol it goes directly to shutdown and bypasses the switch user, shutdown screen and freezes, and 5) when I hit the button to shut it off manually I get a bunch of messages that move to quickly to tell you specifically, but I catch the words, "Network Manager" frequently.

Do you have any suggestions on where to start or what diagnostic messages I might be able to retrieve? Thank you.

El_Belgicano

January 11th, 2009, 02:46 AM

Stevo2979

January 11th, 2009, 06:01 AM

first of all, thanks for this tut. as a new user of linux I've been looking for a simple way to save all my stuff. :KS

second (& the reason why I'm replying), I did the backup as it said to in the first post using the tar.bz2 method & it seem to back up everything fine. the problem is now I have a 8gb backup in my root drive & want to move it to my external & when I tried moving it after getting 4gb done i get a error message saying the file is too large even though i have like 65bg of free space on the external.

Can anyone tell me why I'm getting this message & how to fix the problem plz?

B.T.W. sorry if someone already answered this question. I read upto page 54 & did a search & couldn't find what i was looking for.

KC1117

January 11th, 2009, 09:30 AM

After "making a backup"/"restoring a backup"?
Also, what precise command did you use?

I say "attempted" because in the middle of the backup it said there wasn't enough space on the external mini maxtor and quit. I honestly didn't know what to do to continue so I cancelled it in the middle. I have looked at the mini maxtor and it has plenty of room.
You guys that know what your are doing probably roll your eyes a few times at us newbies, but thats okay with me, I really appreciate the help.

Also, my backup was taking over an hour. Is that right? If not, I must have either picked up the wrong version code when I copied or I didn't properly exclude something.

El_Belgicano

January 11th, 2009, 04:48 PM

I say "attempted" because in the middle of the backup it said there wasn't enough space on the external mini maxtor and quit. I honestly didn't know what to do to continue so I cancelled it in the middle. I have looked at the mini maxtor and it has plenty of room.
You guys that know what your are doing probably roll your eyes a few times at us newbies, but thats okay with me, I really appreciate the help.

Also, my backup was taking over an hour. Is that right? If not, I must have either picked up the wrong version code when I copied or I didn't properly exclude something.

No problems, every ubuntu-wiseguy has once been a noob to others and probably still is (to less people though)...

First I can come up with is that you might have forgotten to exclude some mounted file system (/media or /mnt).
My backup time is 1h15 all together...

El_Belgicano

January 11th, 2009, 04:53 PM

Stevo2979

January 11th, 2009, 10:42 PM

If your external drive is formatted FAT32 it has a filesize limit of 4Gb so if this is the case and the drive is empty, reformat to NTFS or ext3 (examples)...

Thanks for the help. :D I was able to reformat it to ext3 & it fixed everything.

KC1117

January 11th, 2009, 11:04 PM

No problems, every ubuntu-wiseguy has once been a noob to others and probably still is (to less people though)...

First I can come up with is that you might have forgotten to exclude some mounted file system (/media or /mnt).
My backup time is 1h15 all together...

Should I just redo the backup commands or something else?

El_Belgicano

January 12th, 2009, 03:04 AM

Thanks for the help. :D I was able to reformat it to ext3 & it fixed everything.

You're welcome :p

Should I just redo the backup commands or something else?

Post them here, we'll have a look...
the safest is to exclude both /media and /mnt (me thinks, used&worked)

Thomas

adamlau

January 12th, 2009, 03:40 AM

I prefer dar with a configuration file. Here is sample of my /home/foobar/.darrc:

-m 128
-s 650M -D
-R /home/foobar
-g Backups
-g Documents
-g Templates
-Z "*.bz2"
-Z "*.gz"
-Z "*.jpg"
-Z "*.png"
-Z "*.tar"
-Z "*.tgz"
-Z "*.zip"
-X "*~"
-X ".*~"
-X "*.dar"
-y
-v

Here is a sample of a quick copy & pasted command line to backup,verify, and then create parity files for the backup:

dar -c OptiPlex-Xubuntu-Docs-`date -I` && dar -Nt ~/OptiPlex-Xubuntu-Docs-`date -I` && par2 create -n1 -u ~/OptiPlex-Xubuntu-Docs-`date -I`.par2 ~/OptiPlex-Xubuntu-Docs-`date -I`*.dar

KC1117

January 12th, 2009, 12:28 PM

Post them here, we'll have a look...
the safest is to exclude both /media and /mnt (me thinks, used&worked)

Below this paragraph is the commands I got from putting in "history" at the terminal. It looks strange to me and this is why:
Immediately before I did the backup in the same day, I was following this guide: Comprehensive Multimedia & Video Howto,
http://ubuntuforums.org/showthread.php?t=766683
You have to select the code and commands based on what system you have and I'm wondering if I picked up the wrong thing and didn't know and that is also contributing to the problem. I did not get my dvd player to work. I decided I was going to have to take my laptop to get the DVD player fixed as it wouldn't work on the Vista side of my dual boot either and was making weird noises and looking like a hardware issue not a software issue. That is when I followed the backup tutorial and it stopped midstream. I took the laptop in and they cleaned it and clicked on a few things in the Windows OS, but never went into the Ubuntu side. The laptop never left my sight so I don't think they did anything to it.

So after line 272 there should be some commands related to the backup and I don't see them. The last thing 2 commands I see are the ones related to the dvd installation. You can see also on 272 that I picked up the Intrepid commands instead of the Hardy commands. I realized that mistake and redid the commands with hardy instead of intrepid, but you can't see those commands either.

I also think from memory and reviewing the commands on the first post of this thread that when I may not have correctly excluded the media folder eventhough I tried. I just can't find exactly what those commands were and am confused enough now I can't remember. I know the /mnt folder was excluded because I copied it exactly from the instructions. Is there another place to get the commands and/or the output?

257 sudo fc-cache -f -v
258 sudo apt-get remove gnash gnash-common icedtea-gcjwebplugin libflash-mozplugin libflashsupport mozilla-plugin-gnash openjdk-6-jre openjdk-6-jre-headless openjdk-6-jre-lib swfdec-mozilla && sudo apt-get install alsa-oss faac faad flashplugin-nonfree gstreamer0.10-ffmpeg gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-multiverse gstreamer0.10-pitfdll liblame0 non-free-codecs sun-java6-fonts sun-java6-jre sun-java6-plugin unrar
259 sudo apt-get remove kaffeine-mozilla mozilla-helix-player mozilla-mplayer mozilla-plugin-vlc totem-mozilla xine-plugin
260 sudo apt-get install gnome-mplayer gecko-mediaplayer
261 sudo apt-get install libdvdcss2 libdvdread3 libdvdnav4 vlc
262 sudo apt-get install gxine libxine1-ffmpeg
263 gksudo gedit /etc/hal/fdi/policy/shmconfig.fdi
264 dmesg
265 sudo regionset
266 sudo apt-get install build-essential debhelper fakeroot
267 sudo apt-get install regionset
268 sudo regionset
269 sudo apt-get install libdvdcss2 libdvdread3 libdvdnav4 vlc
270 gksudo gedit /etc/gnome/defaults.list
271 sudo su
272 sudo wget http://www.medibuntu.org/sources.list.d/intrepid.list -O /etc/apt/sources.list.d/medibuntu.list
273 wget -q http://packages.medibuntu.org/medibuntu-key.gpg -O- | sudo apt-key add - && sudo apt-get update
274 history

El_Belgicano

January 12th, 2009, 05:10 PM

<snap>
I also think from memory and reviewing the commands on the first post of this thread that when I may not have correctly excluded the media folder eventhough I tried. I just can't find exactly what those commands were and am confused enough now I can't remember. I know the /mnt folder was excluded because I copied it exactly from the instructions. Is there another place to get the commands and/or the output?

I'd see the commands in the first post as a model for you to play with, but since it's Linux, there 101 ways of doing things...
Personaly, I use a modified-to-my-needs set of commands:
tar cpzf /media/external500Gbdrive/backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/media --exclude=/etc/fstab --exclude=/home --exclude=/mnt --exclude=/sys /
excludes:
- /mnt: there might be something mounted here.
- /media: my 500Gb drive is mounted here and I'm backing up to it...
- /etc/fstab: easy to reinstal with a new partition table.
- /home: second basic backup command only for user data. (tar cpzf /pathtobkp/bkp.tgz /home/)
But that's me...

257 sudo fc-cache -f -v
258 sudo apt-get remove gnash gnash-common icedtea-gcjwebplugin libflash-mozplugin libflashsupport mozilla-plugin-gnash openjdk-6-jre openjdk-6-jre-headless openjdk-6-jre-lib swfdec-mozilla && sudo apt-get install alsa-oss faac faad flashplugin-nonfree gstreamer0.10-ffmpeg gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-multiverse gstreamer0.10-pitfdll liblame0 non-free-codecs sun-java6-fonts sun-java6-jre sun-java6-plugin unrar
259 sudo apt-get remove kaffeine-mozilla mozilla-helix-player mozilla-mplayer mozilla-plugin-vlc totem-mozilla xine-plugin
260 sudo apt-get install gnome-mplayer gecko-mediaplayer
261 sudo apt-get install libdvdcss2 libdvdread3 libdvdnav4 vlc
262 sudo apt-get install gxine libxine1-ffmpeg
263 gksudo gedit /etc/hal/fdi/policy/shmconfig.fdi
264 dmesg
265 sudo regionset
266 sudo apt-get install build-essential debhelper fakeroot
267 sudo apt-get install regionset
268 sudo regionset
269 sudo apt-get install libdvdcss2 libdvdread3 libdvdnav4 vlc
270 gksudo gedit /etc/gnome/defaults.list
271 sudo su
272 sudo wget http://www.medibuntu.org/sources.list.d/intrepid.list -O /etc/apt/sources.list.d/medibuntu.list
273 wget -q http://packages.medibuntu.org/medibuntu-key.gpg -O- | sudo apt-key add - && sudo apt-get update
274 history

No tar command in here... probably because it did not complete.

Try running your backup command again but without the verbose flag: "tar cpzf ..." (in place of "tar cvpzf ...") so only critical errors will be printed out...

Thomas

KC1117

January 14th, 2009, 02:38 PM

No tar command in here... probably because it did not complete.

Try running your backup command again but without the verbose flag: "tar cpzf ..." (in place of "tar cvpzf ...") so only critical errors will be printed out...

I got your suggestion and also found my backup command history. I think I figured out what I had wrong and will attempt another backup and report back. Thanks.

PaulWhipp

January 16th, 2009, 04:11 AM

I've used tar to backup along the lines described in this thread.

My latest full backup is a 27Gb tar. Fishing out a file using tar -xzpf backup20090116.tar.gz -strip 3 home/paul/Temp/adobe.reg lead to a series of 'tar: skipping to next header' messages and a very long wait followed by
'gzip: stdin: invalid compressed data--crc error'

Things work OK much of the time so this seems to be hit and miss unless anyone has better data.

Make sure you test your backups!

tar -txpf backup20090115.tar.gz >> backup20090115_list.txt

is a good test (and can be a useful list). Even if it succeeds, check the sanity of the list with something like tail backup20090115_list.txt and ideally test an extraction on one of the last files.

drjonze

January 16th, 2009, 03:08 PM

Hello,

I have used this method before but the set up I had at the time was the my / and /home directories were on the same partition. I am going to be migrating my /home directory to a new partition. Will this method also work for this type of setup? (assuming that I'm restoring to the same setup, that I have run the back up after I have migrated my /home directory and all of my important files are already backed up to DVDs, just in case). In this case, would I exclude /home from the back up, seeing as if I were trying a new distro or upgrade, I would only be touching my / directory and not touching the home partition?

I think I know the answers to my questions but I'd rather be safe than sorry. I've learned the hard way that taking 10 minutes to consult the forum can save me hours of work later.

Thanks! And thanks for this excellent guide, Heliode!

sakthidaran

January 18th, 2009, 05:37 AM

Hi, and welcome to the Heliode guide to successful backing-up and restoring of a Linux system! Most of you have probably used Windows before you started .............

Dear Heliode,

After the search and reading your post and replies, I am still not confident what to do in my case. My problem is given below:

I am new to Ubuntu. I have been using 8.10 for a month. Now, I want to migrate 70 computers in a college lab from MS Windows XP to Ubuntu. I have downloaded 8.10, configured the network, printers etc. I have also applied the updates till the recent one. All these I have done on two systems. I want to take the image of one computer and apply on all the remaining computers. The hard disk is 150 GB. There is only one Linux EXT3 and Swap partition on the disk. No user data is available. All the target machines are of the same type. The target machines can be totally reformatted. Now, I have the following questions:
1. What commands to use for back up (entire disk is not required. Since the partition is large, is it possible to take the image till the data is residing )?
2. If it is going to be more than one DVD size, then how to split the file?
3. How to copy the image on the target machine?
4. Is it required to reformat the target machine individually before loading Ubuntu image?

Thanks for reading the doc up to this!

Fizwidget

January 20th, 2009, 08:38 AM

Hey guys, new user here. I found this guide via google, and I thought I'd give it a try. I followed the instructions in the original post exactly, and it screwed up my system. I tried it a second time, and it screwed up my system again!

After performing the backup, I would get a black screen after rebooting. I seem to have fixed this by loading the recovery option from the GRUB bootloader, and repairing the xconfig (it was 'x' something, not 100% sure that was it!).

I can't be bothered reading through this massive thread to see if anyone else has experienced this. Can someone tell me what's going on?

Sharepointengine

January 20th, 2009, 11:21 AM

Really very great advice share here...
I appreciate for sharing all the information here !!
:KS

PaulWhipp

January 20th, 2009, 06:13 PM

Hi Fizwidget,

Welcome to Ubuntu!

It sounds like you have a video driver issue. These are a perennial problem on computers particularly when updating or restoring.

If your system is working fine now, stick with it.

If the video is unacceptably slow or can't handle the screen resolution you want to use, you may have a little card icon on the right hand side of your top panel that will allow you to install the 'restricted driver' for your particular video card.

Failing that, more details will be needed. For live help try the #ubuntu chat channel http://linuxowns.wordpress.com/2008/04/23/ubuntu-irc-channel/. It can be a bit bewildering at first but if you ask your question there and are patient (~5mins sometimes) you'll probably get someone who can help.

Fizwidget

January 20th, 2009, 07:03 PM

Thanks for the reply PaulWhipp. I have since formatted my computer (don't want to take any chances in case there were deeper problems caused!).

I am more interested in why this backup method caused my problem in the first place. The effect was repeatable, so it wasn't a fluke or coincidence. Following the instructions in the original post definitely causes me to get a black screen.

Here's some information about my setup:

* Specs: Core 2 Duo, 4GB RAM, ATI HD4850
* Ubuntu was fully updated and recommended graphics drivers were installed
* I had used this tweak to stop video flickering - http://ubuntuforums.org/showthread.php?t=723860
* I had Wine installed

So does anyone know why this backup method causes a black screen for me? I really liked the idea of bundling everything into a compressed file, but I'm too scared to try it again!

PaulWhipp

January 20th, 2009, 07:39 PM

As an informed guess:
Once your system is running, certain 'files' are in use. This includes files relating to the video driver.

When you restore the files, these files cannot be restored properly because you are effectively trying to lift yourself into the air by pulling on your bootlaces.

To fully restore the system with the extraction method indicated including all drivers etc., I think you will need to bite the bullet and work from a different image.

Booting the ubuntu CD and running from CD should do the trick. Using that image you could restore the / partition on your hard drive and everything should work as before.

You may need to use 'fdisk -l' to identify the target, then mount it, then restore into it.

Fizwidget

January 20th, 2009, 08:20 PM

Sorry, I should have been more clear. I wasn't trying to restore the backup. After performing the backup and rebooting my system, I got the black screen.

The process of backing up those files must have corrupted the originals somehow...?

PaulWhipp

January 20th, 2009, 08:56 PM

All you did was the 'tar c...' and the system rebooted differently?

If that is the case its likely something else has gone awry. I've used tar for years and it sometimes corrupts its archives (you have to test them) but its never affected the system its archiving.

Unless... its possible you filled up your drive and that caused some issue when rebooting.

bgs100

January 20th, 2009, 09:36 PM

New shbackup version.

Fizwidget

January 20th, 2009, 09:40 PM

I followed the instructions in the OP exactly. I entered these commands in this order:

sudo su

cd /

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

It would take maybe 20-30 minutes to finish, and would give me an error message at the end (I assumed this was normal). The backup file would be located in the root directory.

Upon rebooting the system, the ubuntu logo and loading bar would appear normally, followed by a black screen.

The drive was not full (had hundreds of gigabytes free).

PaulWhipp

January 20th, 2009, 09:58 PM

Sorry Fizwidget, I'm stumped then.

Errors are not normal though and the tar should not have reported an error. It might help if you could paste the result of:
tar cpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

Note that I've dropped the verbose flag. Errors should still show up but we'll avoid all the other stuff. This does mean its likely to just sit there for 20-30 mins without doing much.

Fizwidget

January 21st, 2009, 04:41 AM

Third time lucky :)

I tried the backup one more time, and it worked! This time I set the output directory to the desktop - the previous two times I had outputted to the root directory. Perhaps it screwed up because I left the backup file in the root directory when rebooting? Does this confuse Ubuntu somehow?

Oh well, I'm just glad it's working. Thanks for the help!

PaulWhipp

January 21st, 2009, 04:15 PM

Thats good news Fizwidget!

Writing to root directory should not have made any difference.

It could have done if the partitioning on your machine were unusual but it doesn't sound like that would be the case.

KC1117

January 24th, 2009, 09:45 AM

No tar command in here... probably because it did not complete.

Try running your backup command again but without the verbose flag: "tar cpzf ..." (in place of "tar cvpzf ...") so only critical errors will be printed out...

Thomas

Due to busy-ness of life and fear I haven’t done this yet. I did locate my original backup commands in a different directory than I had been looking:
ORIGINAL:
tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys / --exclude=/media

I thought the order of the commands were important so I changed to this and ran again. Still have the same symptoms and couldn't boot without getting stuck at BusyBox until I read a thread that helped me get past that. Now I just have the same symptoms in my original post.

Everything I try to do it tells me disk full or not enough space.

2ND ATTEMPT
tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=mnt --exclude=/media --exclude=/sys /

These are the comments from the terminal (that it would let me copy) from this latest backup attempt. IS IT SUPPOSED TO BE COPYING WINDOWS STUFF? I do have a dual boot system, but didn’t think it would try to backup both sides. IS THE CODE FOR THE BACKUP COMMAND SUPPOSED TO BE DIFFERENT IF YOU HAVE DUAL BOOT?

SOME OUTPUT FROM 2ND ATTEMPT

/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/CertificateServicesClient/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/Customer Experience Improvement Program/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/Defrag/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/DiskDiagnostic/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/Media Center/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/MobilePC/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/MUI/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/Multimedia/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/NetworkAccessProtection/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/PLA/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/PLA/System/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/RAC/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/RemoteAssistance/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/Shell/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/SideShow/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/SystemRestore/
/host/System Volume Information/SystemRestore/FRStaging/Windows/System32/Tasks/Microsoft/Windows/Tcpip/
.....
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/FileMaps/
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/Manifests/
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/msil_ehepg_31bf3856ad364e35_6.0.6000.16386_none_d9 6c76b2d1fd5d49/
....
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/x86_microsoft-windows-a..ence-mitigations-c5_31bf3856ad364e35_6.0.6000.16485_none_09ee78f1f5 5ea425/
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/x86_microsoft-windows-advpack_31bf3856ad364e35_6.0.6000.16609_none_a9ee2 96df5a1e10e/
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/x86_microsoft-windows-b..uggertransport-1394_31bf3856ad364e35_6.0.6000.16386_none_61949536 f6f76e24/
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/x86_microsoft-windows-cbsapi_31bf3856ad364e35_6.0.6000.16386_none_4c2b11 19f37be620/
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/x86_microsoft-windows-cmiadapter_31bf3856ad364e35_6.0.6000.16386_none_bb de780114480e84/
...
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/x86_microsoft-windows-commonlog_31bf3856ad364e35_6.0.6000.16386_none_7c4 bd8b12aa0f521/
/host/System Volume Information/SystemRestore/FRStaging/Windows/winsxs/x86_microsoft-windows-coreusermodepnp_31bf3856ad364e35_6.0.6000.16386_no ne_74cae93a3000e831/
...
/host/TOSHIBA/IVP/swupdate/swupdtmr.exe
/host/TOSHIBA/IVP/swupdate/Swupduni.dll
/host/TOSHIBA/IVP/swupdate/TaisSoftIcon.exe
/host/TOSHIBA/IVP/swupdate/tips.txt
/host/ubuntu/
/host/ubuntu/curl_log.txt
/host/ubuntu/disks/
/host/ubuntu/disks/boot/
/host/ubuntu/disks/boot/abi-2.6.24-16-generic
/host/ubuntu/disks/boot/abi-2.6.24-22-generic
/host/ubuntu/disks/boot/config-2.6.24-16-generic
/host/ubuntu/disks/boot/config-2.6.24-22-generic
/host/ubuntu/disks/boot/grub/
/host/ubuntu/disks/boot/grub/default
/host/ubuntu/disks/boot/grub/device.map
/host/ubuntu/disks/boot/grub/menu.lst
/host/ubuntu/disks/boot/grub/menu.lst~
/host/ubuntu/disks/boot/initrd.img-2.6.24-16-generic
/host/ubuntu/disks/boot/initrd.img-2.6.24-16-generic.bak
/host/ubuntu/disks/boot/initrd.img-2.6.24-22-generic
/host/ubuntu/disks/boot/initrd.img-2.6.24-22-generic.bak
/host/ubuntu/disks/boot/memtest86+.bin
/host/ubuntu/disks/boot/System.map-2.6.24-16-generic
/host/ubuntu/disks/boot/System.map-2.6.24-22-generic
/host/ubuntu/disks/boot/vmlinuz-2.6.24-16-generic
/host/ubuntu/disks/boot/vmlinuz-2.6.24-22-generic
/host/ubuntu/disks/root.disk

gzip: stdout: No space left on device

THAT WAS THE END OF THE OUTPUT

These were the results of the df -h command. I looked at these various locations, but don't really know what I'm doing.
I know this forum does not recommend suggesting a re-install to fix every problem and I am learning by fixing what is broken so I'm not wanting to reinstall yet, but if I need to let me know. I don't have a lot of personalized stuff and I only have a few programS installed so far.

kelly@kelly-laptop:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/host/ubuntu/disks/root.disk
13G 13G 0 100% /
varrun 950M 112K 950M 1% /var/run
varlock 950M 0 950M 0% /var/lock
udev 950M 48K 950M 1% /dev
devshm 950M 12K 950M 1% /dev/shm
lrm 950M 39M 911M 5% /lib/modules/2.6.24-22-generic/volatile
overflow 1.0M 24K 1000K 3% /tmp
gvfs-fuse-daemon 13G 13G 0 100% /home/kelly/.gvfs

oponto

January 24th, 2009, 08:29 PM

Hello,

I have a problem with this method of backing up:

I created an backup with following commands:

sudo su

cd /

tar cvpzf backup.tgz --exclude=/backup.tgz /
Did not exclude some more files, because I could not find out how
to recreate them after restoring my system with the backup file.
Did not create it directly on external hdd, because newbie (..) .

After copying the backup file to my external hdd, I deleted it:

gksudo nautilus
=> moved it to trash,
=> emptied trash

Did that all at least three times.
Those backup files (I deleted after copying) were about 75 GB (3*25 GB).

Now I checked my disk storage, and my disk usage analyzer shows weird things:

partition size: 122,6 GB
used: 103,7 GB
/ directory: 24,4 GB =>>>> 75 GB left in universe

==============> MEANS: those 75 GB weren't actually really deleted.
though, can't find them.

Wtf, stop that. What went terribly wrong ?

PaulWhipp

January 24th, 2009, 10:10 PM

Hi KC1117,

Looks like you have a couple of problems.

For the space issue, you need to look at the space utilisation on your drive. You can use the 'df' command at the terminal or the Accessories | Disk Usage Analyzer to see what space is available and where. df is quick but lacks the pretty picture you get with the usage analyzer. With the usage analyzer you can get a nice treeview of your filesystem to see where the space is being used but it can take a good few minutes to analyze a sizable drive.

For the backup of windows issue I can't help much. I don't dual boot - I use Windows via VMWare which lets me run multiple windows PCs in ubuntu for various purposes. You probably need to exclude more from your backup (possibly /host since I don't have one of those). If you are using a relatively clean ubuntu then there is little point in backing up the entire system - you probably only need to backup your home directory which will make things easier and quicker for you.

PaulWhipp

January 24th, 2009, 10:28 PM

Hi oponto.

If you run the scan filesystem option in the Disk Usage Analyzer and then view as Treemap chart, you should be able to spot where your problem is.

Here's a picture showing my drive with the main space hog on my external drive highlighted as an example:

http://www.paulwhippconsulting.com.au/images/posts/dua.png

You can roll the mouse out of the offending block of used space to work out the real path to it.

oponto

January 25th, 2009, 08:33 AM

Well, the disk usage analyzer shows the following weird facts:

http://img177.imageshack.us/img177/3082/missingspace50gbactuallue9.th.jpg (http://img177.imageshack.us/my.php?image=missingspace50gbactuallue9.jpg)

[img=http://img177.imageshack.us/img177/3082/missingspace50gbactuallue9.th.jpg] (http://img177.imageshack.us/my.php?image=missingspace50gbactuallue9.jpg)

I guess those 50+ GB are somewhere f.. up between deleting them from / and
moving them to trash and emptying the trash: maybe the trash cant handle deleting 20-25 GB files and so they are still somewhere ..

To avoid misunderstanding, 50 GB are missing not 75 GB; doesnt really matter actually.

thanks :)

KC1117

January 26th, 2009, 12:18 AM

I looked at the the disk usage (attached) but I don't know what is normal to know if somethng is wrong. I was surprised it is picking up the Windows programs. Perhaps if my backup tried to pick it up it was too much for the space I had left.

At this point I'm not worried about a backup but I don't know what to do to get my system corrected. I can't open most programs, can go online, but can't login to anything requiring a password, system won't shut off properlay. Should I just reinstall and start over?

Here is the result of the df command. It says I have 0 available, but when I look at the /.gvfs folder there is nothing there. What am I missing?

kelly@kelly-laptop:~$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/host/ubuntu/disks/root.disk
13563852 13556144 0 100% /
varrun 972052 112 971940 1% /var/run
varlock 972052 0 972052 0% /var/lock
udev 972052 56 971996 1% /dev
devshm 972052 12 972040 1% /dev/shm
lrm 972052 39788 932264 5% /lib/modules/2.6.24-22-generic/volatile
overflow 1024 44 980 5% /tmp
gvfs-fuse-daemon 13563852 13556144 0 100% /home/kelly/.gvfs
/dev/sdb1 978656 375376 603280 39% /media/UDISK_
kelly@kelly-laptop:~$

bgs100

January 28th, 2009, 09:00 PM

new shbackup version 0.0.3-2

oponto

January 30th, 2009, 04:45 AM

my problem is SOLVED

If you delete big files from the root, they are moved to the root trash folder:

/root/.local/share/Trash/files

Files in this folder are NOT shown in the disk usage analyzer !

Do only remove them by deleting the "files" folder with Shift+Remove,
if you try to remove them by moving to trash, they recreate themselves exactly there again (you'd delete them from this place to this place) .

Then recreate the "files" folder.

´silas

February 1st, 2009, 03:52 AM

This worked great :D Thanks a lot

billbear

February 8th, 2009, 10:22 AM

SanskritFritz

February 10th, 2009, 06:40 PM

stronzo

February 11th, 2009, 05:10 PM

without having read the whole thread...

when i use tar, do i still loose my file permissions, access control lists and so on? if so, this is really no good backup option!!

please tell me if this method can be used to backup a running system, while preserving all file permissions and metadata and can be used to restore the whole system (besides mbr) so that it works in the end.

SanskritFritz

February 11th, 2009, 05:23 PM

without having read the whole thread...
when i use tar, do i still loose my file permissions, access control lists and so on? if so, this is really no good backup option!!
please tell me if this method can be used to backup a running system, while preserving all file permissions and metadata and can be used to restore the whole system (besides mbr) so that it works in the end.You dont have to read the whole thread, but I really recommend you to read the whole first post ;)
Short answer: yes, that is it all about!

stronzo

February 11th, 2009, 05:35 PM

but did you read mine? what about the file permissions and acls and so on?
dont want to screw up my server.

SanskritFritz

February 12th, 2009, 04:09 AM

but did you read mine? what about the file permissions and acls and so on?
dont want to screw up my server.Well, since the first post doesn't explicitly assures you about this, here is my answer again. TAR stores every possible aspect of files and folders, including permissions, stiky bits, uids, acls and so on.

But please, do some tests yourself, just don't try it on your server first! Many people here can confirm, that this method is sufficient for backing up and restoring a whole server.

Dieseler

February 14th, 2009, 02:23 AM

I find that i can mount my root partition on /mnt.
My root partition is /dev/sda5,
sudo mount /dev/sda5 /mnt
Now, /dev/sda5 is mounted on both / and /mnt. If you have a /home partition, also mount it on /mnt/home
Then i can just backup /mnt,only need to exclude the backup file itself.
And /mnt/dev/ doesn't contain those virtual files.
When restoring, you don't have to re-create the directories like proc mnt sys,they are there already as empty directories.
Maybe i should exclude lost+found, when restoring to a reiserfs filesystem, there shouldn't be a lost+found. But i guess a lost+found wouldn't hurt reiserfs?
About re-creating lost+found,some say you should use the command mklost+found, does it make any difference than just "mkdir lost+found"?

Wow,
thats thinking outside of the box right there.
How about this...
Could I mount my root partition to another partition where I intended to keep the backup?
(My root partition is sda7. my ext3 backup storage partition is /dev/sda5)
Like,
sudo mount /dev/sda7 /dev/sda5
Then make the backup from right there?
If so, could I get the command to use to tar it from right there?
I'm a little slow with this stuff and I'm not sure I really understand the OP's command to tar it from root. I assume the / at the end of the backup command was designating what to backup, being root. The further example I believe would clear this up for me.
Furthermore, if all of the above is possible, how might I then restore the backup to /dev/sda7 without actually moving the entire file back to /dev/sda7 from /dev/sda5 before restoring it?
May I get the command structure to accomplish this as well?

My main concern is in moving such a large file back and forth and possibly corrupting it. I may be wrong in my assertion that this possible as well. The less I move it the less chance I have of it getting messed up, so in creating it in /dev/sda5 to start with, it will only be moved once when it is restored back to /dev/sda7.
Is my concern in this matter valid?

Jammanuser

February 14th, 2009, 03:13 AM

Wow,
thats thinking outside of the box right there.
How about this...
Could I mount my root partition to another partition where I intended to keep the backup?
(My root partition is sda7. my ext3 backup storage partition is /dev/sda5)
Like,
sudo mount /dev/sda7 /dev/sda5

I don't think that command will work. When you mount a partition, it needs to be mounted to a particular dir on the partition you're mounting from, I believe, and so you can't simply:

sudo mount /dev/sda7 /dev/sda5

However, if you do something like:

sudo mkdir /folder
sudo mount /dev/sda5 /folder
cd /folder
sudo mkdir /secondfolder
cd /secondfolder
sudo mount /dev/sda7 /secondfolder

I think your root partition would be mounted to a dir called "secondfolder" on your sda5 partition. And then from there, you could do the backup and later the restore if needed.

Hope it helps.

-Jam man

:guitar:

Dieseler

February 14th, 2009, 04:18 AM

Ok, I used gparted to format /dev/sda6 to ext3 and labeled it backups.
I then chown'd it over to my username.

I was unable to mount the partiton /dev/sda7 onto /dev/sda6 but,
I was able to create folder backups inside /dev/sda6 and then,
I was able to mount my root partition /dev/sda7 onto /dev/sda6/backups like this.
sudo mount /dev/sda7 /media/backups/backups
Its a funny looking mess, lol.
Upon further analysis, I was in fact able to mount /dev/sda7 onto /dev/sda6
The command was,
sudo mount /dev/sda7 /media/backups
backups being the label of /dev/sda6.

Now if I can get a little help modifying the OP's tar command, that is intended to be called from root, to back up root inside root, lol, I think I will finally have a grasp on this.

This is the original command.

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /
After mouting my root partition sda7 to sda6,
I assume I should cd into /media/backups
Then,
tar cvpzf backup.tgz /media/backups ???
Then I should find my backup of sda7 inside sda6?

Just noticed something really weird in Nautilus, will edit again when I figure it out.
Ok, I went through it again.
When I umount /media/backups to free my root partition from the backup partition it leaves a little ghost of the backup partition backups (/dev/sda6) in Nautilus.
No biggie. I was scared to try it, but the proper thing to do was ,
sudo umount /dev/sda7
And that left no ghost of /dev/sda6 (backups).
This is some mind blowing crap.

neilevan814

February 14th, 2009, 04:38 AM

Dieseler

February 14th, 2009, 04:39 AM

I know the above must be hard to follow so I will get to the gist of what I need from there.

After mouting my root partition sda7 to sda6, confirmed, can be done.
I assume I should cd into /media/backups (Which is the /dev/sda6, the partition I wish to back up to.
Then,
tar cvpzf backup.tgz /media/backups

Then I should find my backup of /dev/sda7(My root partition) inside /dev/sda6 (The Partition I wish to back my root up to for storage)?

I really don't blame you guys if you run screaming from this lol.
:lolflag:

Dieseler

February 14th, 2009, 05:09 AM

First I want to thank you for this tutorial. It works and I found it in the nick of time. Secondly, and maybe it is just my hardy installation, but, my backup file was saving itself to my /home partiton and not to root, so I had to change the --exclude=/backup.tar.bz2 to /home/me/backup.tar.bz2 or that backup file was being backed up and making for a very big file. Just check as your backup starts as to exactly where your file is being saved.. That will save you time and effort in restarting the process with the right --exclude options.

That idea helped a lot. I was getting the .tgz backed up inside the backup while testing. --exclude=/media/backups/backup.tgz rather than just --exclude=/backup.tgz fixed that.
I wish I knew for sure what I think, is that you run it from your /home/me rather than /.
I think that is what I gather from that.

I mounted another partition /dev/sda5 just now to /media/backups(/dev/sda6) for testing.
Then I run this
tar cvpzf backup.tgz /media/backups --exclude=/media/backups/backup.tgz
The backup of /dev/sda5 did in fact show up in /dev/sda6 as expected.
This is cool beans man.
You guys teaches real good.
Thanks.

Edit: After unmounting all and looking back, It put the backup in /dev/sda5 instead of /dev/sda6 where I intended.
So it pretty much defeats the purpose of doing what I was doing all together.
I'm baffled.
I think I will stick with Heliode's way lol.

johnnyhostile

February 20th, 2009, 08:10 PM

Hey all!!

Just wanted to let everyone know that this backup method worked PERFECTLY for me EXCEPT one thing this thread did not cover (from what I can see):

/boot/grub/menu.lst obviously has the entry for yout OS, watch out for the UUID.

In my case, I had to reinstall from scratch and, thus had a new UUID so copying the old menu.lst did not work until i replaced the old UUID with the new.

EXAMPLE:

title Ubuntu 8.04.2, kernel 2.6.24-23-generic
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-23-generic root=UUID=long-string-of-stuff
initrd /boot/initrd.img-2.6.24-23-generic
quiet

In bold is what you need to look out for.

This stumped me for a bit so I thought I would share :D

Thanx to the original poster!!! :KS

bodysoda

February 21st, 2009, 12:13 AM

Thanks for the tips

sofasurfer

February 21st, 2009, 02:53 AM

Instead of using "--exclude=/root/.local/share/Trash* --exclude=/home/daryl/.local/share/Trash*" can I just use "*/Trash/*"? And the same with "temp" and "tmp", by using "*/temp/*" and "*/tmp/*"?

sofasurfer

February 21st, 2009, 10:01 PM

johnnyhostile.
Is the UUID situation what is giving me a non-bootable system? I restore my system and then when I start it up I get an error (15? I don't remember right now) And then I am told to hit any key. Basically, it won't boot. What do I need to do. Here are my backup and restore commands...

Backup...

tar cvpzf working-system-backup.tgz --exclude=media/storage/* --exclude=mnt/* --exclude=cdrom/* --exclude=/root/.local/share/Trash* --exclude=/home/daryl/.local/share/Trash* --exclude=daryl/Desktop/* --exclude=daryl/.wine --exclude=/proc/* --exclude=/lost+found/* --exclude=daryl/.miro /

Restore...

tar xvpfz backup.tgz -C /

kelvinwu_2008

February 23rd, 2009, 02:06 AM

Good guide. I will try backup my system by this way.

Haluci

February 23rd, 2009, 11:33 PM

Uh... I just did something really stupid, so I want to post this to make sure nobody does it too.

If you're backing up your hard drive to an external hard drive, make sure to add --exclude=/media ... unless you want to clobber your external hard drive... :p

EDIT: also, using . instead of / at the end helps if you are backing up another partition from linux onto an external hard drive.

petteriIII

March 6th, 2009, 01:30 AM

This method of 'system-backup' seems to make some changes to the system. But I have not noticed that it affects to anything important and I use it trough scripts those take advantage of high-level (gparted, gedit …) as well as low-level procedures in Ubuntu. And I have not noticed any cumulative effects, but I have not gone over 30 iterations yet; bacups when system has booted normally, when system boots only in rescue-mode and when system doesn't boot at all but you must boot from live-CD and have backup-file in memory-stick.

So it is not useable only to individuals, but at least to small organizations. At this moment this method is only useable for some kind of 'playing', but it is not clear how important it will be; for example think when the time comes when viruses start to make havock. Don't play that card of virtualization with this.

kooldino

March 6th, 2009, 02:53 PM

The problem with this is that it won't copy your MBR...

ABX

March 9th, 2009, 12:18 PM

I got a little problem

CentOS:

I did a backup like it's written on the first post, but when I try to restore I get:

[root@localhost /]# tar xvpfz backup.tgz -C /
./
srv/
.swp
usr/
usr/libexec/
usr/libexec/sudo_noexec.so
usr/libexec/hald-probe-hiddev
usr/libexec/awk/
usr/libexec/awk/grcat
usr/libexec/awk/pwcat
usr/libexec/hald-addon-cpufreq
usr/libexec/gconf-sanity-check-2
usr/libexec/hald-probe-input
usr/libexec/hald-probe-printer
usr/libexec/mysqld
tar: Skipping to next header

gzip: stdin: invalid compressed data--format violated
tar: Child returned status 1
tar: Error exit delayed from previous errors

I tried to backup several times, checked the file under windows (all OK), but when I try to restore the backup I get the error.

Any idea why?

Solved:
Apparently the file got corrupted when moved to a windows FTP server.

petteriIII

March 13th, 2009, 08:27 AM

I have used this method as interesting toy. But it has real use also:

1.Properly constructed back-up file is only ~1 GB. So every machine of network can have it's own personalized backup-file residing in it's own hard-disk and no problems arise from anything; and you can restore 'in place' or remotely.
2.You can harden backup-file against vandals so that they cannot destroy backup-file 'by-accident' even if they somehow succeed to boot with live-CD or USB.
3.You can use this restore-method also when system doesn't boot at all – by booting with live-CD and taking tar-archive from memorystick if needed. At the same moment it is easy to get rid of anything you don't want by first nuking the partition and its associated swap with dd – or even replacing the hard-drive. Because the only method to restore total confidence to a system that has been intruded is 'fresh install'. This method is quick personalized 'fresh install'.

My problem arises from point 3. When you boot to system that has been restored this way you may get some extra boot-lines saying that you cannot write to read-only system. You can use the system normally, but these 'cosmetic failures' may turn out to be nasty in the future, so I am interested to make them disappear. You can make those messages disappear by a change to boot-lines in menu.lst: original line is: kernel /boot/vmlinuz-2.6.27-11-generic root=UUID=8af0ff78-5ce3-4023-a812-99f76411e5c3 ro . In this line you change ro to rw. Why there is ro in the first place ?

chris4585

March 15th, 2009, 08:07 PM

linnuxnub

March 22nd, 2009, 04:17 PM

I am running ubuntu 8.10 and I followed the steps outlined in the first post. I used the bz2 compression. Copy the code exactly from the page.

I went to restore from this backup and it all works fine I re-create the excluded directories (media,mnt,sys,proc,etc). Now I reboot to complete the process and I can no longer boot, grub is giving me error 15. What do I do I do not want to have to reinstall my system from scratch please help.

I seem to be having the same problem as sofasurfer on the previous page, the only difference is that I used the code from the tutorial only modifying it to exclude /media.

linnuxnub](*,)

I solved it, by skipping the step on recreating directories everything worked as it was supposed to and my system was completely restored to the way it was. Thanks to this tutorial on backups all my data is safe.

Mayfairy

March 24th, 2009, 01:30 AM

I would just like to say: THANK YOU! this tutorial works flawlessly. I backed up earlier today, and decided to install a different nvidia driver from the one I have, in short it went horribly wrong and couldn't fix it. I backed up and it worked way better then I thought. This tutorial saved me a bunch of wasted time.

:D

So it will work that way too. I was just about to make a backup after fiddling with the 8.10 install for two days. Now I got a working system but I want to tweak it a bit more (no restricted drivers without breaking X) so I thought this might be the easiest method to save me some time when the things get really bad.
Then again restoring this backup doesn't move the extra drivers and files I have installed afterwards, just the config files that use them. .

In this case I'd need to keep track of installed files so that I don't have any extras laying around everywhere.

(ie. I have a working system with nvidia-glx-177 drivers, I make a backup, fiddle around and install nvidia-glx-180 drivers which in the end won't work and restore the backup so I end up using 177's once more. The 180 files are still around since they're not named the same as 177.)

EDIT: "--exclude=/home" is also a good thing to have, especially if it's on different partition. That is unless you use different partition to store media files and stuff. For me the root dir contains 6 gigs of stuff while /home dir is 90 gigs.

ricky15100

March 25th, 2009, 08:53 AM

Jammanuser

March 25th, 2009, 11:41 AM

hi guys, i have a bit of a problem, i re-installed a fresh copy of ubuntu and then did a restore with the method above.

Now i when it boots i get "error 15 file not found", i have tried re-installing grub which it tells me is successfull but when i re-boot i get the same error.

Help

Thanks Leon

Check your /boot/grub/menu.lst and verify that your Ubuntu entries are pointed at the correct partition. You can run "sudo fdisk -l" (the last letter is a lowercase "L") in the Terminal to find out the sdxy syntax location, and then use it in your menu.lst in the "root (hdx,y)" lines, where the "x" and the "y" stand for drive and partition number respectively. FYI, the "(hdx,y)" notation starts the count for both drives and partitions at 0, while the "sdxy" one, on the other hand, uses letters to indicate the disk, and starts the count for partitions starting at 1. So "sda1" (the first hard drive, first partition) would be the same thing as "(hd0,0)" using the other syntax.

Hope it helps.

-Jam man

:guitar:

ricky15100

March 29th, 2009, 09:16 AM

Check your /boot/grub/menu.lst and verify that your Ubuntu entries are pointed at the correct partition. You can run "sudo fdisk -l" (the last letter is a lowercase "L") in the Terminal to find out the sdxy syntax location, and then use it in your menu.lst in the "root (hdx,y)" lines, where the "x" and the "y" stand for drive and partition number respectively. FYI, the "(hdx,y)" notation starts the count for both drives and partitions at 0, while the "sdxy" one, on the other hand, uses letters to indicate the disk, and starts the count for partitions starting at 1. So "sda1" (the first hard drive, first partition) would be the same thing as "(hd0,0)" using the other syntax.

Hope it helps.

-Jam man

:guitar:

Thanks for the help Jam man, it was the uuid, i had to go into dev/devices to find the uuid and replaced it in the menu.lst, so it appears that when you do a fresh install you need to change the uuid, so i did a new back up and also excluded the boot directory this time when i did the restore it worked fine.

Thanks anyway Jamman much appreciated i am a happy cat now

:P

warpasylum

April 1st, 2009, 10:06 PM

Hey,
Thanks guys, this'll definitely help me out alot in the coming weeks. I finally gave up the ghost as far as Windows on my laptop n' want to experiment with my Ubuntu to get better at it. This'll really help me out if I hopelessly crash my system. Thanks again guys.

crispeto

April 4th, 2009, 02:44 AM

So I made a mistake and now have a 2.7 gig "Backup.tgz" icon. How do I delete it?

YoungQuiz

April 4th, 2009, 12:04 PM

this was helpful!

ambdeep

April 6th, 2009, 12:57 AM

I was wondering if i could install a newer version of ubuntu and still use the restore without causing any problems....also can i install, say, kubuntu and still use the restore generated on ubuntu......some people were also complaining about a grub error 15......could you also guide me through the walk-around of this error as i could not understand the posts which were ment to explain it.......any help will be appreciated......
Thanks in advance!

Lunx

April 6th, 2009, 03:41 AM

As someone with only basic knowledge of things computery, I've also found this thread very helpful and easy to follow. This morning I installed an app that is really cool for making back-ups of your system, or even creating your own distros, Remastersys. Very simple to download and install (once you set up the repository), and simple to use and easy to understand GUI. Took me half an hour max to set up repository, download, install and then create a full backup. I've got a Jaunty install and all the extra apps I like to install first up, the ISO ended up around 1.1GB. Haven't burnt it yet to see if it worked successfully, as I haven't any spare DVDs to hand. There's apparently also a way of running it in a virtual machine, but I have zero experience with VMware yet, so that's the thing I'm just about to go and play with.

Remastersys for Ubuntu (http://www.geekconnection.org/remastersys/ubuntu.html)

And an easy to follow tutorial (http://www.dedoimedo.com/computers/remastersys.html) on usage

weatherguyto

April 6th, 2009, 07:35 AM

Dear Heliode/and forum members,

After you completely backup your system given by Heliode the question is how do you reload the tar file? I happen to tried it but didn't work because I couldn't delete the excludes(proc, lost+found, sys etc)they were lock by the system before I had to rebooted the system.

Could tell me what do I need to do to load the backup Tar file
if by system happen to go bad at one point?

Could provide me steps I need to take.

I think I need to format my ext2 then load the backup tar file,
and add the missing folders (proc, sys, lost+found).But how do you load the backup tar file in this state? When there is no OS install yet, maybe boot into live CD, and then what do I do. I'm a novice to Linux please advise.

The problem is that my system will not shutdown out of X windows(gnome) after I installed the video drivers(nvidia 177)

SO I'm getting a lots of check disk(fsck) restarting up after a shutdown.

Thanks for you help,
William

crazypeppo

April 10th, 2009, 12:51 AM

@the original poster

Do you mind if I give a copy of this to everyone I know?
:)

KEE

April 11th, 2009, 05:11 PM

hey there nice guide and job at explaining everything =)

my situation is I have linux~intrepid 8.10 and windows xp pro on the same drive. I am wondering if this would work for me? windows had the "missing system32/hal.dll" thingy on startup. this stoped the whole drive from booting but I was able to rewrite the boot.ini in windows using ubuntu cdrom. but windows still has that thing on start up and intrepid works just fine and i still have the grub too to select the os's. I am thinking of reinstalling because i dont mind reinstalling winxp. nothing to loss there, but on intrepid! ya really weary on reinstalling linux, its all prettied up and dont want to have to do the customisation all over again. I love that you guide can help me but i need to know if it would work. if you have a guide available for my situation id like if you add your comment in my post, instead of here since it will be easier and less traffic with other post if i come across any complications and I will never lose the thread in ubuntu's enormous thread farm. Please and thank you for your help

http://ubuntuforums.org/showthread.php?p=7054922#post7054922

Ericyzfr1

April 11th, 2009, 07:09 PM

That whole stuff about Grub not working and all is ********, I used Norton Ghost 2003 to back-up and restore Ubuntu with Grub and all works fine including the booting part. :smile:

Ghost and Acronis both restore Grub without any problems and work perfectly with Ext3. However, it is no longer true and can only do a cluster to cluster back-up which can take hours and take a lot of HD space. I did some searching and contacted Acronis support team and they are unwilling to say if and when they will support Ext4.

telekinetic

April 26th, 2009, 03:13 AM

Thanks, this was extremely helpful!

One suggestion though. Instead of excluding an entire directory, why not just exclude the contents? No reason to create sys, mnt, and the rest to restore. You can leave the directory, just exclude the contents by using a wildcard *

tar cvpzf backup.tgz / --exclude="/sys/*" --exclude="/mnt/*" --exclude="/proc/*" --exclude="/lost+found" --exclude-caches
Be sure to wrap the directory to be excluded in "quotes".

sadicote

April 26th, 2009, 03:35 AM

Thank you, Heliode, great guide, but i came across Remastersys only a few days ago, and i just cannot help singing it's praises and raving about it at every opportunity i get. There is also the Back in Time utility that i read about. Stuff like this will forever change the way people use Linux.

markitoxs

April 26th, 2009, 06:45 AM

Really good guide.

dc0m

April 27th, 2009, 04:02 PM

I used the following method successfully for long time now. Make sure you umount everything besides / partition and /newdisk/proc is empty.

cp -ax / /backupdisk

That will copy everything on the / partition to backupdisk.

When you want to restore just do the same thing again.

cp -ax / /olddisk

Where / is in this case the root of the backup system and /olddisk is target.

Good Luck.

You can also boot in the backup system just edit grub and fstab to reflect that partition.

cantthinkofanickname

May 4th, 2009, 01:56 PM

I have BackupPC installed which offers a tar method. Would I achieve the same system if I used this program? Is anyone using it for complete system backups?

mozkill

May 4th, 2009, 07:05 PM

While I think its nifty to do backups using the hardcore method, its SOOOO much easier just to use Clonezilla . Clonezilla uses the standard Gzip backup method except that it has a really nice wizard UI to go with it and the tool is bootable, and is able to backup to thumbdrive, bootable image, or a samba/windows share folder.

http://clonezilla.org/

cantthinkofanickname

May 5th, 2009, 06:45 AM

Yes, sure but I am not using the GUI just access from a networked laptop running Webmin. I will look at Clonezilla. Sounds like it may come from the same stable as Filezilla?

johnaaronrose

May 8th, 2009, 03:14 AM

The disk used on my laptop is approximately 40GB. I want to back it up to a usb 1TB drive, formatted as FAT32. As I understand it, the max file size on a FAT32 device is 4GB. When I try the original heliode method of creating a gzip file, it stops with a message about 4GB file too large. I don't see any option in gzip to split large resultant gzip files. The best option seems to me to be to use Partimage, which allows resultant split files. Any recommendations?

Nyken

May 11th, 2009, 11:34 PM

Hell of a lot better than messing with aptoncd even. I like being sure about what I'm backing up. Thanks !:)

renkinjutsu

May 12th, 2009, 12:09 PM

i recently had to restore my computer using this tar method, but i've been having some problems.

after restoring, i can no longer mount usb drives or mount ANYTHING for that matter.. i have no connectivity (the connectivity icon doesn't even show up) and my sound doesn't work (the volume applet also tells me it's busted)

another thing is that i backed up all my media files in another tar archive (i didn't want to waste the time and cpu power to try to compress media files) so i left them in a .tar file .. however, when i try to restore those files, only the directories and some files are restored.. the tar is a 107GB file (according to 'du' and also the properties tab in nautilus) after extracting that archive into my /home directory, the total disk usage is still just 6GB .. how can i restore the rest of my files?

zero7404

May 12th, 2009, 12:14 PM

why not just use clonezilla ? it's got different compressions to choose from, and from what i remember, my 7GB installation got compressed to about 2GB....

backup and restore process was really fast and simple, and you can get your system back up and running in less than 1 hr (10 mins for me), as opposed to reinstalling everything then restoring with the process the op mentions.....

renkinjutsu

May 12th, 2009, 12:16 PM

it's too late for me to use clonezilla .. it seems i've lost a massive amount of data already (unless i can restore from that tar file)

can't use clonezilla anyway, i'm using the ext4 filesystem which is still being tested by clonezilla

zero7404

May 12th, 2009, 12:31 PM

do you have a single hdd or 2 hdd's in your system ?

renkinjutsu

May 12th, 2009, 12:34 PM

i have a single hdd but with 2 partitions one mounted on / and the other on /home

PaulWhipp

May 12th, 2009, 06:34 PM

As space is so cheap these days, I 'backup' my entire workspace locally using rsync (use 'man rsync' for more details). Its worth checking out - even if you have the backup file system mounted locally, rsync's options give you great control over synchronising the backup directory and its really fast compared to tar or cp based solutions. This has the added benefit that you can mount your backup on another system and carry on working then use rsync to put the work back on your main system later.

My serious project work is further protected by the use of online repositories (subversion) which have the benefit of version tracking and deployment capabilities (I favour launchpad for open source and springloops for commercial work).

zero7404

May 12th, 2009, 10:19 PM

i have a single hdd but with 2 partitions one mounted on / and the other on /home

since thumb drives are convenient and cheap, you can backup to one....

or if you have a portable hard drive, you can backup to that...

in this way, your backups are on separate media, no risk of loss if your drive crashes...

leus

May 13th, 2009, 01:12 AM

since thumb drives are convenient and cheap, you can backup to one....

or if you have a portable hard drive, you can backup to that...

in this way, your backups are on separate media, no risk of loss if your drive crashes...

i have a single hdd but with 2 partitions one mounted on / and the other on /home

Me too i have /home in separate partition ,but I have a second drive formatted with ntfs I think is wrong , ext3 is better for backup my /home?

PaulWhipp

May 13th, 2009, 03:31 AM

It depends how you backup. If you tar or process stuff into a single file you should be OK using an NTFS drive. However if you rsync or cp directly you will find everything comes back with all the permissions enabled which is very irritating.

If in doubt use ext3... and your backup should definitely be removable (and actually removed) if you want to survive fire, flood, lightning strikes etc.

zero7404

May 13th, 2009, 09:35 AM

on my machine, i have 2 320GB disks, they are set up as follows:

disk sda:
sda1 (ntfs) 150GB -- vista installation
sda2 (extended) 148GB -- ubuntu installation
sda5 (ext3) 147GB
sda6 (swap) 1GB

disk sdb:
sdb1 (ntfs) 273GB -- personal data storage and acronis backups
sdb2 (ext3) 25GB -- clonezilla backups

external freeagent go:
225 GB (ntfs) -- used for robocopy (vista) and backup of contents of sdb1

i use the ext3 format wherever it is needed with respect to linux files/backups. otherwise ntfs does fine since ubuntu can read/write and mount/automount to it without a problem...

haneya

May 16th, 2009, 03:56 AM

Hi , I know my question is out of subject a little but I need some help ,

I backed up my ubuntu system with home directory , I need now to restore only home directory from the archive I tried with file-roler but it take to much time , I hope some one tell me how to extract only HOME from this archive.

Thanks

antonya

May 16th, 2009, 05:02 AM

Hello,

I hope this is the right forum as I am completely new to Ubuntu and the Linux functionality and of course these forums. I have left the Windows ranks after much pressure from experienced Linux friends and I must say I like what I see so far.

My issue is with transferring my Firefox bookmarks etc. that I saved in my Windows Vista configuration using Mozbackup. I now have a Mozbackup archive that I'd like to integrate into my new Ubuntu installation, but I see that Mozbackup isn't compatible with Ubuntu.

Could anyone please tell me how I can transfer my original Mozbackup backup into my new Ubuntu installation?

Many thanks in advance,
Antony

Wookiee

May 16th, 2009, 04:22 PM

Should I be worried that the original partition is 295 GB, but the tar image is barely 25 GB?

I've had bad experiences with that factor of compression.

Nyken

May 19th, 2009, 05:16 AM

OF all the tips that I've read since migrating to Linux and Ubuntu a couple of weeks ago, th is one has saved my ****. As I learn how to set things up and get them working, I back up my system with this guy's simple idea. Guess I did something wrong, since my File System desktop icon isn't working for one thing, so untaring the backup fixed things nicely.

Just wanna drop a thanks a million.

dmizer

May 19th, 2009, 06:52 AM

Should I be worried that the original partition is 295 GB, but the tar image is barely 25 GB?

I've had bad experiences with that factor of compression.

The backup tutorial doesn't do a flat image of the whole drive, it only backs up data. So, you should compare that 25 GB file to the size of the actual data on the drive.

RootFish

May 26th, 2009, 03:06 PM

Does this work well when backing up systems that are running a database like a LAMP server or mail server? Will I end up with corrupt files after a restore?

oldefoxx

May 29th, 2009, 08:30 PM

Alright. I've seen some worthy discussion on this topic, but in reading through the thread, I can't get it in my head exactly how to do this. Let me esplain my situation a bit further:

Two large hard drives and multiple partitons. This includes three bootable Ext3 partitions with Ubuntu, three Ext3 partitions for future use or temporary storage, one NTFS partition with XP installed, one NTFS partition with Windows 2000 Pro installed, tws NTFS partitions with applications and data on them, and three swap partitions. Now I have True Image version 11, which sort of works. And I want to back up onto an external USB hard drive with a good amunt of compression. And of course I need to consider the cases where I may have to do one of the following:

(1) Completely restore one or both hard drives, which may not be bootable bwfore the restore starts. These may also be replacement drives, meaning they need to be partitioned and formatted to agree with the backup layout.

(2) Restore specific partitions back into their original locations. Two modes involved, either just a system restore with data left intact, or a full restore if need be.

(3) Restore specific folders and files back to their original or a different partition. Also have the option to only overwrite older files, or skip ever existing files.

I bet that someone is going to say that I really need a commercial package. I would like to see if someone else can think of a way to do all or most of this with the tools available to Linux

You might think this is overkill, but I'm not throwing away what I already have as I move forward to something new. Moving on to higher capacity drives means I should not have to.

Okay, decide who is first at bat, and give it your best shot!

JoelMay

June 3rd, 2009, 02:11 AM

Good guide. After you compress it if the file is too big to fit on a DVD or something you can use the split command to make it into smaller chunks, then use the cat command to combine the files later.

split -b 650m -d - "largefile.tgz"
cat < largefile.tgz.*

all_gears

June 10th, 2009, 01:53 PM

thanks for the tip. hopefully, I won't have to use it, but I'm making the backup right now.

two things I've noticed:

first, it might be nice to exclude /var/cache/apt (if I remember the directory correctly... most people know what I mean). I ended up with 400 megs of already-installed .deb files in my backup. not tragic, but slightly inconvenient.

as far as excluding the home folder, I wonder if there is any way to exclude everything except the hidden/config folders like .kde, etc. It would be nice to keep all the settings whenexcluding home.

also, what's the verdict on /dev ? I read about 6 pages of discussion about it, i don't want to read another 40-something :KS.

SilverSnakeEyes

June 15th, 2009, 02:40 PM

If you wanted to say...backup to external media and exclude it from being in the backup how would you do it? I tried it just now and it does not work. My external media's name is "Ninja Drive".

cantthinkofanickname

June 16th, 2009, 07:57 AM

I have now done a backup using the original set of instructions. This post is quite old and have searched all the subsequent posts for any major corrections to it and found none so I used the original.

Comment welcome on that just in case I've missed something?

My command is:

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/mnt --exclude=/sys --exclude=/office/backups --exclude=/backup.tgz /

I've excluded my office directory because that is where I store networked windows backups and the fact that this is an hdb mount point for the servers 2nd drive which I assume if I tried to restore by untarring would give problems.

Comment welcome if I'm not thinking correctly there?

Has a new method arisen since the original post? I use webmin and this has a tar command but assume it is something different.

rohitfeb14

June 16th, 2009, 10:23 PM

If i have no other operating system and just ubuntu how would i restore the folders when i lose it.. so i need to install windows or so and then copy it and install grub...

Or can it be done by a live CD or something on Pen Drive....

Anyways if i want to make a media that would do everything copy install grub and all such stuff, how can it be made...

rohitfeb14

June 16th, 2009, 11:26 PM

:popcorn: similar to changing the installation script that after installing basic system it copies my files to the required location... how to do that?

rohitfeb14

June 17th, 2009, 03:17 AM

Suppose i have no other operating systerm and i need to restore my backup

then how can i restore it...... assuming i have no os installed at present..

i mean using LIVE CD or so...:popcorn:

pro003

June 22nd, 2009, 12:25 PM

But it's a commercial application.... :?

I agree. - Why use commercial appz?

Nice HowTo. Will try ;)

deepakbm

July 8th, 2009, 10:25 AM

seanscott83

July 8th, 2009, 11:55 PM

I have a question, if I am running a Ubuntu Server 9.04 32-bit and wanted to move the install to a 64-bit Ubuntu server 8.04 on different hardware. Would I be able to just move the directories that contain the important server files (excluding drivers and grub boot info) or would the files not be compatible as they are different hardware/ O/S versions.

I am running the server as a LAMP webserver as well as a Bind9 DNS server. If not woud I be able to backup the MYSQL databases manually using the tar command?

Thanks,
-Sean

pro003

July 10th, 2009, 02:03 PM

I ve encountered some problem after backing up and restoring
Here is what i did
i followed the steps in first page
only exception was that i excluded the media drive .
I reinstalled ubuntu 8.10 and restored it usung the tar file in /
After i rebooted and selected ubuntu from grub it shows an error

Error 15 File not found
any way i can access windows
I tried installing grub,but didnt work
Please help

insert install cd and select repair/fix broken system
drop to root shell when being asked choose the partition where your ubuntu is installed, for example sda3 or sda4... when you get into shell type

grub-setup /dev/sda

for more options type
grub-setup --help

then if it shows up, select UBUNTU REPAIR MODE from grub menu and then choose UPDATE GRUB BOOTLOADER.

REBOOT

================================================== ============
method 2 => would be downloading super grub disk, you can google it, it's free and its image is about 4-5 megabytes, burn it on cd, boot from that cd and follow the instructions, it's very simple, you can boot into OS windows or linux and other options available also, and you can automatically re-install grub boot loader.

GOOD LUCK.

ducksun43

July 11th, 2009, 05:15 AM

I found those links and therefore I do it that way:

First I get a list of all installed packages from the original/old system (a cron job does this on a regular)
- http://sunoano.name/ws/public_xhtml/time.html#cron
- http://sunoano.name/ws/public_xhtml/debian_notes_cheat_sheets.html#get_and_or_set_list _of_installed_packages

then I install the system
- http://sunoano.name/ws/public_xhtml/debian_notes_cheat_sheets.html#install_debian_from _USB_stick

and finally sync all data from the backup eg a USB disk
- http://sunoano.name/ws/public_xhtml/unison.html

bishounen

July 12th, 2009, 08:07 PM

I'm wondering:

If you are (like me) moving your Ubuntu install to a larger HD to be dual-booted with Windows, would it not be wiser to to this: Follow the full backup instructions as listed in post #1 of this thread. Swap Hard Drives. Install Windows, and then install Ubuntu afresh thus creating a new boot loader with TWO operating systems listed in it. At this point run the restore as listed, BUT exclude the /boot directory when un-tarring your backup.

Would this not prevent breaking grub, or am I missing something?

The only reason I ask is that I am attempting to set up the above situation (moving to bigger HD for a dual-boot for gaming) and all I managed to do so far is waste most of a day setting it all up and then completely breaking everything so badly that grub can't even see the HD anymore.

Would my idea even work?

bishounen

July 12th, 2009, 10:01 PM

Would my idea even work?

Ok, got an update:

Nope, it doesn't work. I suppose if you excluded the /boot directory during the initial backup it would work, but the --exclude command does nothing when un-tarring the backup. it still overwrites the /boot directory.

HOWEVER I have found a method that DOES work.

Having just gone through the trouble to reinstall BOTH Windows and Ubuntu after my initial utter failure, before running the untar command to restore my old Ubuntu settings, I had the foresight to BACK UP my /boot directory to the same external HD that my Ubuntu backup was on. So I basically did the following:

1) Ensured that my freshly installed Ubuntu was fully up to date, including recent kernel releases, the same as my original Ubuntu install.

2) opened a terminal window and entered the command:

sudo su

to work as root.

3) I then CD'ed into my external HD.

cd /media/disk

4) I then copied over my /boot directory to the external HD.

cp -a /boot /media/disk

5) I then ran the restore commands as listed in post #1.

6) Once that was complete, I ran:

rm -r /boot

to remove the freshly copied /boot directory.

7) I then ran:

cp -a /media/disk/boot /

which then rewrote the original /boot directory back to the primary drive, including the proper grub install with both operating systems.

I rebooted, and everything works!

:guitar:

This is an even shorter grub fix than the first two listed on the first page, as it doesn't require rebooting with the install disk a second time. You can fix it from right inside your fresh install! (Although it does require use of the terminal, AND the "rm -r" command.)

I hope that helped anyone stuck in the same situation as me.

nhanquy

July 14th, 2009, 09:24 PM

nmaster

July 17th, 2009, 06:56 PM

Does this work if you are dual-booting? I have a recovery partition (came with the laptop), a partition for Vista, and a Jaunty partition. I obviously wouldn't want to affect the other partitions when I backup Jaunty.

nismotuner

July 27th, 2009, 10:02 AM

I used the procedure, tar and untar with some exclude and bz2

I think it's better with no compression but well had some trouble with bz2 after transfert with ssh maybe thats the other part of the work that went wrong...

anyway just tgz not compressed, u can transfert file by ssh who will itself compress files

by SCP

so this way, less time to use cpu to tar ( backup)

less bandwith to transfert

nismotuner

July 27th, 2009, 11:58 AM

Yes, until then I won't pay $49.99....

We need open source! theres a bug we can get inside.. Ubuntu lifestyle!

linuxvacuum

July 28th, 2009, 09:02 AM

Everytime I restore my backup, my wired networking is screwed up and I get a read only file system error on boot. How can I fix this?

emeraldgirl08

July 28th, 2009, 02:17 PM

My Jaunty partition isn't very big (3.2 Gb) so how about I just clone that partition and make it a little bigger. Then I could paste it onto an external Hard Drive???

Would that work???

my.self

July 30th, 2009, 09:01 PM

pro003

July 31st, 2009, 07:51 AM

How can i delete all files before i untar my backup ?

I'm a linux newbie and tried to install several things.
So i copied totally planless files everywhere in the filesystem and i want them to be deleted to get a clean system again!

you don't delete them, they will be overwritten during untar, you just need to reboot after untar, of course if you have done everything before that properly.

adrian-g

July 31st, 2009, 10:20 PM

I have successfully made a backup and transferred it to my external hdd. Does the backup include all of the applications I have installed (I did not change the command)?

mjstelly

August 2nd, 2009, 02:06 PM

I'd like to add my experience. Heliode's howto served its purpose for a time. Ubuntu has evolved greatly since this original post 4 years ago. There are far easier and more reliable means to back up and restore a system.

I tried this method a few days ago completely trashing the install. Not a reflection on these instructions, but my naiveté. With the help of others, I learned that Clonezilla, among others, can do the exact same thing with far less chance of user error.

I recommend foregoing this manual process in favor of the automated means provided by Clonezilla.

Hope this helps.

my.self

August 2nd, 2009, 04:32 PM

you don't delete them, they will be overwritten during untar, you just need to reboot after untar, of course if you have done everything before that properly.

Yes,
but what if i want also to remove files that are not inside my backup?

e.g. i copied 10 text files to "usr" folder...
If my backup don't contains files with the same name they will still be in "usr" folder after restoring..

tin_truc22

August 15th, 2009, 10:30 AM

Hello everybody,

I have some experiences about broken file system after suddently shutdown PC. (Go to here http://forum.vnoss.org/topic5932.html if you want to see what was my trouble)
Then I an idea about Ubuntu recovery.

You can choose recovery File system from Boot menu (Grub). It sound like (Recovery Partition) in the laptop. But I think this method is better because your don't need another Partition to save the backup (just create /backup folder). And it doesn't waste all your partition so you can keep /home folder or /var/cache/apt

I think I can customize initrd and insert it to menu.lst. It will promt user a menu to recovery (keep and overwrite all old file, keep home folder, delete all old file and extract backup.
The most importance is user just pressed some button to restore all the system so I can give this computer for somebody without worrying about reinstall Ubuntu after the system is broken.

But I wonder is there anything likes this idea because I don't want to re-invent the wheel. If not, where can I begin to rebuild the initrd and put the script to restore the system. I can make the script to backup the system and put it in the PPA but I don't have experience with initrd.

Thank you,

huanhuan19

September 28th, 2009, 09:56 PM

It is good to know! THanks.

I am going to restore my Windows vista. THe thing is the ubuntu was installed in the windows vista, I am not sure how do I restore the backup I just made?

THanks a lot,

huanhuan19

September 29th, 2009, 12:28 AM

I have a problem when I backup, it also backups all my windows partitions. It is huge file.

So what else do I need to exclude then?

THanks,:popcorn:

newb85

October 6th, 2009, 10:29 PM

I have a problem. I'm creating a backup of my root directory on an external harddrive, and to avoid backing up what's on the external hd and the backup file itself, I included "--exclude=/media" in the command. (I also included "--exclude=/mnt".) Yet for some reason, the path and names of files on the external still come rolling through the screen when I run the command. Does this mean I've failed to successfully exclude the files, or does the tar command flash the names of files it's excluding, too? It would seem to me that sifting through the excluded folders would be a waste of processing time.

I guess what I want is either someone to tell me that tar needlessly sifts through excluded directories, or someone to tell me how to successfully exclude the removable hd.

Thanks!

mocha

October 7th, 2009, 04:07 AM

Regarding all this talk about excluding, I just use a text file which contains a list of all the directories I want to exclude and then pass the option

--exclude-from="/home/user/exclusions_list"

Or if you want to exclude directories at the command line just do like this

--exclude="/home/user/some directory/*" --exclude="/home/user/something/directory 2/*"

Using "" allows you to pass directories with spaces in their names.

The exclusions_list would simply look like this

/home/user/mythtv/*
/home/user/ISOs/*
/home/user/Long\ Name/*

Note the use of \ when having directories with spaces in their name.

Excluding specific files is done like this

--exclude=/home/user/backup_file.tar.gz

I think this should cover everything regarding exclusions.

newb85

October 7th, 2009, 06:40 AM

mocha

October 7th, 2009, 11:39 AM

Thanks, Mocha, I think you supplied the answer I needed. I would like something clarified, though. When excluding a directory and its content, I need to use an asterisk (e.g., /home/user/some\ directory/*), right? In Heliode's tutorial, /proc, /sys, /lost+found, etc. were excluded without an asterisk. Are they not directories?

I just do that out of habit. Yes, they are directories. If you're trying to exclude your external hard drive you have to find out its mount point. I suppose you could do that via Nautilus, I would do it at the command line though. Then just do something like --exclude=/media/disk/*

newb85

October 7th, 2009, 07:09 PM

I think I've found the source of my problem. Apparently, I was including more exclusions in the command line than the tar command would take. (Apparently, there is a limit, although a brief internet search did not yield the exact number.) However, by stashing all these exclusions in a text file and using the -X command, I met with success.

Heliode, it might be a good idea to add a note to your tutorial about this.

kevinguillorytraining

October 9th, 2009, 01:44 PM

Excellent guide. Thanks

ssjdennis

October 17th, 2009, 07:46 PM

Thx a lot of this Tutorial
I love it

lamadredelsapo

October 19th, 2009, 05:58 PM

Great howto

petteriIII

October 20th, 2009, 12:54 AM

Yes,
but what if i want also to remove files that are not inside my backup?

e.g. i copied 10 text files to "usr" folder...
If my backup don't contains files with the same name they will still be in "usr" folder after restoring..

It is quite possible, I made it with my system. To do it you must nuke your hard-disk-partition in your restore-program then bring your system back to life forexcample from backup in USB-key. But one error or messing with power and you are playing with nuked system.

Happy re-installations. petteriIII

miltonhork

October 20th, 2009, 01:23 AM

Hi
Well,Thanks for giving such a nice information regarding Backup and restore system .You have create a good awareness among others regarding these topic.I like your article very much.I want to add my knowledge regarding Benefits of this topic.You are truly knowledge giver and thanks once again for these information.

Thanks

kartal

October 30th, 2009, 02:24 PM

newb85

October 30th, 2009, 06:01 PM

Hey would you mind giving the whole command line you have there for this text file option?

I created a file which I called "exclusions.list" in the root (/) directory:
/proc/*
/mnt/*
/media/*
/lost+found/*

The command line was then:
tar -cvj --exclude-from=exclusions.list -f backup.tar.bzip /or simply:
tar -cvjX exclusions.list -f backup.tar.bzip /* The extension .list was only for my reference. You may make it anything or omit it altogether. You may also place the file anywhere and simply include the path, such as:

tar -cvjX /home/[username]/exclusions.list -f backup.tar.bzip /Btw is there a file size limit with the tar method?

I don't know of any.

kartal

November 1st, 2009, 12:03 AM

Thank you so much. I managed to back up this way. Although I went ahead with Remastersys solution as the final method for myself.

Roger Allott

November 1st, 2009, 09:46 PM

As this thread is so long, I'm sorry but I've no idea whether this question has already been asked, but ......

Is there any reason why one has to use tar via the CLI instead of running Archive Manager as root?

The benefit of using the GUI interface of Archive Manager seem rather obvious to me:
One doesn't have the potential problem caused by spelling errors. For example, if you mistype media (as mediia, for example) in the exclude list, you'll end up creating an enormous file containing all data on all disks connected, including your Windows C:\ drive in most people's configurations.
It is also much easier to create the backup file somewhere other than your root partition, which might be quite full.
It is better protection from total disk failure if your backup file is on a completely different disk to the data being backed up.

Something I'm not sure about though is how one preserves permissions using Archive Manager.

mjprashanth

November 12th, 2009, 05:40 AM

Great work..>Thank you so much...

Mustard

November 12th, 2009, 01:54 PM

It may have been mentioned in this thread already, but dont make the mistake I made and save your backup to a fat32 filesystem (on a USB drive for instance). I had forgotten that fat32 didnt support files over 4 gig, so my backup ended up being corrupted with an 'unexpected EOF' (end of file) error.

I wasnt using the straight 'tar method', but rather the sbackup application, which failed silently, leaving me in the dark and without a backup.

Another error I made using a pretty straightforward method of simply copying my entire /home to a separate external drive using the cp command. I didnt pay much attention to what I had actually copied over, but I had failed to copy over 'hidden files' so I lost all my old config settings.

mjprashanth

November 13th, 2009, 03:49 AM

Hope you have an answer already...Anyways, you just need to check if the media mount point is mounted with the Windows partitions. If yes, then exclude them in the backup command :)

Cheers...

I have a problem when I backup, it also backups all my windows partitions. It is huge file.

So what else do I need to exclude then?

THanks,:popcorn:

Neezer

November 13th, 2009, 04:50 PM

So I have a separate partition that is /home, and I have another 13GB partition that is mounted as /backup

the purpose of the /backup is for the backup file....How can I tell tar to write the file to /backup?

if I add --exclude=/home --exclude=/backup will that make it so that I don't backup those partitions?? I don't want to backup my /home cause that is where all my movies and music is and it would just be too large...I think.

ubfu

November 14th, 2009, 02:19 PM

Having about 200,000 files on my ubuntu partition .How long would it take to backup ? Having a x2 2Ghz processor ?

Logan1985

November 17th, 2009, 05:13 AM

Will this method work for software as well as documents?

I would like to re-install Ubuntu to take the option to encrypt the home partition and I'm trying to do it in the most painless way possible.

Kypdurron5

November 27th, 2009, 05:14 AM

Long story short- the restore didn't work for me. Specifically, after doing a full restore some things just didn't work like sound and internet, and I couldn't figure out why/how they were broken.

The long version is this....I have a ubuntu box with mythtv running as an HTPC. It was a real pain to setup because I had to do some video driver trickery and manually map channels (where switching to one that doesn't work crashes the frontend) to get the whole thing setup. SO, I really wanted to back it up exactly as-is so that I never had to do that again. Well, I upgraded to Karmic and found out that LIRC is broken with no fix as of yet (with my specific capture card). SO, I thought...no problem, I'll just load the backup. At first I tried the restore from within the OS using the overwrite option (I know, probably a rookie error), but the terminal froze half way through. So I formatted the partition from the Live CD and ran the restore and everything was fine (well, not exactly- I had to edit the menu.lst just to get it to boot because for some reason the HDD UUID wasn't the same...). Once I got it up and running though I realized the sound driver was messed up as was the internet. So I tried it again, this time using the Karmic backup I made before doing the restore and had the exact same problem.

Any ideas? This leads me to believe that there's some flaw in this backup method where it's not copying every file exactly as-is. I've resolved the problem by manually reinstalling and reconfiguring everything all over again...but now I need a backup solution that I KNOW will work because really, I'd commit suicide before doing this all over again :).

Ron_

December 1st, 2009, 05:18 PM

newb85

December 1st, 2009, 07:22 PM

I read the front of this thread, but even as a newbie I was extremely wary of sudo su. Too many tales of woe around that one. I am similarly afraid to sudo nautilus.

I elected instead to try SBackp for my LVM2 Ext4 system. I (think I) want to be able to restore individual directories rather than my whole /home partition.

SBackup produced root-owned files that reside on the same physical volume as the rest of my home/office system. Not much protection against fire/theft/head crash.

Can anyone provide a "safe" command line that I can use by rote (with deeper understanding later) to transfer sbackup output to dvd? Or is this a bad choice because of file size limitations?

If you've opted to use SBackp instead of the method in this tutorial, you would probably have better luck getting help in a thread related to SBackp.

If you only want to back up directories within your /home directory, you shouldn't have to sudo su to tar these files, unless you're backing up files for multiple users.

Ron_

December 1st, 2009, 10:00 PM

Thanks, newb85. Actually, I have tried other threads with no success. I am not the only one who has asked this question, and I am frankly astonished that I can find no answer amid years of replies.

This thread seems to be not of a single mind on the value of using tar. (See, for example, reply #948.) Other replies recommend SBackup (e.g., #174, #597) or other tools as better alternatives.

Thanks for pointing out that there is no need to use sudo for /home. True, of course. Just a bad example on my part. (I actually ended up with some root-owned files under /home, but it is not a generalizable problem.)

brendanpiater

December 5th, 2009, 06:41 AM

Check out "backintime" in the repos for Karmic. Apologies if it's been mentioned before.

slakkie

December 5th, 2009, 06:45 AM

But it's a commercial application.... :?

Then use clonezilla.

I don't get this howto tbh, use clonezilla and you don't need to tar anything... And it is a lot easier too.

TryingLinuxAgain

December 10th, 2009, 10:28 PM

First of all, THANK YOU for the great article.

I am using a simpler version of this just to backup "my" files:

tar cvpzf backup.tgz -exclude=backup.tgz ~

This seems to work fine, but it gives me a message "file changed during write". I suspect it is referring to the fact that my home directory (~) has changed because of the creation of backup.tgz

does the command above look OK to you guys as a command for a basic backup of everything in my HOME directory?

Signed: A complete linux Noob,
Thanks!

Ishachaitanya

December 11th, 2009, 03:52 AM

Backing up by AptonCD and restoring packages offline.

First one have to install aptoncd in the offline computer or the computer which has no internet connection.

The AptonCd can be downloaded with all dependecies in a folder of a internet computer and then can be installed in that internet computer and in any other computer which have no internet connection. To do this one has to be a superuser or administrator with password for the internet computer.

the command is as follows

root@ubuntu:/# apt-rdepends aptoncd 2>&1 | grep "^[a-zA-Z0-9_].*$" > /tmp/x1

apt-rdepends to be installed first in the internet computer if not already installed.
The above command is for creating a list for all dependencies for aptoncd with the main aptoncd package.

Next commands are

root@ubuntu:/# for f in `cat /tmp/x1` ; do apt-cache show $f | grep Filename | cut -f 2 -d ' ' ; done >/tmp/x2

mkdir mycache
[to keep the downloaded aptoncd package with all dependencies in the mycach folder under root]

root@ubuntu:/# cd mycache

root@ubuntu:/mycache# wget -B http://np.archive.ubuntu.com/ubuntu/ -i /tmp/x2

[if wget is not already installed then it is to be installed first before executing the above command]
this command will start downloading packages & all dependencies of aptoncd application.

Now copy the mycache folder to the home directory of the computers which have no internet connection and install it in those computers.

In the internet computer also install the aptoncd and make backup of all packages with upgrades and dependencies and make a CD/DVD/ISO image. Carry them to the computers having no internet connection and by using aptoncd application restore the packages and upgrades with all dependencies.

So by using only one computer with internet one can install applications in other computers having no internet connection at all.

newb85

December 11th, 2009, 08:05 AM

tar cvpzf backup.tgz -exclude=backup.tgz ~

does the command above look OK to you guys as a command for a basic backup of everything in my HOME directory?

Should be "-cvpzf" and "--exclude=backup.tgz".

Be aware that if you've mounted any other drives to folders in your HOME directory, this command will also back them up.

Also, I recommend additionally running a command to create an index of your backup:

tar -tvzf backup.tgz > backup.index.txt

Keep this index with your backup; it will make it a lot easier to find individual files later.

joes7

December 11th, 2009, 11:19 AM

Thanks, this is extremely useful:)!

Paul86fxr

December 12th, 2009, 06:09 AM

I tried all you had talked about and had the following error when moving the TGZ file from my disk to the root directory. any ideas?

AlexanderDGreat

December 19th, 2009, 10:34 AM

This completely worked for me I backup & restored my Mozilla Thunderbird Profile from my laptop to my PC at home. Here's what I did from the tutorial.

1. Go to my laptop
2. Open terminal
3. sudo su
4. cd /home/me/.mozilla-thunderbird/
5. tar cvzpf mozthun.tgz --exclude=./mozthun.tgz *
Wait to backup
6. cp mozthun.tgz /media/myusbdisk/

Restoring
1. Go to my Home PC
2. Open up terminal
3. sudo su
4. cd /media/myusbdisk/
5. cp mozthun.tgz /home/me/.mozilla-thunderbird/
6. cd /home/me/.mozilla-thunderbird
7. tar xzvpf mozthun.tgz -C .
Wait

Opened up Mozilla Thunderbird and all my message, settings, passwords, ARE THERE, unbelievable!

rikilshah

December 20th, 2009, 12:13 AM

This is awesome.But I have a problem...My File System drive is having very little space say around 3 GB not enough for backup.So What I want is to create tar in another drive.Is it possible?It would be great if you can show command.

PS:I am a little child in the world of Ubuntu.:)

newb85

December 20th, 2009, 11:57 PM

If you're trying to create the backup on another hard drive, hard drive partition, or large flash drive, you must first mount the drive. Then, in the command line, instead of just giving the name of the backup file you're creating, give the path/filename, where the path includes the mount point.

For example, usually if you plug in a thumb drive or external HD, Ubuntu automatically mounts it on the Desktop. Then, your path/filename would be something like ~/Desktop/8.0\ GB\ Flashdrive/Somefolder/backup.tbz

Of course, if the folder you're backing up includes the mount point, you need to exclude the mounted drive, or at least exclude the backup file.

I don't think it's possible/practical to try to write the file to a CD/DVD as it's created.

rikilshah

December 21st, 2009, 12:18 AM

Thanks buddy.For that first I'll have to learn Tar commands.I'll try it out and will put up result soon.:)

rikilshah

December 25th, 2009, 11:23 AM

After an usual Googling I found few good utilities which replaces command line with easy to use GUIs and plus extra features.

1>SBackup-Few Clicks and you are all set to backup!!Amazing thing is that you can schedule Backup hourly,daily,Monthly....and can restore any time you need..

2>APTonCD-All your installed packages would be on single iso file.So whenever you install your system no need to download them all again.It is very useful for people with slow internet(like me stuck@20KBPS).

This is very easy to use and very helpful.....

matthew.ball

December 27th, 2009, 12:25 AM

In a similar vein, I wrote a bash script which saves a copy of the fstab, the sources.list and exports a file containing the currently installed packages.

I also have a complementary script which does the opposite and restores the files, and imports the package list.

It sort of relies on having a separate /home partition. It was just for me as I have broken my system a few too many times and remembering the names of installed packages was becoming tedious, I found some stuff about backup/restore online, and I saw the opportunity to automate the process (and learn some bash!), someone on irc asked about the script and I figured it could come in handy.

It's only about 12 lines per file, pretty easy to replicate but I'll happily give it to anyone who is interested.

Edit: I was going to have it copy the contents of /var/cache/apt/archives and tar it up, and could it easily add it. I just figure most people clear it out anyway so they'd rarely have anything to save.

I don't have an upload for it, sorry. If you're interested send me an email (details should be in my profile), and I'll forward the files (and a README) on.

AlexanderDGreat

December 27th, 2009, 04:41 AM

Hi, can I download your script? I want that. :)

imousavi

December 27th, 2009, 09:47 AM

I want 2 restore from a directory with this address : /media/New Volume

how should I edit this code
tar xvpfz backup.tgz -C /

BarryM

January 4th, 2010, 10:24 AM

Hello there, I hope someone out there can help me with this. I want to make a backup of my entire Ubuntu drive to an external (USB Drive). As far as I can gather, the method suggested by Heliode at the very start of this thread puts the backup on the desktop. Am I right there?

I have got my external drive connected and mounted (it is a FreeAgent drive, and appears as "FreeAgent Drive" in the directory tree). I have managed to navigate to it via the Media directory (I found out that I needed to put the name in quotes for Ubuntu to recognise it in the root terminal).

If I now want to make my backup, having got to the directory in the FreeAgent drive that I want to store the backup in, will it work with the "tar cvpzf backup.tgz --exclude=/... (exclude list) /"? How do I stop the contents of the FreeAgent drive being included in the tarball? Should I add --exclude="/media/FreeAgent Drive" in the exclude list?

Any advice will be greatly appreciated.

Thanks

SecretCode

January 4th, 2010, 11:05 AM

I admit to having not read all 99 previous pages of this thread.

Instead of using tar or rsync for a full drive backup, I would use an imaging solution like Clonezilla. With this, you select either the whole drive or one or more partitions on the drive. No problem with excluding external drives. And if you need to restore, you can restore to bare metal - you don't need a working installation, just the Clonezilla disc.

There are a couple of gotchas: for example you must not mount the external drive first - Clonezilla will make you select from unmounted drives to save the image on.

You can download the Clonezilla Live CD, or certain other CDS that include it like PartedMagic.

pro003

January 4th, 2010, 12:46 PM

smiffysnail

January 6th, 2010, 03:50 PM

This totally worked for me!!! thanks and cheers.

idn't get to try the right way as exactly in the tut.

what i did

went to vista - deleted ubuntu folder

re-install from cd

moved backup file to root

followed the command in tut.

rebooted

it whined about something, so, did a 'long-hold' on power button to power down

booted

fine!

so i've condensed your tutorial down to the bare essentials as i have used them. hope you don;t mind...

first boot up from install disk as a visitor, then go to terminal and begin...

### become root
sudo su

### cd into root(can go anywhere you want backup to end up, including remote or removable drives)
cd /

### full backup command
tar cvpzf backup.tgz / --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/host --exclude=/dev --exclude=/sys --exclude=/media

### explanation
"The 'tar' part is, obviously, the program we're going to use.

'cvpfz' are the options we give to tar, like 'create archive' (obviously),
'preserve permissions'(to keep the same permissions on everything the same), and 'gzip' to keep the size down.

Next, the name the archive is going to get. backup.tgz in our example.

Next comes the root of the directory we want to backup. Since we want to backup everything; /

Now come the directories we want to exclude. We don't want to backup everything since some dirs aren't very useful to include. Also make sure you don't include the file itself, or else you'll get weird results.
You might also not want to include the /mnt folder if you have other partitions mounted there or you'll end up backing those up too. Also make sure you don't have anything mounted in /media (i.e. don't have any cd's or removable media mounted). Either that or exclude /media.

At the end of the process you might get a message along the lines of 'tar: Error exit delayed from previous errors' or something, but in most cases you can just ignore that."

### restore
tar xvpfz backup.tgz -C /

### warning
"this will overwrite every single file on your partition with the one in the archive!"

### re-create excluded directories
mkdir proc
mkdir lost+found
mkdir mnt
mkdir sys
etc...

### reboot

smiffysnail

January 6th, 2010, 03:54 PM

you can store the file on the seagate, but you have to copy it into file systems before doing the restore command. thats what i ended up doing anyway. all the best, this really does work.

BarryM

January 7th, 2010, 05:36 AM

Many thanks to the various respondents on this. I successfully made the backup using tar. I did find it took rather a long time, especially with Olympus RAW pictures and a BBC iPlayer video, I realise I should perhaps have excluded the Videos folder as well!

I tried the Clonezilla live CD but got a slight problem with getting to the right drive for storing the backup, so will try again later (I am just doing all of this in preparation for an upgrade, in case anything goes badly wrong, so I can get back to square 1).

I suppose I can use a tarball to back up my Windoze drive as well (sadly I have to use that other operating system for some applications that do not have Linux alternatives)?

erniej

January 8th, 2010, 03:41 PM

I'm a new convert running Karmic, and I love it. A trial run of Heliode's backup-and-restore method worked fine for me. However, I have a new desktop on the way and, although I've searched this and other threads, I can't find direct answers to the following:

1. Is a disk image or a simple automatic sbackup the best way to backup the old HD and restore to the new? I had a devil of a time getting the settings right for a Linksys router and a Canon printer. I still don't know what I did right to get them to work, and I'm a bit paranoid about losing the settings to those devices in particular.

2. The old box is 2002 vintage and runs a 32-bit OS. The new one is 64-bit. Will the drive-image/sbackup-restore method transfer successfully to the new one? Will the sbackup automatic version? Should I install 64-bit Karmic first, then transplant the old files?

3. The new machine will have a new graphic card and other internal hardware. If I restore the disk image will that cause something to implode?

Many thanks in advance for any replies. And please use small words in any responses: I'm obviously very new to this and on the wrong side of 65 to boot. :-)

abdulmajid

January 11th, 2010, 10:42 PM

What if the UIDs of the swap areas are different?

XaeroDegreaz

January 14th, 2010, 03:44 AM

It should be noted that when you restore from your backup, your backup file will ALSO be deleted if it is on the hard drive.

tirengarfio

January 16th, 2010, 06:15 PM

Hi,

I'm getting this error with the backup command of the post #1.

/sbin/halt
tar: /: file changed as we read it
tar: Error exit delayed from previous errors
$

I remember the command:

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

Any idea?

Javi

Marius Cloete

January 21st, 2010, 05:51 AM

Hi all,

Here is a shell I have put together from this post. I am not a guru on shell scripts this was put together with the help of google search :P.
Feel free to comment and/or change it.
I saved it as backupAndRestore.sh and you can run it as follows
sudo ./backupAndRestore.sh
enjoy

#!/bin/bash
S1='b'
S2='y'
echo "Please enter b for backup or r for restore:"
read action

if [ $action = $S1 ]; then

echo "These Directories will be excluded:"
echo -e "\033[1m\033[32m /proc /lost+found */backup.tgz /mnt /sys /dev"
echo -e "\033[0m"
echo "To chance these values edit this shell"
echo "continue (y/n):"

read action
if [ $action = $S2 ]; then
echo "Backing up PC ~ timestamp " ;date
tar cpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=*/backup.tgz --exclude=/mnt --exclude=/sys /
fi
else
echo -e "\033[1m\033[31mWARNING: this will overwrite every single file on your partition with the one in the archive!"
echo -e "\033[0m"
echo "Please enter name and location of the archive to restore:"
read fileName
echo "You are about to restore $fileName do you want to continue?(y/n)"
read action
echo -e "\033[1m\033[47mJust sit back and watch the fireworks.This might take a while. When it is done, you have a fully restored Ubuntu system!"
sleep 5
echo "start"
if [ $action = $S2 ]; then
tar xvpfz $fileName -C /
echo "Creating excluded directories"
#Just to make sure that all excluded directories are re-created
mkdir /proc
mkdir /lost+found
mkdir /mnt
mkdir /sys

fi
fi
echo -e "\033[0m"