Howto: Backup and restore your system!

[scorp123]

there would be another simple option though: dump. "dump" is only useful if you can 100% guarantee that you will always backup + restore to / from the same hardware with the same disks and the same disk geometries ... E.g. "dump" is OK if you work with commercial UNIX-es and have e.g. your server room full of shiny new SUN server hardware and your storage room full of identical spare parts ...

But for a home user? Naaah, "dump" is too inflexible.

"tar" offers several clear advantages:

1. it does not care about the underlying filesystem ... whether it's ext3, reiserfs, XFS .. for "tar" this doesn't matter the least bit. You can without troubles restore the contents of a "ext3" filesystem to a new filesystem with "reiserfs" or SUN's ZFS or Windows NTFS ... It doesn't matter. It will work. With "dump": NOPE. Whatever filesystem you had when you did the "dump" will be the same filesystem you get when you do the "restore".

2. "tar" doesn't care about the underlying disk geometry. Whether you packed a 250 GB " / " filesystem and then unpack it onto a 50 GB drive ... "tar" doesn't care for as long as there is enough free space. But "dump" cares: If you change the disk geometries or play around with the partitions ... bye bye data, bye bye music collection. Unless I have missed any vast progresses "dump" is still the same stupid program it was 20 years ago and it just doesn't like it too much when you try to restore the dumped filesystem to a different geometry.

3. "tar.gz" files are de facto a universal standard. Anyone can read those files. Anytime. Anywhere. On any platform. Even on Windows, e.g. there are several tools such as WinZIP which are able to open "tar.gz" files and let you access your data. But "dump"? Nope. If for some stupid reasons you don't have the needed tools ready you won't be able to read those backups you made ... and a backup that can't be read is de facto worthless.

I am not saying that "dump" is just plain "bad" ... it has its uses and there are scenarios where I'd rather go for "dump" (see above). But fact is: for the average home user "tar" is the better bet.

Also ... something people seem to misunderstand: With "tar" you do file backups and that's what most people would want. With "dump" you do filesystem backups ... and that's not exactly the same thing.

[shankscomp]

Question... Is there a possible way at all to restore a ubuntu system without having to use the LiveCD??? I had to restore my ubuntu server last night, and it was a pain to take and move it from the shelf and connect it to a keyboard/monitor just to run the livecd. I'm wondering if there is a work around to do this remotely.
Thanks
Tim

[Patsoe]

"dump" is only useful if you can 100% guarantee that you will always backup + restore to / from the same hardware with the same disks and the same disk geometries


I should say I haven't really looked into this in detail, but you can in fact restore individual files (see eg. here: http://surf.ap.seikei.ac.jp/~nakano/dump-restore/dump-restore-mini-HOWTO.en.html#s4 )

The other two points indeed speak strongly in favour of tar; it works on any file system and can be read in every OS. I think I'll reconsider my choice! Thanks.

[scorp123]

but you can in fact restore individual files Of course. But it's a pain in the a** ... You'd need to do a lot of things manually here, which takes time, and from that POV "tar" beats "dump" again, because even if you only wanted to restore a single file from a "tar" archive: oh well. You just specify what you want on the command line and "tar" will automagically do it for you, requiring very little if none at all interaction from you. And that's a big plus, especially if you have other things to do (who doesn't??) you can easily leave "tar" in the background and let it do its stuff. Not so with "dump" and its restore tool if I remember it right.

Best thing is this: Try it out. Install a PC and then make two backups:
- one with "tar"
- one with "dump"

And then destroy the data on the PC, e.g. format the disks so all data is lost. And then do a restore, a complete "bare metal" recovery. And then repeat the procedure. And then destroy a few important files and see how quickly you can get those files back ...

You will see which one works best for you.

[PJSingh5000]

nofear1101,

Your missing files problem is exactly why I like to boot with the Live CD before making a backup.

The last time I ran into your problem. I only had the /media folder (and its required contents) missing, so I manually recreated the missing files using konqueror:

$ kdesudo konqueror /

(By the way, don't use kdesudo with Dolphin).

I compared the "restored" system with a second working installation on another computer, created the missing files, and set the file permissions identically.

May be someone else can suggest an easier way to identify and recreate the missing files?

[bizzu]

Hi guys, very nice tutorial ;)
What if I want to encrypt my backup?

[PJSingh5000]

Try using KGpg to encrypt/decrypt. You should store your back-up (whether or not it is encrypted) in a safe place-- perhaps burn it to a DVD or copy it to a separate physical drive. As per step 4 below, keep a copy of your Secret Encryption Key, which requires a password to function.

(1) Install Encryption Utility for Kubuntu...
$ sudo apt-get install kgpg
(Ubuntu users: $ sudo apt-get install seahorse)


(2) Start Encryption Utility...
"K" | "Utilities" | "KGpg"
(You may have to click on the "Lock" icon in the taskbar to maximize the GUI).


(3) Create an Encryption Key...
From the menubar, select "Keys" | "Generate Key Pair"
In the "Name" text box, enter a name for your key, for example "My Key".
Select "RSA" from the "Algorithm:" dropdown.
Click "OK".
Click "Continue"
Enter, and verify, your secret password in the "Password" fields.
Click "OK".
(You may chose to set this key as your default key).
Click "OK".


(4) Export (Backup) your Secret Key...
In the "Key Management" window, right-click on your Key.
Select "Export Secret Key..."
(Make sure you select "Export Secret Key...", not "Export Public Key...")
Click "Export".

(Note, be sure to keep your exported Secret key file in a safe place, so you can import it into KGpg in case you have to restore your system from an encrypted backup. If you lose this key file, you will not be able to decrypt an encrypted file even though you know the password!!! To import the key file, simply drag it into the "Key Management" window, click "Import, and click "OK".)


(5) Encrypt Your File...
Navigate to your file in Dolphin.
Right-click on your file.
Select "Actions" | "Encrypt File"
In the dialog that appears, select the key you created in step 3 (ex. "My Key") and click "OK"
An encrypted file will be created with an *.asc extension.
Note: a large file (such as a backup) may take some time to encrypt.
Delete your original *un*encrypted file.


(6) Decrypt Your File...
If necessary, import the key file into KGpg by dragging it into the "Key Management" window. Click "Import, and then click "OK".
Navigate to your encrypted file (with the *.asc extension) in Dolphin.
Click on the encrypted file.
In the dialog that appears, enter your password.
Note: a large file (such as a backup) may take some time to decrypt.

[Nightwalker07]

http://www.truecrypt.org/ :)

[Xebec]

My simple implementation of Apple Time Machine like backup. Hopefully it will be useful for somebody.
Disclaimer: rsync is pretty new thing for me.

These are the contents of my archive.sh:

#!/bin/sh
BACKUP_ROOT="/media/sdc1/backup/rsync/"
THIS_BACKUP="$BACKUP_ROOT`date|sed \"y/ /_/\"`"
cd $BACKUP_ROOT
PREV_BACKUP="$BACKUP_ROOT`ls -rt|tail -1|sed \"y/ /_/\"`"
rsync -axy --link-dest="$PREV_BACKUP" /home/xebec "$THIS_BACKUP"
BACKUP_ROOT is the folder where all backups are stored. Make sure it does not contain spaces and is located on linux partition. Make sure /home/xebec is replaced with the list of folders and files you want to back up.

Basically, rsync will create fully identical copy of your folders every time you start the script. However, If you mount some other drive into your home folder, rsync will ignore it, since -x option is specified.

The reason I use rsync instead of plain old `cp -a` is that it can compare your folders specified for backup with previous backup. And this is the coolest part: in case the identical file is already there, rsync will not copy it over again. Instead it will create a hard-link, preserving your hard disk space and time. I have about 13 gigs to backup, but each new backup takes only about 50 megs of space on average. That's why I don't even bother deleting my old stuff.

In case you want to delete some old folders, the hard-links in your newer folders are going to be safe, since your file system will delete only the contents of the files which do not have any hard links left.

You probably want to schedule automatic backups. So you open
sudo gedit /var/spool/cron/crontabs/root
Here is my entry to the crontabs for automatic backups every day at 4 am:
0 4 * * * /home/xebec/my_scripts/archive.sh >>/root/backup.log 2>&1
/root/backup.log can be replaced with /dev/null if you don't need backup logs.

[kaginken]

If you only want to backup certain directories and then use ncftp to get the tar file off the machine to a local ftp site...this works great! You'll need to have ncftp installed, and tweak the 'ftpserver', 'ftp*' spots, etc. Paste this into a file, save it as 'backup.sh', set it to run nightly as a cron job and you're done.

Notes: This backs up any dirs listed in DIRS - just list as many as you want seperated by a <space>. Most of what needs tweaked are in the first two sections (System Setup and FTP Server Setup).
-------------------- shell script follows ----------------
#!/bin/sh
# System backup script
# Full backup day - Wed (rest of the week do incremental backup)
# ---------------------------------------------------------------------

### System Setup ###
DIRS="/var/www/html /etc/nagios /root"
BACKUP=/tmp/backup.$$
NOW=$(date +"%d-%m-%Y")
INCFILE="/root/tar-inc-backup.dat"
DAY=$(date +"%a")
FULLBACKUP="Wed"

### FTP server Setup ###
FTPD="ftpdirectory/incremental/"
FTPU="ftpusername"
FTPP="ftppassword"
FTPS="ftpsharename"
NCFTP="$(which ncftpput)"

### Other stuff ###
EMAILID="you@hotmail.com"

### Start Backup for file system ###
[ ! -d $BACKUP ] && mkdir -p $BACKUP || :

### See if we want to make a full backup ###
if [ "$DAY" == "$FULLBACKUP" ]; then
FTPD="/ftpserver/full"
FILE="fs-full-$NOW.tar.gz"
tar -zcvf $BACKUP/$FILE $DIRS
else
i=$(date +"%Hh%Mm%Ss")
FILE="fs-i-$NOW-$i.tar.gz"
tar -g $INCFILE -zcvf $BACKUP/$FILE $DIRS
fi

### Dump backup using FTP ###
#Start FTP backup using ncftp
/usr/local/bin/ncftp -u"$FTPU" -p"$FTPP" $FTPS<<EOF
mkdir $FTPD
mkdir $FTPD/$NOW
cd $FTPD/$NOW
lcd $BACKUP
mput *
quit
EOF

### Find out if ftp backup failed or not ###
if [ "$?" == "0" ]; then
rm -f $BACKUP/*
else
T=/tmp/backup.fail
echo "Date: $(date)">$T
echo "Hostname: $(hostname)" >>$T
echo "Backup failed" >>$T
mail -s "BACKUP FAILED" "$EMAILID" <$T
rm -f $T
fi
-------------- end shell script ----------------

Alternatively, here's where I generated that script, a cool script generator, it just prompts you for some info and spits out a script:

http://bash.cyberciti.biz/backup/wizard-ftp-script.php

Enjoy! :guitar:

[traxtar3]

quick question..

i have my /home on a seperate partition, when i do the backup i want to exclude it. my question is when i restore it, will it screw with anything in the /home dir?

[xfile087]

quick question..

i have my /home on a seperate partition, when i do the backup i want to exclude it. my question is when i restore it, will it screw with anything in the /home dir?

If you exclude /home in your backup then when you restore it, it shouldn't touch it at all. I do this myself and it works a treat, plus it's quicker to backup!

[BassKozz]

Would it be better to use 7z (http://en.wikipedia.org/wiki/7z) (p7zip (http://p7zip.sourceforge.net/)) to do this (rather then using tar)?
This "should" create a smaller backup archive file... correct?

[herger]

Hi all!
The advice to use a tar.gz or tar.bz2 to restore your system is WRONG!!!
DON'T TRUST THIS METHOD!!!!
I followed the instructions and obtained a 4 Gb tar.bz2 file.
My laptop crashed when I tried to configure virtualbox (another bad chance) and I tired to restore my system by extracting the backup.tar.bz2 as explained at the beginning of the thread.
Result: it failed in extracting such a huge file and I had to reinstall everything from the beginning.
Pay attention: DON'T TRUST THE "tar" METHOD TO RESTORE YOUR SYSTEM!!!
I think I will do an image of my hd and keep it very carefully on a separate hd.........
Your depserate

H

[erginemr]

Would it be better to use 7z (http://en.wikipedia.org/wiki/7z) (p7zip (http://p7zip.sourceforge.net/)) to do this (rather then using tar)?
This "should" create a smaller backup archive file... correct?

As far as I know, 7-zip does not preserve file owner and permissions information, which is crucial for a successful restore, and therefore, cannot be used as a replacement to tar:
http://www.ohloh.net/projects/3769/reviews

[erginemr]

Hi all!
The advice to use a tar.gz or tar.bz2 to restore your system is WRONG!!!
DON'T TRUST THIS METHOD!!!!
I followed the instructions and obtained a 4 Gb tar.bz2 file.
My laptop crashed when I tried to configure virtualbox (another bad chance) and I tired to restore my system by extracting the backup.tar.bz2 as explained at the beginning of the thread.
Result: it failed in extracting such a huge file and I had to reinstall everything from the beginning.
Pay attention: DON'T TRUST THE "tar" METHOD TO RESTORE YOUR SYSTEM!!!
I think I will do an image of my hd and keep it very carefully on a separate hd.........
Your depserate

H

This is a very interesting phenomenon. Thank you very much for pointing out the file size limit problem for the uninformed Ubuntu user.

I have googled for a while on this problem and have found these:

1. http://answers.google.com/answers/threadview?id=25116
This page asserts that, depending on the file system used, there is no limit to the maximum file size in "tar". Note also the last user comment therein.

2. http://www.codecomments.com/archive309-2005-11-691598.html
This page draws the attention to "gzip" (and also possibly "bz2"), which has a size limit of 2 GB, due to the nature of a "zip" file.

3. http://aplawrence.com/Bofcusm/2078.html
This page suggests using "pipes" to extract the tar file from a large-scale gzip / bz2 file, as a promising workaround to the problem.

[El_Belgicano]

Hi all!
The advice to use a tar.gz or tar.bz2 to restore your system is WRONG!!!
DON'T TRUST THIS METHOD!!!!
I followed the instructions and obtained a 4 Gb tar.bz2 file.
My laptop crashed when I tried to configure virtualbox (another bad chance) and I tired to restore my system by extracting the backup.tar.bz2 as explained at the beginning of the thread.
Result: it failed in extracting such a huge file and I had to reinstall everything from the beginning.
Pay attention: DON'T TRUST THE "tar" METHOD TO RESTORE YOUR SYSTEM!!!
I think I will do an image of my hd and keep it very carefully on a separate hd.........
Your depserate

H

Maybe you did something wrong, it work perfectly for me, except some minor problems that were solved within 5 minutes ...

1) tar backup:
sudo tar cvpzf /wheretobackup/backup.tgz --exclude=/etc/fstab --exclude=/tmp --exclude=/boot --exclude=/proc --exclude=/lost+found --exclude=/media --exclude=/mnt --exclude=/sys /
2) bad things happens to my installation making it unusable
3) fresh install with 7.10 CD
4) untar the whole archive
5) fix these minor problems

total time to be back up and running: 90 minutes or something like that.
archive was about 4.5 gigs...

[BassKozz]

As far as I know, 7-zip does not preserve file owner and permissions information, which is crucial for a successful restore, and therefore, cannot be used as a replacement to tar:
http://www.ohloh.net/projects/3769/reviews

Bummer :(
You think the guys who work on the 7z archive format might add this feature?

[PJL]

Hi,

I've just had an experience similar to Nightwalker07 in post #393 when unpacking a tarred backup of my ubuntu 7.10 server. The responses from tech9 about excluding /boot fixed the problem - hooray! (I figured out that I could save a copy of the /boot from the fresh installation somewhere else before unpacking the tar file and then overwriting the /boot with the saved copy. Using --exclude at the unpacking stage didn't seem to work)

However, my question is: Can someone edit the original post to add the requirement to exclude /boot from the tar file? This could save people like me a lot of hassle - I know I was lazy for not reading all fifty-odd pages of the thread in the first place, and I know that tech9 has subsequently started a new thread on the topic, but *this* is the thread that came up when I searched the topic, and it has the appearance of being definitive so to be hit with a problem at the restoring stage was a bit of a pain (fortunately it happened at a non-critical time though).

I would also just like to say that these forums ROCK! Thanks to everyone for contributing and helping me to learn so much.

[neverliveneverdie]

Ok I think you're right. I'll edit the howto.
But if you just recreate the dev directory after restoring, does the system automatically recreate the device nodes?
Somehow I don't think i've ever had the problem you discribe. My backups have always been somewhat proportional to the amount of data that was actually on the disk; I didn't have it suddenly double in size or anything. But if Linux just recreates the device nodes by itself then I guess it is save to exclude 'dev'.

Hi i checked to see if it recreated the device nodes and it did infact recreate them on the boot. :-D

[rally4life]

Okay, this may not be the best place for this, but it seems to be related.

Here's my situation. I teach in a computer lab in an elementary school with about 26 PCs. In the past I have used Ghostcast 7.5 to reimage the computers with Windows XP. I decided that I wanted to Dual Boot and tried to reimage with that setup, but everytime I try to broadcast the image I get an error. Is there another way to do this, I really want to get Ubuntu on these computers but I don't wanna have to install and configure every computer individually.:confused:

[terry_gardener]

this is excellent.

just one question when i move the file from / to the back drive on NAS using:

sudo mv /backup.tgz /media/backup

gets message saying unable to preserve file premissions of file premission denied.

the file copies. looking at the premissions tab it changes from owner root and group root, to owner terry group nogroup.

does this mean all the folders within the backup file lost the premissions as well or is it just the backup file itself.

thanks

[scorp123]

Hi all!
The advice to use a tar.gz or tar.bz2 to restore your system is WRONG!!! And how so? I use this since 1996 and it works. You just have to do it right :)

I followed the instructions and obtained a 4 Gb tar.bz2 file. And? Modern Linux file systems have file size limits somewhere in the 2 TeraByte = 2000 Gigabyte range. A file of just 4 GB in size is no problem for Linux file systems. According to Wikipedia:
http://en.wikipedia.org/wiki/Ext3#Size_limits

A filesystem with 4KB blocks (seems to be standard? All my filesystems have that according to dumpe2fs command) has a max. file size limit of 2 TB = 2000 Gigabytes and a max. file system size of 16 TB = 16'000 Gigabytes.

I am sorry to say but without further details I'd assume you did something wrong as a mere 4 GB tar file should really not be a problem whatsoever.

[herger]

Hi. Yes, You could be right, but I have followed the procedure described at the beginning of the thread in each single reported step .
Did You try to restore Your system with a 4 Gb or 2000 Gb single file?
Can You provide any more details?
Even in the restoring, I have follwed any word of the steps: as I told You, it didn't work.
Also some friends of mine confirmed that it's quite hard to restore an intere filesystem with a .tgz single file (the advice is to use PartImage)....
I would only understand where I could have make a mistake in following a procedure that doesn't appear to be so terrible....

Thanks

H

[scorp123]

Did You try to restore Your system with a 4 Gb or 2000 Gb single file? The biggest single archive file I so far had to deal with in my career with was 54 GB. But other than that: Yes, I regularly face large files, also on my laptop (HP dv2108ea, old 32-bit Intel Core Duo 1.6 GHz, 2 GB RAM, 100 GB disk). No problems whatsoever.

I would only understand where I could have make a mistake in following a procedure that doesn't appear to be so terrible.... Well, you have to know what you do. The system can't help it if you blindly type in some commands ... chances are that the commands are not 100% right for your system and thus will not produce the results you expect. It can't harm to read the manuals, the so called "man" pages. The manual of "tar" can be accessed with this command (use cursor keys, PgUp and PgDown to navigate ... Use the "Q" key to quit!): man tar

I myself once wrote this "Backup How To" for the Linux Mint forum --- and this stuff works. And yes, it is complicated but at least this will get the system back no matter what (been there, done that, saved my job and my life ... ). BUT: as I keep repeating in that text: you really have to know what you do!

http://linuxmint.com/forum/viewtopic.php?t=3969

There are a few other threads where people discuss other backup + cloning (which is sort of the same thing) options:

http://ubuntuforums.org/showthread.php?t=616707
http://ubuntuforums.org/showthread.php?t=675933

As I wrote earler ... doing backups via "tar" has several huge advantages:

- the tool is already there out of the box on all Linux and UNIX variants and available on many Live CD's
- it works the same on all the many platforms
- even Windows tools (WinZIP, WinRAR, many others) can read *.tar.gz files

In other words: With *.tar.gz files chances are very good that you will get your files back no matter what the circumstances, if you do it right, that is.

[agentnixon]

Hi, I am having trouble with this method. I recovered all my files and everything, but when I restarted, the computer froze up at the ubuntu loading screen. I can't get into the system anymore. any help is appreciated...
Edit: Never mind. There is a solution to this problem on page 39. Search is your friend. :)

[scorp123]

Hi, I am having trouble with this method. I recovered all my files and everything, but when I restarted, the computer froze up at the ubuntu loading screen. I can't get into the system anymore. any help is appreciated...
Edit: Never mind. There is a solution to this problem on page 39. Search is your friend. :) Let me guess: Problems with the now different UUID's? That's why I in my tutorial (see the link above to the Linux Mint forums) wrote to get rid of them and use old classic device names instead, partition UUID's really can cause complications when you restore your system.

[dmitrijledkov]

I want to change partition table of my computer and I want to use this method to backup and then restore ubuntu installation.

Will anythingl freak out after restoring since the partition table changed? My thought is grub will.

Is there a way to backup grub separately? So that this works?

Backup all; Change partition map; Install Ubuntu; backup grub; Restore all; Restore grub.

EDIT: I guess I just need to backup menu.lst and everything should be fine =D.

[z-vap]

I didn't read this whole thread, so I don't know if this was already mentioned (51 pages)...

when you do the excludes, you could have a /* added to the end of each directory, that way the directories are recreated upon restore, and all the temp stuff inside are not. Saves a step when recreating the directories.

ex

--exclude=/lost+found/* --exclude=/proc/* --exclude=/media/* --exclude=/dev/* --exclude=/mnt/* --exclude=/sys/* --exclude=/tmp/* /

[jeremycollins08]

Just a quick question...

I have ubuntu Gutsy installed with dual boot alongside Windows XP. Will this still work? I put in the code exactly as you did in the beginning, and it's currently creating the file. I have separate Windows and Ubuntu partitions. Will this back up windows also?

[erginemr]

Just a quick question...

I have ubuntu Gutsy installed with dual boot alongside Windows XP. Will this still work? I put in the code exactly as you did in the beginning, and it's currently creating the file. I have separate Windows and Ubuntu partitions. Will this back up windows also?

No, it won't. That's what the line "--exclude=/media/*" is for. I also have been using the same backup technique for quite some time in my dual-boot system and can safely recommend it.

[jeremycollins08]

No, it won't. That's what the line "--exclude=/media/*" is for. I also have been using the same backup technique for quite some time in my dual-boot system and can safely recommend it.

Ok...Thanks! This is a very easy way to backup the system! Way better than windows!

[herger]

Hi.

For dmitrijledkov:
I really recommend to follow scorp123 thread here

http://linuxmint.com/forum/viewtopic.php?t=3969

It is much more detailed and explained than the advice at the beginning of the thread.
It answers also to Your doubts regardin the grub manager.
Thanks to scopr123 and to all of You.
Regards

Herger

[gnumd]

Nice how-to!

In reading these 54 pages.... whew! my head hurts.

I am noob... running an eeePC and Ubuntu off an SDHC and WINXP off the SSD (dual booting). (I have an 8G and four 8gig SDHC cards and one USB SDHC card reader).

I would like to back-up my machine particularly as I learn and try various things I find on the forums.

A Few questions:
1) Can I back up my SDHC to another SDHC and restore from the back-up SDHC to either the SSD or the original SDHC? I imagine yes... knowing what is what is a bit difficult with strange folder names and identities for the various drives. Also the exclude media option probably would exclude my installation... I don't know? Please help.
2) Since I am running Ubuntu off an SDHC, should I presume linux is powerful enough to access and back up the WINXP data on my SSD? I hope so because I would like to move my WINXP installation as a back-up file and restore it on the SDHC (ie. Can this method back-up my windows install from the SSD to an SDHC and then restore it onto an SDHC instead of the SSD?). Why would I want to do this? Well simply putting my middle finger firmly against the SDHC card to eject/remove WINXP would be a fantastic power trip - plus I'd rather move to using Ubuntu as my primary but still have the full XP that works as needed on a Stick.
3) In other words, can the windows install and ubuntu installs be switched by using this back-up method in question 2? If so, how does one fix the GRUB loader to go to the correct places for running each OS?
4) This is how-to is one of the best and very cool and appears very powerful... so many caveats and my own stupidity has me afraid to do anything. But I am motivated to give it an informed try and learn along the way.

My fear has me wanting a more automated method with a GUI so I know I am doing the right things? If I understood the terminal commands better then that would not be the case... but I am uninformed and getting the right information on these forums can be daunting.
Thanks to this thread I did find this link - it almost seems too simple:
https://help.ubuntu.com/community/BackupYourSystem/SimpleBackupSuite

I welcome help/guidance and simple idiot proof instructions.
I love Linux and especially Ubuntu... but I'm always worried about bricking my system and not knowing how to undo what I did wrong.
Being able to back-up and restore efficiently would be awesome and empowering!

Thank you for everyone's knowledge and hard work in answering the multitude of questions!

-Steve

[Sit3UserX]

Hello.

Is it possible to do minimal backups from some user configs/settings and also do a thing like get a list of the apps in the main menu? I'm not really interested in do full and big backups of my ubuntu partitions.

*******

Edit:

I see a could use sbackup

[Berean]

Nice how-to!

In reading these 54 pages.... whew! my head hurts.

I am noob... running an eeePC and Ubuntu off an SDHC and WINXP off the SSD (dual booting). (I have an 8G and four 8gig SDHC cards and one USB SDHC card reader).

I would like to back-up my machine particularly as I learn and try various things I find on the forums.

A Few questions:
1) Can I back up my SDHC to another SDHC and restore from the back-up SDHC to either the SSD or the original SDHC? I imagine yes... knowing what is what is a bit difficult with strange folder names and identities for the various drives. Also the exclude media option probably would exclude my installation... I don't know? Please help.
2) Since I am running Ubuntu off an SDHC, should I presume linux is powerful enough to access and back up the WINXP data on my SSD? I hope so because I would like to move my WINXP installation as a back-up file and restore it on the SDHC (ie. Can this method back-up my windows install from the SSD to an SDHC and then restore it onto an SDHC instead of the SSD?). Why would I want to do this? Well simply putting my middle finger firmly against the SDHC card to eject/remove WINXP would be a fantastic power trip - plus I'd rather move to using Ubuntu as my primary but still have the full XP that works as needed on a Stick.
3) In other words, can the windows install and ubuntu installs be switched by using this back-up method in question 2? If so, how does one fix the GRUB loader to go to the correct places for running each OS?
4) This is how-to is one of the best and very cool and appears very powerful... so many caveats and my own stupidity has me afraid to do anything. But I am motivated to give it an informed try and learn along the way.

My fear has me wanting a more automated method with a GUI so I know I am doing the right things? If I understood the terminal commands better then that would not be the case... but I am uninformed and getting the right information on these forums can be daunting.
Thanks to this thread I did find this link - it almost seems too simple:
https://help.ubuntu.com/community/BackupYourSystem/SimpleBackupSuite

I welcome help/guidance and simple idiot proof instructions.
I love Linux and especially Ubuntu... but I'm always worried about bricking my system and not knowing how to undo what I did wrong.
Being able to back-up and restore efficiently would be awesome and empowering!

Thank you for everyone's knowledge and hard work in answering the multitude of questions!

-Steve

Steve,

May I suggest that you start a new thread so that your questions are addressed individually, and not lost within the myriad pages of this thread? Regards.

[ACGarland]

Something else to consider: there are various factors which contribute to limit the maximum size of the archive produced by tar.

http://backuppc.sourceforge.net/faq/limitations.html#maximum_backup_file_sizes

Even if tar and the source OS allow for fairly large files, the destination file system (e.g., FAT32 on a USB drive) may further reduce the maximum.

I ran into this limitation trying to use this technique to backup about 36 GB of data (which includes large ISO and virtual machine images). I didn't even get up to the tar size limit (8 GB for some older versions of tar from what I've read on the net)--because the FAT32 destination file system can't handle anything larger than 4GB.

It appears that modern versions of tar can handle very large files:

https://answers.google.com/answers/threadview?id=25116

[HW_Hack]

Dazed and confused --- as an IT person I know my way around doing backups with MS based crap and of course doing disk imaging ala Symantec products.

Pretty clueless on the deeper parts of linux - I came across this post and thought great I can do a disk image -- which I guess you can but I've not had success -- and reading all 50 some pages just hurts.

Now I can easily backup all my linux user crap (music - pics - docs - etc) to a USB drive. But I want to plan for a disaster and easily recover ...

For example I'm using Gutsy - I get everything all installed upgraded - wireless working - etc. Now I want a image snapshot -- so I do the tar command as presented in this post ...

Now I somehow really screw-up my install .... how do I restore ? I think some steps / assumptions are missing here ...

So I boot the 7.10 live CD and re-install the base system - I plop the tar file into the root dir become su and execute the tar restore file


Ok heres the problem --- we excluded /proc and other directories BUT they were re-created by the re-installing the 7.10 base image -- and the original instructions of course say to create these after the restore tar

I've tried the above method and the linux system boot to a text prompt that says (initramfs)

So it seems like a step or 2 are missing
1 - we have a tar file that needs to be plopped into the root dir of "something" ... but that something doesn't have a /proc dir ?
2 - Still sounds like you need to have an emergency boot environment and ultimately it would be great if it was a graphical one so you could easily mount a USB drive - move files etc.

I'm confused

[scorp123]

how do I restore ? The hardcore method (tested + tried + it works!): Please read the postings here:

http://linuxmint.com/forum/viewtopic.php?t=3969

That pretty much covers everything from backup to "bare metal recovery" including change of partitioning (e.g. you partitioned the to-be-restored system differently than the original one).

As for your questions:

we excluded /proc and other directories BUT they were re-created by the re-installing the 7.10 base image Not exactly true. /proc is a virtual file system and its contents will be recreated after every reboot.

It's like having a direct conduit to talk to your running kernel. Instead of having to bother with special commands to get the kernel to talk to you all the kernel's inner workings are being displayed as files underneath /proc ... Example: What were the exact parameters your running kernel was started with? Execute this:
cat /proc/cmdline ... And you have the answer.

Or another example: Notice all those numbers in the directory? These are PID's = programm ID's of running tasks ... e.g. if you execute a command such as "top" or "ps -efH" you will get the very same numbers in the PID column. Example taken from my system:

I have "Amarok" running here ... so let's ask the system what the PID is: pidof amarokapp ... And I get this back: 28196

Now I can go into /proc and see what's underneath the directory with the very same number: ~ > ls -al /proc/28196
total 0
dr-xr-xr-x 6 ddj ddj 0 2008-04-13 14:52 .
dr-xr-xr-x 170 root root 0 2008-04-06 03:55 ..
dr-xr-xr-x 2 ddj ddj 0 2008-04-13 22:35 attr
-r-------- 1 ddj ddj 0 2008-04-13 22:35 auxv
--w------- 1 ddj ddj 0 2008-04-13 22:35 clear_refs
-r--r--r-- 1 ddj ddj 0 2008-04-13 14:53 cmdline
-r--r--r-- 1 ddj ddj 0 2008-04-13 22:35 cpuset
lrwxrwxrwx 1 ddj ddj 0 2008-04-13 16:40 cwd -> /home/music
-r-------- 1 ddj ddj 0 2008-04-13 22:35 environ
lrwxrwxrwx 1 ddj ddj 0 2008-04-13 14:52 exe -> /usr/bin/amarokapp
dr-x------ 2 ddj ddj 0 2008-04-13 16:40 fd
dr-x------ 2 ddj ddj 0 2008-04-13 22:35 fdinfo
-r--r--r-- 1 ddj ddj 0 2008-04-13 22:35 maps
-rw------- 1 ddj ddj 0 2008-04-13 22:35 mem
-r--r--r-- 1 ddj ddj 0 2008-04-13 22:35 mounts
-r-------- 1 ddj ddj 0 2008-04-13 22:35 mountstats
-rw-r--r-- 1 ddj ddj 0 2008-04-13 22:35 oom_adj
-r--r--r-- 1 ddj ddj 0 2008-04-13 22:35 oom_score
lrwxrwxrwx 1 ddj ddj 0 2008-04-13 16:40 root -> /
-rw------- 1 ddj ddj 0 2008-04-13 22:35 seccomp
-r--r--r-- 1 ddj ddj 0 2008-04-13 22:35 smaps
-r--r--r-- 1 ddj ddj 0 2008-04-13 16:40 stat
-r--r--r-- 1 ddj ddj 0 2008-04-13 22:35 statm
-r--r--r-- 1 ddj ddj 0 2008-04-13 17:20 status
dr-xr-xr-x 9 ddj ddj 0 2008-04-13 22:35 task
-r--r--r-- 1 ddj ddj 0 2008-04-13 22:35 wchan See all those files? "fd" for example is short for File Descriptors ... In other words: Which files have been opened by this task? So instead of issueing any mysterious commands you simply could take a file browser and investigate the contents there or issue simple shell commands such as "ls -al" or "cat name_of_file" to investigate this.

What I am trying to tell you .... /proc is more or less irrelevant for your backup.

It's a virtual file system ... Reboot and it will have changed. Stop or open new tasks and it will have changed again. Backing up /proc is therefore so totally not relevant here!

That's why most backup tutorials out there (incl. mine above) tell people to exclude /proc from the backup process.

I hope this clears it up a bit?

[djbon2112]

I have a question.

I'm a little concerned about the line "WARNING: this will overwrite every single file on your partition with the one in the archive!" If I've excluded a folder (say, "/home") and I extract the TAR archive with " tar xvpfz backup.tgz -C / ", will it actively delete the excluded folder when extracting, or will it be left alone? I ask because I excluded my /home and /usr folders (both on separate partitions) and the last thing I need right now is to loose all my data and applications! I'd assume they'd be left alone, but the wording of the lines underneath it ("You'll have to recreate <list of locations>") makes me a little uneasy with my assumption.

[scorp123]

If I've excluded a folder (say, "/home") and I extract the TAR archive with " tar xvpfz backup.tgz -C / ", will it actively delete the excluded folder when extracting If you have excluded a folder when you created the archive then there are no contents of that folder in the archive in the first place. So when you extract your files again then there should be nothing that the excluded folder(s) can be overwritten with. And nope, it should not delete any files on the filesystem just because they are not present in the archive. That would be deeply counter-productive and I have never observed such behaviour so far.

But you could check ... Have tar list the contents, e.g.
tar tzvf backup_filename.tar.gz | more .. that should list the contents of the archive and you'd see if anything would be overwritten. So in your case there shouldn't be any files going into /home ... If that's the case then nothing underneath /home should be overwritten or deleted.

[hackNperl]

Has anyone ever used CloneMaxx? http://www.pcinspector.de/Sites/clone_maxx/info.htm?language=1

[thecure]

Instead of ghosting the whole drive image, I prefer to use rsync to back up my data on an external drive but works great with NAS. Lots of resources to look at on how to do it... I'm sure on the forums somewhere.



http://www.arsgeek.com/?p=637

http://users.softlab.ece.ntua.gr/~ttsiod/backup.html

https://help.ubuntu.com/community/rsync

rdiff looks very interesting also

http://www.nongnu.org/rdiff-backup/

[TheTouti]

Hi Forum,

I decided to get rid of Vista (I know, please don't even say it :oops:) and move to Linux so I've installed Ubuntu Desktop. The first thing I want to learn is how to backup/restore so I checked this thread.

I didn't read all the 55 pages as it would have taken too much time and I apologize if the answer to my question is already there somewhere but I'm wondering how one would restore from a compressed file in the case of a hard disk crash.

I understand that if my OS or data gets corrupted or something I can boot from Live CD, copy my backup file back on the disk and decompress it but what if I have to change the HD ?

Would I have to install Ubuntu first and decompress my backup on top of it ? Would that restore all my programs and configurations and make my new Ubuntu installation the same as the previous one ?

I know I would certainly not try this in Windows.

[ACGarland]

One approach is to use MondoRescue which allows you to backup and recover your machine from 'bare metal', even in situations where the restore hardware (partitions, etc.) differ from when the backup was made.

http://www.mondorescue.org/

[Patsoe]

...Would I have to install Ubuntu first and decompress my backup on top of it ? Would that restore all my programs and configurations and make my new Ubuntu installation the same as the previous one ?

I know I would certainly not try this in Windows.

You would use a LiveCD to partition and format your new hard disk, then you would decompress the archive onto the new hard disk. At this point, you'd already have a working system except it can't boot itself yet. However, you can then chroot to "hand over control" from the LiveCD to the restored system, and then tell the system to reconfigure the GRUB boot loader.

In fact, even with another motherboard and chipset than in your original system you still have a good chance of pulling this off (unless the hardware is too new). And no, you wouldn't be able to do something like that with a Windows system :)

(edit: actually I just ran into this post http://ubuntuforums.org/showthread.php?t=764684 - you need a few more steps to get it going on a system that's set up to mount by uuid)

[rugbert]

What kind of compression rate does tar have?

Im not sure but I think my backups are missing somethings.

using df I calculated that my total disk usage is about 9.3 gigs but my backups are only 3 gigs. I mean thats some crazy stuff good compression right there.

also - somewhere in the thread was an option to have cron make logs for the backups right?

[psyion]

I have a dual boot setup for vista and ubuntu (latest) i have almostf inished setting up both windows and linux teh way i want after i formated a week ago.

If i use this method will it back my windows and linux systems up so i can just run teh restore command and get a totaly clean fully customised windows and linux? Will it work for both or just linux?

Thanks guys

[erginemr]

I have a dual boot setup for vista and ubuntu (latest) i have almostf inished setting up both windows and linux teh way i want after i formated a week ago.

If i use this method will it back my windows and linux systems up so i can just run teh restore command and get a totaly clean fully customised windows and linux? Will it work for both or just linux?

Thanks guys

This method will back up only your Linux partition. For Windows, you will have to use some proprietary software, such as Norton Ghost...

[Patsoe]

This method will back up only your Linux partition. For Windows, you will have to use some proprietary software, such as Norton Ghost...
The particular recipe given in this tutorial will indeed only back up the Ubuntu file system. That doesn't however mean that you need proprietary software to back up a Windows partition. You could well do that with e.g. dd. Also, partimage has experimental support for NTFS.

[oyvinst]

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /


You don't need the '-p' option when creating archives. The -p option is only relevant for extraction.

[kennethmsmith]

ok.. I'm going to give this a try as soon as I get Ubuntu 8.04 configured the way I want it. But if this kills my drive, I'm coming after you guys :P lol... just kidding. Thanks for the detailed how-to's. Keep-em coming.

[scorp123]

You don't need the '-p' option when creating archives. The -p option is only relevant for extraction. Not true! If you omit "-p" then the permissions might not be recorded and hence the files might have the wrong permissions when you extract them!

[oyvinst]

Not true! If you omit "-p" then the permissions might not be recorded and hence the files might have the wrong permissions when you extract them!

How about reading the manual ? And what do you mean by "might" ? You really are not sure, are you ?

For your pleasure:

Create two tar archives of the same directory tree as root, omitting '-p' for the first one and including it one the second. Make sure you have some files/dirs with different owners/perms in the tree, so that tar can record it.
Compare the MD5 sums of the two archives.
Go read up on the tar info pages: http://info2html.sourceforge.net/cgi-bin/info2html-demo/info2html?(tar.info.gz)Option%2520Summary

[scorp123]

How about reading the manual? The same could be applied to you.

And what do you mean by "might"? Are you familiar with "umask" and those things, yes? Depending on who is doing what and where and when permissions might get screwed up. Go look it up, it's a fact. Not all variations of "tar" behave the same. I guess you know what I mean by this? So to avoid troubles from start I'd suggest to always use the "-p" parameter and you never ever have to worry about that. Remember: beginners are reading this. Can't hurt to have them develop the habit to use the "-p" parameter. Just in case.

Knowledgeable folks like you probably know when and how to leave what parameter away and when not ... the many beginners here most likely don't.

[oyvinst]

The same could be applied to you.

And why is that ? I have read it, and I have not stated anything that contradicts what the manual says.


Are you familiar with "umask" and those things, yes? Depending on who is doing what and where and when permissions might get screwed up. Go look it up, it's a fact. Not all variations of "tar" behave the same. I guess you know what I mean by this? So to avoid troubles from start I'd suggest to always use the "-p" parameter and you never ever have to worry about that. Remember: beginners are reading this. Can't hurt to have them develop the habit to use the "-p" parameter. Just in case.

Knowledgeable folks like you probably know when and how to leave what parameter away and when not ... the many beginners here most likely don't.

I know what umask is. Read my initial reply, again. Yes, '-p' matters for extraction. And so I said. This is what the tar manual refers to as 'Writing', i.e. tar writes files when extracting.

A HOWTO should contain correct examples of usage. That's why I pointed it out, and that's why I bothered to reply to your claim that my statement wasn't true. But hey, doesn't really matter, forget about it, it'll work for people however they do it, as long as they include '-p' when restoring their stuff.

[scorp123]

That's why I pointed it out, and that's why I bothered to reply to your claim that my statement wasn't true. OK, I'll admit that saying "not true!" was like shooting mosquitos with a 120mm cannon .... My apologies then.

But hey, doesn't really matter, forget about it, it'll work for people however they do it, as long as they include '-p' when restoring their stuff. Agreed. :)

[kennethmsmith]

Wow... I just looked back and realized how long this thread is. I had only read the first page when I posted my last comment about giving this a try. 557 posts later, there is still discussion about the code and implementation of this procedure?? I'm not so sure I want to try it now :)

So someone PLEASE... for all of us worn out by reading all the content in this thread... PLEASE...summarize is the original HOW-TO is flawless and works in Ubuntu 8.04, and/or should we use one of the other multitudes of options posted by various users in this thread. I'm very confused now as to whether I should use this method or not.

[kennethmsmith]

or maybe post a shorter step by step how-to that cleans up this 557 post thread.

[miiko]

To the original poster, thanks a bunch. The original command string worked fine and pulled my chestnuts out of the fire at least twice now with my old T22.

[frunns]

Would this work on a Windows partition? I'm looking for a decent way to back my system up "now in the beginning", so I won't have to f'ing reinstall it later.

[Patsoe]

Would this work on a Windows partition? I'm looking for a decent way to back my system up "now in the beginning", so I won't have to f'ing reinstall it later.

Sounds like you're looking for something that allows a "bare metal recovery" of your Windows system. I think tarring the Windows partition is not the best option for that.

For a rather comprehensive but maybe not so easily readable guide to doing what you want, look here: http://www.backupcentral.com/components/com_mambowiki/index.php/Linux_%26_Windows_Bare_Metal_Recovery
(It is a free chapter from the O'Reilly book "Backup and Recovery").

[frunns]

Sounds like you're looking for something that allows a "bare metal recovery" of your Windows system. I think tarring the Windows partition is not the best option for that.

For a rather comprehensive but maybe not so easily readable guide to doing what you want, look here: http://www.backupcentral.com/components/com_mambowiki/index.php/Linux_%26_Windows_Bare_Metal_Recovery
(It is a free chapter from the O'Reilly book "Backup and Recovery").

I ended up doing a PartImage image instead, sounded good enough at least. :)

[Stefanie]

Excellent guide :)

I'd like to try this, but I have very limited space on this hard drive (laptop) and I'm pretty certain I don't have room to generate the backup here. Is there a way to redirect the resulting backup.tgz to shared folder on my windows network instead of a local partition? Perhaps by using smbmount to mount the share, then create the backup.tgz from that location?

how exactly do you do this? or suppose you want to create the tar-file on another partition?

[rugbert]

So my backups are running at half mass. My file system is about 10 gigs and the largest backup Im getting is about 5.5 gigs.

I ran it live (and also piped the output to a file) and it looks like it stops at /sbin/mii-tool

[mocha]

Just my 2 cents, try to keep your tar.gz archives to less than 2 GB of source files. Anything more than this is going to take FOREVER to restore from, this is personal experience on an overclocked dual core. Especially if you are like me and occasionally restore just a few files from a large archive. Keeping the archives smaller also makes it easier to transfer them via FTP or burn them to DVD.

[irshadcharm]

can anyone post the command for backing up and restoring my ubuntu system to my external WD Passport? How to edit the command? Please the post the detailed instructions on how to do that.... Im a newbie and please post the command asap....Im not able to get into my /media/WD Passport folder to store my backup...plz help guys...i love ubuntu and trying to learn ...this is my first hand experience with ubuntu....i havent used any linux system so far but trying to get into the linux world after a great disappointement with vista...waiting for a reply....

[slvrmoontiger]

Is this method feasible when working with RAID. Specifically I am setting up a Server to run RAID 5 with exception of /boot and swap which will be on RAID 1. I will be using Software RAID to do this.

Thanks,
slvrmoontiger

[scorp123]

Is this method feasible when working with RAID. Why shouldn't it be? "tar" doesn't care about your disk setups, it will read whatever is on the filesystem, it does not bother how the disks are physically setup. IMHO this also makes it ideal for e.g. migrations, e.g. migrating data off from one system to a system with a different disk setup.

[koleasamoila]

but if the /home dir(even other hd) is on the other partition (even other hd, should i do a separately backup of the files which are in it?

[Damiox]

Nice guide.

But I used Norton Ghost which works perfectly for me. Norton Ghost 2003 supports EXT2/EXT3 filesystems. Older versions of Norton Ghost DON'T support EXT3 filesystems or Linux altogether.

Please go to this site (http://service1.symantec.com/SUPPORT/ghost.nsf/docid/1999021909463125?Open&src=&docid=2000033111503625&nsf=ghost.nsf&view=docid&dtype=&prod=&ver=&osv=&osv_lvl=) for a feature compare between the different Norton Ghosts versions: Norton Ghost compatibility with Linux. (http://service1.symantec.com/SUPPORT/ghost.nsf/docid/1999021909463125?Open&src=&docid=2000033111503625&nsf=ghost.nsf&view=docid&dtype=&prod=&ver=&osv=&osv_lvl=)

I ended up with a backup-image that's about 1 GB big, the amount of data on the disk that was backed-up by Norton Ghost was about 2 GB. So as you can see the compression was very good. I went from 2 GB to 1 GB. :grin:



is it just me or does that sound like somebody who works for norton...tring to sell uss some crap like norton when linux has "free" back up tools...Norton has been crap ever since thay stoped making norton commander....Ha and im the retail it consult....guess what other program im supposed to push and and end up giving the customer the addy to avg....HEHE...Down with the corporations....

[scorp123]

but if the /home dir(even other hd) is on the other partition (even other hd, should i do a separately backup of the files which are in it? That depends on what command you issue and what parameters you give to "tar" .... If you tell "tar" to backup "/" and that it should not worry about traversing onto other filesystems, then "/home" will be backed up too. On the other hand, if you backup "/" and issue the "--one-file-system" parameter, and "/home" is on another filesystem and/or on another disk, then it will not be included here.

You need to know what you want and why. For both scenarios above are perfectly valid reasons. If in doubt check the manual: man tar

[MachineHead]

Hi all,

I have a question about backing up files in another sense than this guide but nevertheless, backing up files.

I switched from Windows to Ubuntu, indeed. (Ubuntu Hardy 8.04)
When I would re-install my windows, I would always have my movies, music and important files on a physical different hard drive than my OS.
After the re-install, Windows would read the partition as D; or E:, whatever and I can acces my files again.

I have Ubuntu in dual boot at the moment with Win Xp Pro Sp2.
And I need to re-install Windows, thus re-install Ubuntu, because I've heard that Windows doesn't like to be second choice.

I need to be 100% sure of the answer concerning my question.

Can I put my movies, music, important files on a physical different hard drive than my Ubuntu OS, format that hard drive as Ext3 and use all my files after an Ubuntu re-installation????

So Ubuntu won't see the data on that drive as corrupted in any sense and I would lose it all?
I am talking homework, essays, loads of information for classes I want to teach in the future, pictures, just all common home office files a person can have.

It's just too much to put on an external, I can get my personal files on a USB stick though, which I will do aswell to be sure.
But my movies, music etcetera, it's just way way way too much.

Please reply to this and let me know.


Regards,

MachineHead


I just added a new physical drive to my system of 1 TB and partitioned it using GNOME Partition Editor,but now I can't find my new partition anywhere?
How do I get a shortcut for my new partition.
Can't find it in /media also.

[Patsoe]

I have Ubuntu in dual boot at the moment with Win Xp Pro Sp2.
And I need to re-install Windows, thus re-install Ubuntu, because I've heard that Windows doesn't like to be second choice.

I think whoever told you that may have been confused... maybe, before going through the trouble of reinstalling all that, you'd want to open a forum thread about your problem?

So Ubuntu won't see the data on that drive as corrupted in any sense and I would lose it all?

That should work just fine, but nobody can (or should) guarantee that. If the data is important, back it up. Use your new 1TB volume for it!


How do I get a shortcut for my new partition.

It needs a mention in /etc/fstab (see man fstab for details). If you have more questions, those should probably go into a new thread (because it's off-topic in this one).

[scorp123]

And I need to re-install Windows, thus re-install Ubuntu Huh?? Nope. That's not neccessary and total and utter and complete overkill. It will be just fine if you re-write the boot block again. The other half of your posting is true: Windows doesn't like other Operating Systems and it will mercilessly overwrite whatever boot block was on your harddisk if you install Windows after an alternative OS ... Hence the rule to always install Windows first, because other OS'es behave better.

But even if: You could Google for "Super GRUB disk". That's a minimal Linux CD ("minimal" in the sense: There isn't much on the disk!) that will get your boot block back. Just read the instructions on their web site and pay close attention to the on-screen menus ... getting your boot block back should take you less than 5 minutes.

Or you may want to take a look into this thread:

"How to restore Grub from a live Ubuntu cd."
http://ubuntuforums.org/showthread.php?t=224351

... As far as I can tell they pretty much discuss your problem over there, ie. what happens if you install Windows as 2nd OS and how to get your boot block back without having to reinstall .... :)

[sofasurfer]

This thread has gone on WAYYYYYY to long. After 3 years, are the instructions at the beginning of this thread still relavant? I know there have been new versions since it was begun so I wonder if the process has changed?

I have looked for a long time for a backup process that is easy for me to understand. Well I finally found it, I think/I thought, until I now see that this thread is 3 years old.

Forgive me for being lazy ( I just spent a week trying to get my tv tuner working and ended up with a glitchy system that I now have to redo), but I just can't bring myself to read the 51 pages of this thread. Will I be ok to just follow the instructions in the first post?

Thanks.

[scorp123]

This thread has gone on WAYYYYYY to long. After 3 years, are the instructions at the beginning of this thread still relavant? The "tar" program was first created in 1970. Current versions are based on a variation created in 1988 or so. In other words: this thread could be 40 years old, it would still be relevant. "tar" is a tested and tried backup method and the "why and how" hasn't changed so much over the past 20+ years or so, as far as I can tell.

I know there have been new versions since it was begun so I wonder if the process has changed? I once wrote a similar guide for the Linux Mint forum. Basically it's more or less the same as this one here, but I keep maintaining it. So if you're interested in this method and want a to read a thread that isn't yet 50+ pages long, please go here:

"How to backup your stuff UNIX-style"
http://www.linuxmint.com/forum/viewtopic.php?f=42&t=3969&start=0&st=0&sk=t&sd=a

... until I now see that this thread is 3 years old. And the methods described here are almost 40 years old ;)

Will I be ok to just follow the instructions in the first post? I personally find that first post a bit too shallow, e.g. it doesn't really tell you why you have to do what you do. And it doesn't go into deep technical details. I personally prefer to know *WHY* I have to do something when I do it and I always assume that others think the same, hence my own thread to backups covers far far more than the first posting here. But basically they describe the same thing, e.g. how to backup your system using "tar". If you are fine with quickie instructions and don't need to know / don't want to know too many technical details then yes, the first posting in this thread should do, there is nothing wrong with it.

[MachineHead]

I personally find that first post a bit too shallow, e.g. it doesn't really tell you why you have to do what you do. And it doesn't go into deep technical details. I personally prefer to know *WHY* I have to do something when I do it and I always assume that others think the same, hence my own thread to backups covers far far more than the first posting here. But basically they describe the same thing, e.g. how to backup your system using "tar". If you are fine with quickie instructions and don't need to know / don't want to know too many technical details then yes, the first posting in this thread should do, there is nothing wrong with it.

I see this problem recurring in many HOWTO's and tutorials.
Although many folks carry goodwill and contribute to the community, some HOWTO's are stated like it's obvious.

I started using Ubuntu Linux for real, day in day out less than 4 months ago because of the lack of hardware support I experienced before 8.04.
Now I can use Linux, but the entire philosophy of using my pc is changed.
The entire filesystem, all extensions I have ever known as executable are now just...data?
At least that is as much as I understand or misunderstand.

The "Why" might be even more important than the actual HOWTO, understanding why something has to be done in a certain way makes it easier to comprehend the steps you have to take to get the result desired.

@Heliode...

This is in no way a complaint on your person.
It's my objective view of the contents in many many HOWTO's and tutorials.


Respect,

MachineHead

[tankerkevo]

There's 58 pages of replies, sorry if I am duplicating a post. But see also http://ubuntuforums.org/showthread.php?t=581680 for another thorough tutorial on creating a recoverable image of a disk.

[scorp123]

http://ubuntuforums.org/showthread.php?t=581680 for another thorough tutorial on creating a recoverable image of a disk. I personally don't recommend using disk images. With disk images you're copying *WAY TOO MUCH* ... You copy *everything*. So if you have a 200 GB disk on which there are only 30 GB of data then by creating a disk image via the "dd" utility you are still copying the full 200 GB ... and not just the used sectors. Even if you compress the image ... getting your data back is quite a piece of work: Compressing and uncompressing 200 GB of data is way more CPU- and RAM-intensive than unpacking single files from an archive which is only a 10th that size, maybe even less.

And what if you e.g. somehow ruined your "good PC" (e.g. a really expensive gaming machine?) and all that's left is your 7+ years old "unused old PC"? You really want to find out how long it takes to unpack e.g. 200 GB of data on an old Pentium-3 with 512 MB RAM? With "tar" I could get my data back even on Windows or on a Mac if I had to (programs such as WinRAR, WinZIP and others have no problems whatsoever reading a Linux *.tar.gz archive!) ... with an image created with "dd"? You'd need enough disk space and if you just need to get back a few single files you will need to "loop mount" the dd-image first and then extract the files from the filesystem ... And if you just thought "Huh what? Loop mount? What's that ... ???" then the "dd" method *definitely* isn't for you, because you just demonstrated that you don't know how you'd need to get your data back if you need to ;)

Also: What if you change your disks one day? As soon as your disk geometry changes you are in trouble again. Sure, you can unpack a 200 GB disk image onto a 500 GB disk. What about the lost 300 GB then? You will soon find yourself confronted with lots of complications, e.g. how to increase partitions filesystems without ruining them, and so on.

And what if you want to unpack your backup on a totally different disk setup? e.g. a set of RAID disks or logical volumes? Because you copied the entire disk incl. the filesystem headers (MBR, partition tables, etc.) you will likely overwrite the headers on your new disk setup too, unless of course you know what you do and you know how to get around that .... If you don't know it then you're again in trouble :)

"tar" doesn't bother about the disk geometry at all. It just packs and unpacks files, so from that point of view it offers many advantages, e.g. you can move your data between totally different machines and can focus on getting your files back, and don't need to worry about the physical disk setup aside from having enough free space for the unpacking, but that's more or less it.

Ultimately: Do what you think works best for you. But "tar" IMHO indeed deserves the title of being a "simple + stupid + it works!" backup solution. For the simple reason that getting any lost files back is just a matter of opening an archive file, and that archive file is standardised, so even many non-UNIX / non-Linux programs (e.g. WinRAR, WinZIP, many others too) can read the format if really needed. The more programs can read such a format the higher your chances of *somehow* getting your data back in one piece, no matter how weird the circumstances.

Aliens just landed and kidnapped every Linux user you ever knew on this planet and at the same time you'd need to get your data back but you can't remember the exact parameters and for some weird reason the user manual ("man" shell command) doesn't work anymore, and because everybody is blogging about the aliens and up- and downloading videos from and to YouTube, the Internet has completely broken down, you can't reach the forums here, you can't download any Linux Live CD, you can't chat with anyone and ask for the precise instructions ...? Ask your Windoze friends: Their "WinZIP" shareware can read your backup too and can get your data back! With "dd" disk image? Nope!

Yes, that was a weird example :lolflag:

[stldirty]

i got almost all the way to the end when i got this error

tar: Error exit delayed from previous errors

it stopped on /usr/share/foo2oak/icm/

what can be the problem?

[scorp123]

what can be the problem? Most likely no problem at all, you just reached the end of your filesystem and there is nothing left to pack or to compress.

If you are not sure:
tar tzvf nameOfArchive.tar.gz | more ... and then compare the names of the files you get with what's on the harddisk, especially take a good look at the file lengths:
ls -alR /path/to/dir | more

e.g. you could do a "tar tzvf" on your archive and then do a comparison with what's in your /usr folder:
ls -alR /usr | more

My guess is that the file you mentioned is just the very last one and there is nothing left to be packed, hence "tar" exits.

[stldirty]

Most likely no problem at all, you just reached the end of your filesystem and there is nothing left to pack or to compress.

If you are not sure:
tar tzvf nameOfArchive.tar.gz | more ... and then compare the names of the files you get with what's on the harddisk, especially take a good look at the file lengths:
ls -alR /path/to/dir | more

e.g. you could do a "tar tzvf" on your archive and then do a comparison with what's in your /usr folder:
ls -alR /usr | more

My guess is that the file you mentioned is just the very last one and there is nothing left to be packed, hence "tar" exits.
i have a lot of stuff in that tar. tens of thousands of files. is there a better way to do it than to scroll line by line looking at each file?

EDIT: nvm i figured out that it actually was the whole thing. thanks man.

[sofasurfer]

I backed up the system and it worked fine. My question is this, Why is there a copy at /backup.tgz and also at /home/daryl/Documents/backup.tgz?

I then removed the one in documents by right clicking and choosing 'move to trash'. The one at /backup.tgz was removed by "rm -R /backup.tgz. Hopefully this won't cause me any problems.

[drewster1829]

The original HowTo needs to be updated for newer systems using UUIDs instead of device names in /boot/grub/menu.lst and /etc/fstab.

(That is if you're restoring to a different system, or you managed to bork your partitions, I think. I'm not sure how UUIDs work, but after overwriting my partitions, the UUIDs for the new partitions changed, even though it was the same hard drive and same system.)

The key is to do:
sudo gedit /etc/fstab

and uncomment (remove the #) from the /dev/hdXX lines, add a # to the UUID lines, and cut and past everything after the end of the UUID (starting with mount point / something). In other words, start with this:

# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
#/dev/sda1
UUID=0d715e99-8762-4d17-9a23-fd433dbcfd8e / ext3 defaults,errors=remount-ro 0 1
#/dev/sda5
UUID=bffbc922-af74-4e3c-b7c2-3b6be6d7f343 none swap sw 0 0
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto,exec 0 0

Change it to this:

# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
/dev/sda1 / ext3 defaults,errors=remount-ro 0 1
#UUID=0d715e99-8762-4d17-9a23-fd433dbcfd8e
/dev/sda5 none swap sw 0 0
#UUID=bffbc922-af74-4e3c-b7c2-3b6be6d7f343
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto,exec 0 0

For the /boot/grub/menu.lst (this is required to boot! If you system will not boot, pop in a LiveCD and mount your systems hard drive, then open up sudo gedit (or w/e text editing program you like) and edit this file), change this section (at the bottom):

title Ubuntu 7.10, kernel 2.6.22-14-generic
root (hd0,0)
kernel /boot/vmlinuz-2.6.22-14-generic root=UUID=0d715e99-8762-4d17-9a23-fd433dbcfd8e ro splash quiet
initrd /boot/initrd.img-2.6.22-14-generic

To this:

title Ubuntu 7.10, kernel 2.6.22-14-generic
root (hd0,0)
kernel /boot/vmlinuz-2.6.22-14-generic root=/dev/sda1 ro quiet splash
#UUID=0d715e99-8762-4d17-9a23-fd433dbcfd8e
initrd /boot/initrd.img-2.6.22-14-generic


Where /dev/sda1 is whatever your boot hard drive is. NOTE: The above examples are from my system. DO NOT copy and paste from this post, but from your own files. Makes backups if you're not sure what you're doing. The parameters might be different for your setup, and I was only providing an example of what needs to be done.

This really should be added to the original HowTo, as it's a necessity on any Ubuntu system using UUIDs. Otherwise, there is a good chance that your restored system will refuse to boot.

[santius]

Thank you very much! Its really usefull information for me, I will do it so next time I have problems I will try this.
I have a question, wich files in my system you recomend me to have backuped in case something goes wrong, for example, I have my Xorg.conf backuped, but, do you recomend something else to have backuped? Yesterday for example, my sound card stoped working, and I had to reinstall all sound related drivers...

Thanks!

[sjude]

I want to restore from my laptop to my desktop partition.

On my laptop I installed ubuntu within windows (c:\ubuntu directory) and on my desktop can I restore it to a Physical Linux partition. After restoring I get Grub error. I followed the procedure to restore Grub in both the installation method and the Live CD method from http://ubuntuforums.org/showpost.php?p=169013&postcount=6 but the command install-grub is not found.

[weetfield]

I'm a complete beginner on Linux, looking for a bit of sanity after years over the M$ barrel. I wasn't going to get too deep into Kubuntu 8.04 Lts without finding some kind of 'System Restore' method to get out of my ongoing mistakes.

This thread has been very good for me and I have read a fair amount after the search for a system restore etc. turned up page 50. Thank you all. A couple of my own beginners comments that might help others in the same boat:-

I tried using both the live Kubuntu 8.04 cd and SystemRescueCd from the partimage stable (http://www.sysresccd.org/Main_Page), using both the partimage system and tar.tgz

A lot of my early attempts failed because I was trying to save to an external USB drive. This always seemed to save OK but nearly always failed to restore. I don't know whether this is just my hardware or a system bug. There is an advantage in using the partimage system in that you can do a trial restore without overwriting your existing files. Both work fine on any other internal drive (inc NTFS, mount -tntfs-3g )

I only worked on the Kubuntu partition itself, now about 3 GiB, all my personal data (I guess /home) is on a completely separate drive.

One problem noted in an earlier post. Continuous cycling of the login page. I got this after restoring a tgz file. Nothing actually wrong with the file or the restore itself though. After the restore I had exited without unmounting the partitions. Kubuntu reported an improper shut-down during start-up and "fixed" it. It then booted OK to the login page but looped repeatedly. The same tgz file restored OK later if both the partitions were unmounted before exiting.

Last little point. rm -r /mnt/installation/* used to clear the system partition for restore asked me to confirm every file. -rf worked a lot faster.

Thanks again

weetfield

[BLTicklemonster]

On the 59th page, I still see stuff that would just blow my mind, lol.

I so want to be able to do this, but just don't have confidence in it after reading this thread off and on since it's inception.

I know, I know, other people do it all the time without any problems.

[weetfield]

I know, I know, other people do it all the time

To some extent I guess this is the ONLY way to do it.

Like any emergency procedure, you can only rely on it if you:- i) know how to use it, and ii) have proved that it works.

My personal data is separate from the OS, backed up and (proven) bombproof. All I am risking at the moment is a fresh install of kubuntu with one simple data mount, say one hour max from the live CD. I've got to admit that I have done this many times over the last week. Boring but with no real angst involved.

In six months time it might be very different, with several complicated applications installed, all configured and linked to the data files. It's about then that many people might start to think about backup, but it's a bad time to learn about the possible complications of restore.

At least after a week's work I'm pretty certain that, if I foul up the OS in future, I can just re-format the system partition and put a good earlier version back again. It's just one way of doing things. One colleague seems to do all his development work in multiple installations then plays with grub.

GRUB, now that really is frightening!

weetfield

[uzi09]

Would this be the ideal procedure to use if I wanted to switch from Hardy over to Gutsy (or another version of Ubuntu or Linux)?

[weetfield]

Sorry, when I wrote "earlier version" I meant an earlier version of my own installation. If I foul something up on my system now, I don't know enough to sort things out then clean up again. My way round it is to back the system up before any serious experiment, then restore it if I goof big time.

If you want to try some other distro on your machine as well as Hardy, dual installation / boot menu is probably the way to do it. That way you have both on your machine and choose which one to run during boot up. That isn't a backup-restore procedure though, it's a fresh install of the second system. I'm a beginner here but I'm sure that there will be another thread in this forum dealing with this technique.

If you are going to try this though, it's another good reason to keep your personal stuff (photos, music, mail, videos etc.) on a separate partition, to make it easier to get at from either distro. Also you won't have a synchronisation problem (ending up with say, your music downloads in two different places).

Some applications will even let you put the configuration files on your data disk. Thunderbird mail client for instance will let you put your whole profile on it (mail files, address book, accounts, server settings etc.). Then, if you just do a basic install on your new distro you can point it to your central profile using its profilemanager service.

Finally, there is less chance of losing separate data if the dual boot goes wrong and you have difficulty starting your original Hardy installation.

[Elderlygent]

Ok, there's a lot of questions in there, I'll try to answer them one by one.

1: The howto on the first page is what I use and what works for me. I've used it to backup several systems and it worked every time for me
2: If you want to have a backup of /home, just cd to / and run the correct command. In your case probably something like this:

# cd /
# sudo tar cvpfz homebak.tgz /home


If you want to backup /, just cd to /home and run the regular backup command, but this time exclude /home since it is on a seperate partition. This would make the commands look something like this:

# cd /home
# tar cvpfz backup.tgz / --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys --exclude=/home


If you want to restore just the backup of the homedir, you would need something like this:

# cd /
# tar xvpfz backup.tgz -C /home


Be careful though: If you have a backup of / in your home-dir, it'll be overwritten when you restore your home-dir. (unless, of course, it is included in the archive but that wouldn't be very space-efficient ;))

The best would be to store the backup of /home on the / partition, and store the backup of the / partition on another machine or burn it to DVD or whatever.

As far as I know, it is possible to restore / on a running system. I've never tried it before but since it is also possible to erase everything in / on a running system, I figure why not! ;) I'm gonna need some confermation here though.
Of course you could always just use a live-cd. Usually when you have to restore /, you don't have an option anyway! ;)

Hope this helps!

Yes it did, many thanks Heliode for this and previous.

[sofasurfer]

I did the backup using the instructions in post #1. At the end of the process I did get the error 'tar: Error exit delayed from previous errors' which is said to be nothing to worry about.

Now I did the restore. This was my output:

daryl@ubuntu:~$ sudo su
root@ubuntu:/home/daryl# cd /
root@ubuntu:/# tar xvpfz backup.tgz -C /

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error exit delayed from previous errors

What happened?


EDIT::

So I decided to try restoring with a livecd since, like someone said, "Theres a good chance you won't be able to boot anyway, if you are needing to restore your system". I was unsuccessful here also because you can not access your file system with a live cd, just the file system on the cd.

Or am I missing something?

[scorp123]

root@ubuntu:/# tar xvpfz backup.tgz -C /

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error exit delayed from previous errors[/code] Please execute the "file" command on that file! e.g.
file /path/to/backup.tgz ... I'd like to see what this file is. Chances are you used different parameters when you created the file and the file might indeed not be gzip packed.

Example from here:
> file vodafone.tar.gz
vodafone.tar.gz: gzip compressed data, from Unix, last modified: Mon Jun 23 13:35:26 2008

Can you do that please? It would be interesting to know what kind of file we're dealing with.

I was unsuccessful here also because you can not access your file system with a live cd, just the file system on the cd. You are misinformed. Unless your harddisks require some really exotic driver (e.g. fibre channel storage driver??) it should be possible to access them from a live CD too. People have been doing this for ages now.

Once you're in a live CD environment, please issue this command:
sudo fdisk -l ... this should list all harddisk partitions that were detected, live CD or not. Example from here:
# fdisk -l

Disk /dev/sda: 146.6 GB, 146684248064 bytes
255 heads, 63 sectors/track, 17833 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xdaa138b9

Device Boot Start End Blocks Id System
/dev/sda1 * 1 16 128488+ 83 Linux
/dev/sda2 17 17833 143115052+ 5 Extended
/dev/sda5 17 259 1951866 83 Linux
/dev/sda6 260 988 5855661 83 Linux
/dev/sda7 989 1231 1951866 83 Linux
/dev/sda8 1232 9012 62500851 83 Linux
/dev/sda9 9013 16793 62500851 83 Linux
/dev/sda10 16794 17036 1951866 83 Linux
/dev/sda11 17037 17833 6401871 83 Linux

Disk /dev/sdb: 64.4 GB, 64424509440 bytes
64 heads, 32 sectors/track, 61440 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 61440 62914544 83 Linux

Disk /dev/sdc: 32.2 GB, 32212254720 bytes
64 heads, 32 sectors/track, 30720 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdc1 1 30720 31457264 83 Linux

Disk /dev/sdd: 32.2 GB, 32212254720 bytes
64 heads, 32 sectors/track, 30720 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdd1 1 30720 31457264 83 Linux

Disk /dev/sde: 53.6 GB, 53687091200 bytes
64 heads, 32 sectors/track, 51200 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sde1 1 51200 52428784 83 Linux

Also take a look at the output of this command:
dmesg ... The output of that one will be very long and it will mention all kind of things. It should also mention what harddisks have been detected during system boot. Example from here (edited: many lines removed):
[ 151.192646] scsi3 : ata_piix
[ 151.192782] scsi4 : ata_piix
[ 151.193725] ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xfff0 irq 14
[ 151.193811] ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xfff8 irq 15
[ 151.531027] ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
[ 151.531298] ACPI: PCI Interrupt 0000:00:1f.2[B] -> GSI 21 (level, low) -> IRQ 21
[ 151.531473] PCI: Setting latency timer of device 0000:00:1f.2 to 64
[ 151.531531] scsi5 : ata_piix
[ 151.531651] scsi6 : ata_piix
[ 151.531759] ata3: SATA max UDMA/133 cmd 0xec00 ctl 0xe800 bmdma 0xdc00 irq 21
[ 151.531845] ata4: SATA max UDMA/133 cmd 0xe400 ctl 0xe000 bmdma 0xdc08 irq 21
[ 152.095564] scsi 2:0:0:0: Direct-Access SUN LCSM100_F 0670 PQ: 1 ANSI: 5
[ 152.102526] scsi 2:0:0:10: Direct-Access SUN LCSM100_F 0670 PQ: 0 ANSI: 5
[ 152.103152] scsi 2:0:0:11: Direct-Access SUN LCSM100_F 0670 PQ: 0 ANSI: 5
[ 152.103756] scsi 2:0:0:12: Direct-Access SUN LCSM100_F 0670 PQ: 0 ANSI: 5
[ 152.104324] scsi 2:0:0:13: Direct-Access SUN LCSM100_F 0670 PQ: 0 ANSI: 5
[ 154.654363] XFS mounting filesystem sda9
[ 154.791516] Ending clean XFS mount for filesystem: sda9
[ 154.848755] XFS mounting filesystem sda7
[ 154.882792] Ending clean XFS mount for filesystem: sda7
[ 154.939226] XFS mounting filesystem sda8
[ 154.998981] Ending clean XFS mount for filesystem: sda8
[ 155.067625] XFS mounting filesystem sda11
[ 155.141243] Ending clean XFS mount for filesystem: sda11
[ 155.196037] XFS mounting filesystem sda6
[ 155.271758] Ending clean XFS mount for filesystem: sda6
[ 155.330263] XFS mounting filesystem sda10
[ 155.395941] Ending clean XFS mount for filesystem: sda10
[ 156.748189] scsi 2:0:0:0: Attached scsi generic sg4 type 0
[ 156.749640] sd 2:0:0:10: [sdb] 125829120 512-byte hardware sectors (64425 MB)
[ 156.749792] sd 2:0:0:10: [sdb] Write Protect is off
[ 156.749794] sd 2:0:0:10: [sdb] Mode Sense: 77 00 10 08
[ 156.750070] sd 2:0:0:10: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 156.750905] sd 2:0:0:10: [sdb] 125829120 512-byte hardware sectors (64425 MB)
[ 156.751053] sd 2:0:0:10: [sdb] Write Protect is off
[ 156.751056] sd 2:0:0:10: [sdb] Mode Sense: 77 00 10 08
[ 156.751320] sd 2:0:0:10: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 156.751325] sdb: sdb1
[ 156.755580] sd 2:0:0:10: [sdb] Attached SCSI disk
[ 156.755626] sd 2:0:0:10: Attached scsi generic sg5 type 0
[ 156.756994] sd 2:0:0:11: [sdc] 62914560 512-byte hardware sectors (32212 MB)
[ 156.757140] sd 2:0:0:11: [sdc] Write Protect is off
[ 156.757142] sd 2:0:0:11: [sdc] Mode Sense: 77 00 10 08
[ 156.757403] sd 2:0:0:11: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 156.758237] sd 2:0:0:11: [sdc] 62914560 512-byte hardware sectors (32212 MB)
[ 156.758388] sd 2:0:0:11: [sdc] Write Protect is off
[ 156.758390] sd 2:0:0:11: [sdc] Mode Sense: 77 00 10 08
[ 156.758651] sd 2:0:0:11: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 156.758654] sdc: sdc1
[ 156.765636] sd 2:0:0:11: [sdc] Attached SCSI disk
[ 156.765680] sd 2:0:0:11: Attached scsi generic sg6 type 0
[ 156.767734] sd 2:0:0:12: [sdd] 62914560 512-byte hardware sectors (32212 MB)
[ 156.767890] sd 2:0:0:12: [sdd] Write Protect is off
[ 156.767892] sd 2:0:0:12: [sdd] Mode Sense: 77 00 10 08
[ 156.768160] sd 2:0:0:12: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 156.769150] sd 2:0:0:12: [sdd] 62914560 512-byte hardware sectors (32212 MB)
[ 156.769292] sd 2:0:0:12: [sdd] Write Protect is off
[ 156.769294] sd 2:0:0:12: [sdd] Mode Sense: 77 00 10 08
[ 156.769552] sd 2:0:0:12: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 156.769555] sdd: sdd1
[ 156.772804] sd 2:0:0:12: [sdd] Attached SCSI disk
[ 156.772838] sd 2:0:0:12: Attached scsi generic sg7 type 0
[ 156.774114] sd 2:0:0:13: [sde] 104857600 512-byte hardware sectors (53687 MB)
[ 156.774263] sd 2:0:0:13: [sde] Write Protect is off
[ 156.774264] sd 2:0:0:13: [sde] Mode Sense: 77 00 10 08
[ 156.774526] sd 2:0:0:13: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 156.775233] sd 2:0:0:13: [sde] 104857600 512-byte hardware sectors (53687 MB)
[ 156.775372] sd 2:0:0:13: [sde] Write Protect is off
[ 156.775374] sd 2:0:0:13: [sde] Mode Sense: 77 00 10 08
[ 156.775636] sd 2:0:0:13: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 156.775639] sde: sde1
[ 156.782151] sd 2:0:0:13: [sde] Attached SCSI disk
[ 156.782183] sd 2:0:0:13: Attached scsi generic sg8 type 0

Please check what "dmesg" has to say about your system. The harddisks should definitely be listed there, live CD or not.

[sofasurfer]

Well, I ran to backup and restore commands one more time and it appears that it worked. But I am NOT able to access my file system with a live cd. I did a 'fdisk -l' while running the livecd and my drive was listed as /dev/sdb. It is actually /sda, and sdb is really my second drive. So I disconnected my second drive and then 'fdisk -l' showed my drive as /sda.

It appeared that 'fdisk -l' showed my file system because the partition sizes looked the same. But I will go verify that in a minute by comparing the exact sizes.

When I used the browser to access the file system under livecd I was only able to access the file system on the livecd. I know it was not my file system because there was a /home/ubuntu directory which my file system doed not have. And there was none of my info in the files.

dmesg is a little hard for me to decipher but I will go and try again.

[linux6994]

Easiest way to backup and restore is to install sbackup. You can schedule, decide what to include and exclude, and even backup to an external device. It works great for doing full and or incremental backups.

[sofasurfer]

I ran dmesg under livecd and my hard drive IS listed as being 250 gig. But I did not see much more info than that. I then went to terminal and tried with NO LUCK to cd into my /home/daryl directory.

Am I supposed to used a certain kind of livecd? I am using Ubuntu 8.04 livecd.

EDIT::

linux6694.
sbackup is a good thing and I have tried it. But I and striving to learn how to do a disk image backup.

[scorp123]

But I am NOT able to access my file system with a live cd. You could maybe explain a little more about your system then, e.g. which drive is what, and then maybe post the output of the live CD's "fdisk -l" here? (assuming that you get an Internet connection with the live CD).

I did a 'fdisk -l' while running the livecd and my drive was listed as /dev/sdb. This doesn't have to mean much. Sometimes drives and their names get moved around, e.g. what you see as /dev/sdb might then be seen as /dev/sda in a Live CD session and vice versa.

When I used the browser to access the file system under livecd I was only able to access the file system on the livecd. I assume you didn't get any icons on the Live CD desktop showing your harddisk drives ....? Did you try to mount them manually?

dmesg is a little hard for me to decipher but I will go and try again. Just post it here ;)

[sofasurfer]

Can't do much now, as I'm at work.
"Mount them manually"!! AHA!! I bet thats the answer. I will try that before I complain any more. Thanks.

[The Minder]

Ok, the thought of losing data scares the bejeezus out of me. Not so with the o/s as that can always be recreated. So for my ex's business I set up a Ubuntu server that had /home and /group directories for 4-5 users who logged in from Windoz boxes. The server's storage was 2 by 250Gb hot swappable drives configured for Raid 1. Cool, me thinks, now if a drive crashes all is not lost and the chance of 2 drives going out at the same time is pretty slim.

Unfortunately, the business was in tornado/thunderstorm alley and a month ago the Almighty decided to send 500 megajoules down the power line. The jolt fried the power conditioner, the server mobo and one of the HDs. The other drive was damaged, but a tech managed to recover enough data to keep the business running, just. So my idea of having redundancy/backup didn't work 100%. In hindsight, I think a better option would have been not to RAID the drives but instead to have a master drive with everything on it and use the second drive as a backup that would only be inserted when the backup was to run and be kept in safe storage at all other times.

If I read this howto properly, this may answer my prayers, So let me run my plan past you gurus and feel free to shoot me down.

1. Rebuild the system, with all user info, data etc on a single drive.
2. On a weekly (or other convenient) basis, insert the second drive.
3. Run the code mentioned in the howto and backup EVERYTHING to the second drive,
4. Unmount the second drive and store it somewhere safe... like in grandma's knickers (nothing ever gets there).
5. Repeat steps 2-4 and wait for the next catastrophe.

Assuming the system does get slammed again:
6. Replace any damaged components and boot the server using a live-cd.
7. Retrieve the backup drive from grandma and insert into the second drive bay... mount if necessary.
8. Restore from the second drive to the new first drive as per this howto.
9. Shut the server down, remove the second (backup) drive and the live-cd.
10. Reboot the system.

Obvious questions is, will this work or is some other intervention required?

Regards

[scorp123]

1. Rebuild the system, with all user info, data etc on a single drive. Why not use mirroring? And while you're at it: Have an electrician take a look at your server room and let him suggest proper measures to make sure that such disasters don't happen again. We have a ton of UPS's in our server room which make sure that power remains on a constant level. And if power fails the UPS's will tell the servers to properly shut down (it's just a matter of configuring this).

2. On a weekly (or other convenient) basis, insert the second drive. Spinning a harddrive up and down shortens its lifetime. Attaching harddrives for the purpose of making backups is OK for the home user (I do that myself via USB harddisks), but you're doing this for a business, right? Why not invest in magnetic tapes? DDS/DAT or DLT/LTO tapes can survive a lot more and for decades longer than a harddisk. Current LTO tapes can take as much as 800 GB of data. And the best thing is: You don't even need special backup software. "tar" will happily talk to your tapes too (so instead of specifying a *.tar.gz file that you are writing your backup to you will specify the tape drive as target instead, e.g. /dev/nst0 ). So you'd regularly have your stuff backed up onto those 800 GB LTO tapes, and once every week you send one tape off-site to a secure location, e.g. a storage facility? Or a bank safe? Or to one of the Swiss "data bunker" companies here who have their own private nuke-proof bunkers deep inside the Swiss Alps? It's all just a question of how important your data really is to you and how much money you are willing to spend for keeping it safe and outside the reach of natural disasters.

3. Run the code mentioned in the howto and backup EVERYTHING to the second drive, Whatever you do: TEST IT + DOCUMENT IT. A backup is worth absolutely NOTHING if you can't get your data back when you need to. So whatever way you take in the end: Have this stuff tested!

1. Recreate your servers
2. Make backups
3. Then simulate a disaster: Swap the perfectly working harddrives with empty ones (simulating data loss) or if you have the guts: Destroy your own data ... yes, on purpose

4. Find out if your backup was worth anything by restoring.
5. Have someone measure the time and observe what is going on, have people take notes.

6. Once you got the restore working it's time to sit together and to discuss the observations: What worked OK? What did not work OK? Which improvements are required? Does everyone who needs to know really know what to do if and when disaster strikes?

4. Unmount the second drive and store it somewhere safe... like in grandma's knickers (nothing ever gets there). As I said ... If this is for a business I'd suggest you buy a tape based backup solution. A simple DDS/DAT tape drive (fits into a standard harddrive slot) should do. DDS tapes can hold between 4 GB (DDS2) up to 160 GB (DDS160; introduced 2007) per tape.
http://en.wikipedia.org/wiki/Digital_Data_Storage

LTO tape solutions are way more expensive, but if you have lots of files to backup it pays off; present-day LTO-4 tapes can hold up to 800 GB
http://en.wikipedia.org/wiki/Linear_Tape-Open

Assuming the system does get slammed again:
6. Replace any damaged components and boot the server using a live-cd.
7. Retrieve the backup drive from grandma and insert into the second drive bay... mount if necessary.
8. Restore from the second drive to the new first drive as per this howto.
9. Shut the server down, remove the second (backup) drive and the live-cd.
10. Reboot the system. Hmmm ... Why not configure a second server as standby machine? This would keep downtime shorter.

[The Minder]

Scorp123,

Thanks for the response, you offered much good advice.

The system in question did had UPS, but the lightning strike took it out, and almost every electrical device in the town. This was lightning of biblical proportions and not uncommon in this area, so the chances of a re-occurrence are not remote. It therefore comes down to an issue of cost as decent lightning protection systems can run into the $100k's; worth more than the business. Hence the need to be able to toss out damaged equipment and get the client back up and running with ease and minimal data loss. Some sort of system/data backup (as explained in the howto) would seem to fit the bill.

I agree that tapes are a good option and will consider it closely for the next build, however the cost of hard drives these days makes them almost a disposable item. I also agree that off site storage of the 'backup' is the only way to go. I have seen too many instances where a catastrophe has taken out a system and the backup because they were stored within inches/feet of each other... fire has no friends.

Again, thanks for the response.

[sofasurfer]

I managed to mount my file syatem using livecd. It is mounted at /mnt/ubuntu.

When I open the file browser I go to /mnt/ubuntu and there is my file system such as /mnt/ubuntu/home/daryl/etc....

At the start of this thread in post #1 it says to restore my system by issueing "tar xvpfz backup.tgz -C /", AFTER doing a "cd" into the "/" of my system.
That works fine from inside my OPERATING system. But not from the livecd.

From inside a livecd what do I put in place of the "/"? Do I "tar xvpfz backup.tgz -C /" or "tar xvpfz backup.tgz -C /mnt/ubuntu" or what?

I can do a "cd /mnt/ubuntu/home" for instance, but how do I make sure that I am inside the "/" of my file system to do a restore using "tar xvpfz backup.tgz -C /"?

I hope this makes sense.

[sofasurfer]

Another way of stating this is, Assuming that my systen is trashed, how do I restore my system, as stated in post #1, from a livecd?

Thank you.

[dlmoak]

I recently installed linbaku, which makes a compressed file of your entire system. I had it create the file on an external hard drive. To test the program, I re-installed 8.04. I did no updates. I installed linbaku and ran the restore from the external disk. My system was restored with no problems. One note - if you are installing on 64-bit Ubuntu you have to isntall the 32-bit utils first.

[scorp123]

Another way of stating this is, Assuming that my systen is trashed, how do I restore my system, as stated in post #1, from a livecd? Maybe it's inpolite to "sound my own trumpet" here ... but please read this here:
http://linuxmint.com/forum/viewtopic.php?t=3969

Relevant for you is "Step #4: Packing your partitions into *.tar.gz archives" (a bit further down that page) and "How to restore your partitions via the archives" right after that.

It explains the necessary shell commands step by step.

[sofasurfer]

dlmoak and scorp123.
Thanks. Good stuff. I was always intimidated to try the tar command but it turns out that theres nothing to it. Still working out some details but getting closer.

Will try linbaku this weekend. Looks good.

[KhaaL]

Maybe it's inpolite to "sound my own trumpet" here ... but please read this here:
http://linuxmint.com/forum/viewtopic.php?t=3969

Relevant for you is "Step #4: Packing your partitions into *.tar.gz archives" (a bit further down that page) and "How to restore your partitions via the archives" right after that.

It explains the necessary shell commands step by step.

Thanks for that. I did a backup/restore today with this method but my system was rendered useless still because of the issue with UUIDs in fstab and grub. Your guide helped two systems back to life so thank you again :-)

[SanskritFritz]

Will try linbaku this weekend. Looks good.Report please your findigs.

[sofasurfer]

I started using the tar command to backup to /media/sdb1. When I would browse to /media/sdb1 I would see a tar.tgz file. I then managed to trash my system because I got my drives mixed up. So I then the package, pysdm, a storage device manager.

I don't know if this has anything to do with this or not, but now when I save a tar to /media/sdb1 and then browse to /media/sdb1 I see a complete copy of my file system NOT A COMPRESSED FILE. Why? Shouldn't the file system by compressed?

Here is the command I used after 'sudo su'...

tar cvpzf fresh-install-backup.tgz.07.01.08 --exclude=/proc/* --exclude=/lost+found/* --exclude=/mnt/* --exclude=/sys/* --exclude=/media/sdb1/* /

[jmz2]

Thanks to all for making this trhead that has helped me a lot.

However I need to now if I must worry about the 'socket ignored' error that I am receiving after the tar execution.

The translation is:
el `socket' no se tendrá en cuenta ----> socket ignored

This is the error message:

/bin/tar: /dev/log: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7586/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7586/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7586/testAutomation-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7586/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7586/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7586/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30701/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30701/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30701/remoteDevice-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30701/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30701/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30701/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19179/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19179/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19179/remoteDevice-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19179/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19179/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19179/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/13286/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/13286/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/13286/remoteDevice-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/13286/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/13286/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/13286/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/28293/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/28293/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/28293/remoteDevice-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/28293/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/28293/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/28293/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19301/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19301/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19301/remoteDevice-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19301/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19301/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/19301/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31761/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31761/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31761/testAutomation-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31761/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31761/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31761/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/3263/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/3263/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/3263/remoteDevice-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/3263/testAutomation-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/3263/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/3263/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/3263/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31751/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31751/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31751/testAutomation-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31751/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31751/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/31751/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7594/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7594/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7594/testAutomation-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7594/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7594/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/7594/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/8926/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/8926/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/8926/remoteDevice-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/8926/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/8926/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/8926/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30059/vmx-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30059/servercontrol-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30059/remoteDevice-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30059/mks-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30059/ui-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/user1/30059/control-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/root/15999/server-vmxvmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/root/15999/nfc-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/root/15999/vmx-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/root/15999/fsserver-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/root/15999/server-vcvmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/root/15999/server-vmdb-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmware/root/15999/server-fd: el `socket' no se tendrá en cuenta
/bin/tar: /var/run/vmnat.19970: el `socket' no se tendrá en cuenta

Thanks for your help.

[scorp123]

I don't know if this has anything to do with this or not, but now when I save a tar to /media/sdb1 and then browse to /media/sdb1 I see a complete copy of my file system NOT A COMPRESSED FILE. Why? Shouldn't the file system by compressed? Sorry, I don't understand what you mean. You do get a tar.gz file, yes? So there you go, that's your compressed archive.

[scorp123]

However I need to now if I must worry about the 'socket ignored' error that I am receiving after the tar execution.
http://en.wikipedia.org/wiki/Unix_domain_socket

In other words: "sockets" are temporary files used by programs that need to talk to each other. Copying them is pretty much pointless (sockets are virtual files and there is no content that could be saved) and that's why "tar" is ignoring them.

Should you crash your system and restore everything from backups you only have to worry about getting your programs back running. Guessing from the many references to "vmx" I suppose this is VMware? So you'd restore your system, get VMware and the guest OS's running again ... and those "socket" files will be recreated dynamically as needed once the relevant processes are running and start talking to each other again.

[imolafem]

Do you mind sharing how this would be scheduled in cron for nightly backups?

[scorp123]

Do you mind sharing how this would be scheduled in cron for nightly backups? https://help.ubuntu.com/community/CronHowto

[imolafem]

Thanks, that is a good start. But what I'd like to do is do a full backup every night of the week to a different file per night of the week. Then Monday's gets overwritten the following Monday. So how do I write the crontab stuff with what variables to accomplish that?

[imolafem]

Thanks, that is a good start. But what I'd like to do is do a full backup every night of the week to a different file per night of the week. Then Monday's gets overwritten the following Monday. So how do I write the crontab stuff with what variables to accomplish that?

Just thought I'd show you guys what I put in crontab:

amanda@kubuntu804:~$ sudo crontab -l
# m h dom mon dow command
01 0 * * 0 tar cvpzf /archive/sun.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 1 tar cvpzf /archive/mon.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 2 tar cvpzf /archive/tue.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 3 tar cvpzf /archive/wed.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 4 tar cvpzf /archive/thu.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 5 tar cvpzf /archive/fri.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 6 tar cvpzf /archive/sat.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
amanda@kubuntu804:~$

Hope it works =)

[scorp123]

Just thought I'd show you guys what I put in crontab:

amanda@kubuntu804:~$ sudo crontab -l
# m h dom mon dow command
01 0 * * 0 tar cvpzf /archive/sun.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 1 tar cvpzf /archive/mon.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 2 tar cvpzf /archive/tue.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 3 tar cvpzf /archive/wed.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 4 tar cvpzf /archive/thu.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 5 tar cvpzf /archive/fri.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
01 0 * * 6 tar cvpzf /archive/sat.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /
amanda@kubuntu804:~$

Hope it works =) I guess this could be simplified. Take a look at the manual page of the "date" command: man date ... Here we can see that e.g. a command such as date +%a ... would spit out the day of the week. Example: > date +%a
Thu

So instead of repeating the same command seven times, you could replace it with this one command: tar cvpzf /archive/`date +%a`.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys / ... just make sure it gets executed every day, e.g. 01 0 * * * tar cvpzf /archive/`date +%a`.backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/archive --exclude=/mnt --exclude=/sys /

[Elderlygent]

Hi there,

Thanks to Heliode I backed up my Home directory and I'm very glad I did. It's on a cd.

Now that I've zapped my old home changing from Gutsy up to Hardy, how do I go about getting the stuff off the disk and into my new /home?

I've tried messing about with the Terminal, but I'm really not very good at it, sorry.

LATER: I've worked out how to do it.

[mastergunner]

subscribe

[rugbert]

I remember seeing somewhere in the thread how to make a log based off this backup script. can someone link me? my backups started failing for no reason....

[Capa Pinbacker]

This thread is awesome! I use this information to routinely tar my Ubuntu partition, and copy it to other partitions to allow multi-boot, etc.

This is what I do:

To copy Ubuntu installation:
1. Boot from live CD (or boot into another Linux partition if avail)
2. Let's say that the installation we want to copy is mounted at /media/disk-1, and we want to store the archive in the volume /media/DATA... first cd /media/disk-1, then type:
sudo tar cvpzf /media/DATA/ubuntu_backup.tgz *

To unzip Ubuntu archive:
1. Boot in Live CD mode or else in a second Linux partition
2. Let's say that the target volume is mounted at /media/disk-1, and the archive is stored in the volume /media/DATA... first cd /media/disk-1, then type type:
sudo tar xvpfz /media/DATA/ubuntu_backup.tgz
3. ***BEFORE*** rebooting the computer, if the target partition was reformatted before the restore, then it is necessary to change the UUID of the formatted partition back to that of the original installation. Do the following:
a) find out the UUID of the target drive by opening the file "/etc/fstab".
e.g., # /dev/hda1
UUID=558a338f-ebc1-4050-82f2-086da21170af / ext3 ...
b) change the UUID of the target partition back to what the fstab file expects:
sudo tune2fs /dev/hda1 -U 558a338f-ebc1-4050-82f2-086da21170af
c) now you can reboot the computer
4. If the target partition is different than the original backup partition, and if the target partition holds the active grub application for the disk, then (before reboot) edit menu.lst and fstab and modify the UUID's for each partition as required in each file (the command "blkid" prints the UUID's of each partition on the computer):
/boot/grub/menu.lst
/etc/fstab
Then reboot the computer.

[Dieseler]

I bumped this old thread earlier. I'm a 3 week Linux newbie and I'm just wondering if anyone is using this script that lunde wrote. If I had a test machine I would try it but I don't and its a pretty old thread so I'm not sure if it will work with Hardy 8.04.

http://ubuntuforums.org/showthread.php?t=47999

[MattiViljanen]

Thanks for this little piece of information!

I just did some restoring on live system, and I think it's not necessary/wise to copy PID files from /var/run when performing restore to live system... Nothing disasterous, but nasty shutdown error list, though :popcorn:

[kihjin]

This is the command I use for creating a recovery partition

tar -c --exclude='/media/*' --exclude='/dev/*' --exclude='/home/*' --exclude='/proc/*' --exclude='/sys/*' --exclude='/tmp/*' --exclude='/var/run/*' --exclude='/var/lock/*' --exclude='/lib/modules/*/volatile/*' / | tar -xv --atime-preserve -C /media/sda10

/media/sda10 is another partition, which has a valid mounted filesystem. Best part is you can modify your lilo.conf or menu.lst and boot into it! I make these after virgin installs to preserve the base system in case I hose the configuration at some point...

If you wanted to just create a backup file then this will work:

tar -vcf backup.tar --exclude='/backup.tar' --exclude='/media/*' --exclude='/dev/*' --exclude='/home/*' --exclude='/proc/*' --exclude='/sys/*' --exclude='/tmp/*' --exclude='/var/run/*' --exclude='/var/lock/*' --exclude='/lib/modules/*/volatile/*' /

It's important that the exclude's be in single quotes, where there is a *. without the quotes, the * isn't treated as a literal *, instead it will be expanded to a list of filenames.

[janesc]

I've been tearing my hair out trying to use crontab for my backups. Removing the verbose switch fixed it!

Thanks

[rcrcomputing]

how would one go about making this into a cron job to happen at certan times? could it be made into a script some how?
thanks.

I've not finished reading this thread, but we may be forgetting the old trusty filesystem backup option in webmin.
Years ago, I needed to move the current servers to servers with scsi drives. After much reading and thought, I used webmin filesystem backup to get only these directorys, as home is/can be needed or not when you may be restoring to another machine.. It'd be good to clean out the apt cache before your backups.. (Of course, webmin gives you that option to..)

/etc
/var
/usr
/root
/sbin

Then installed debian on the new servers, installed webmin and restored. This worked like a charm and the servers still run today..
The beauty is, webmin does the ssh connection and scheduling for you. All with a gui you can get to from anywhere.. Nice tool!

I'm about to try it with Hardy. We'll see..

[sofasurfer]

Maybe I nissed this but how do I restore just 1 particular file or directory? Say I only want to restore /home/blah/blahblah. How do I modify my command line?

[sofasurfer]

Maybe I nissed this but how do I restore just 1 particular file or directory? Say I only want to restore /home/blah/blahblah. How do I modify my command line?

[scorp123]

Maybe I nissed this but how do I restore just 1 particular file or directory? Say I only want to restore /home/blah/blahblah. How do I modify my command line? http://www.google.com/search?hl=en&q=tar+extract+specific+file&btnG=Google+Search&meta=

[t.letum]

'cvpfz' are the options we give to tar, like 'create archive' (obviously),
'preserve permissions'(to keep the same permissions on everything the same), and 'gzip' to keep the size down.


The 'p' option is needless here. Yes, it preserves permissions, but only when you extract the archive. Tar saves permissions by default - 'p' is necessary only during the extraction process.

[the real omni]

I've followed these steps to the tee and my system now stalls during boot-up. It gets to the screen that says UBUNTU and has the little status bar scooting back and forth but it doesn't get past there... the little orange block just keeps scooting back and forth in the status indicator.

I'm pretty sure the reason for this is that my old partition was (hd0,2) and my new partition is (hd0,4).

Is there an easy way to sift through the tarball and change anything that was referencing the old partition to reference the new partition?

[dmizer]

I've followed these steps to the tee and my system now stalls during boot-up. It gets to the screen that says UBUNTU and has the little status bar scooting back and forth but it doesn't get past there... the little orange block just keeps scooting back and forth in the status indicator.

I'm pretty sure the reason for this is that my old partition was (hd0,2) and my new partition is (hd0,4).

Is there an easy way to sift through the tarball and change anything that was referencing the old partition to reference the new partition?

Boot your system with a live cd, and edit /etc/fstab with the correct partitions.

[the real omni]

Boot your system with a live cd, and edit /etc/fstab with the correct partitions.

Ok here are the contents of my /etc/fstab

# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
# /dev/sda3
UUID=aaf74eb6-791f-41ee-9821-25a41eebdfce / ext3 relatime,errors=remount-ro 0 1
# /dev/sda4
UUID=0b106a2d-66d8-44f5-bf86-0831194ef8b5 none swap sw 0 0
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto,exec,utf8 0 0

I'm not exactly sure what to change there...

The only ext3 partition on the system is my Ubuntu partition and that's /dev/sda5

[dmizer]

Please post the output of:
df -Th
swapon -s
This will list your partitions in human readable format. We need to know which partition is swap as well.

[the real omni]

Please post the output of:
df -Th
swapon -s
This will list your partitions in human readable format. We need to know which partition is swap as well.

Ok here's the output of 'df -Th':
ubuntu@ubuntu:~$ df -Th
Filesystem Type Size Used Avail Use% Mounted on
tmpfs tmpfs 976M 13M 963M 2% /lib/modules/2.6.24-19-generic/volatile
tmpfs tmpfs 976M 13M 963M 2% /lib/modules/2.6.24-19-generic/volatile
varrun tmpfs 976M 104K 976M 1% /var/run
varlock tmpfs 976M 0 976M 0% /var/lock
udev tmpfs 976M 60K 976M 1% /dev
devshm tmpfs 976M 12K 976M 1% /dev/shm
tmpfs tmpfs 976M 16K 976M 1% /tmp
gvfs-fuse-daemon
fuse.gvfs-fuse-daemon 976M 79M 898M 8% /home/ubuntu/.gvfs


And here's the output of 'swapon -s':
ubuntu@ubuntu:~$ swapon -s
Filename Type Size Used Priority
/dev/sda6 partition 2931820 0 -1

[the real omni]

Boot your system with a live cd, and edit /etc/fstab with the correct partitions.

Ok I found in another forum (http://encodable.com/tech/blog/2006/10/30/Ubuntu_Linux_Hard_Drive_Upgrade) that in a terminal, the command 'sudo vol_id /dev/sda1' will reveal the UUID of the given drive. (Replace /dev/sda1 with the device you want to know the UUID on. To find out what /dev/(drive) handle/title to use, run gparted)

I've done that to find out the UUID of the drive with Ubuntu on it and the UUID of the swap partition.

I've replaced the UUID entries in /etc/fstab and it's now getting a bit farther along the boot stage but now after about 60-120 seconds on the screen with the status indicator scooting back and forth, it goes to a command-line screen that says:


BusyBox v1.1.3 (Debian 1:1.1.3-5ubuntu12) Built-in shell (ash)

(initramfs)


If I try booting to the recovery mode option, a ton of commands whip across the screen super fast (I'm guessing this is just verbose mode for debugging purposes) and it hangs on:

( 24.34315) sd 2:0:0:0: Attached scsi generic sg1 type 0


It hangs there indefinitely and doesn't pass through to the BusyBox shell.

Please help! I'm stuck in Vista until I get my Ubu back!

VISTA for chrisssakes... sheesh.

("Help help I can feel the walls closing in around me! *gasp* Can't breathe! *Cough* No 3D Desktop Cube!!")

[mazz72]

Hey guys,

I'm sorry if this has been answered already, but this thread is really long so I may have skipped a few pages. ;)

I just want to clarify something. Is it necessary to be in root just to do a /home backup, or is that only for a full system backup? Or is it not really necessary at all?

Thanks all. :)

[dmizer]

And here's the output of 'swapon -s':
ubuntu@ubuntu:~$ swapon -s
Filename Type Size Used Priority
/dev/sda6 partition 2931820 0 -1


It's not necessary to use the UUID, it's easier to use the physical location of the partition. In this case, the swap partition, can be changed to to /dev/sda6 like so:
/dev/sda6 none swap sw 0 0


Unfortunately, the rest of your output didn't tell me what I wanted to know. You'll need to take a look at the partitions (again, using the live cd) with the partition editor under system > administration

Look for your ext3 / partition listed as sda# (where # is some number)

[the real omni]

Thanks but at this point I've just given up.

I realized the amount of time I've spent troubleshooting this problem has exceeded the amount of time it would have taken to just pave from scratch and rebuild/reconfigure my system brick by brick.

Thanks for the help but this guide is obviously only applicable to people who aren't going to be making any changes to the partition table. (It would have been nice to have that noted right at the beginning so I didn't nuke my partition...)

[scorp123]

but this guide is obviously only applicable to people who aren't going to be making any changes to the partition table. Not true. If you do it right then changes to the partition tables is no problem.

I know ... I shouldn't be doing this here ... But again: I too once wrote a "backup & restore" guide. Basically it's the same commands as here; except that my guide is structured differently. So maybe it might work better for you and easier to understand? Give it a try:

"How to backup your stuff UNIX-style"
http://www.linuxmint.com/forum/viewtopic.php?f=42&t=3969&start=0&st=0&sk=t&sd=a

[M4rotku]

Hello,

I have used this guide in the past to back up my system and it has worked flawlessly. However, this time I decided to use the second option for extra compression. This is the command that I entered in my terminal:

joey@joey-laptop:/$ tar cvpjf backup.tar.bz2 --exclude=/proc --exclude=/lost+found --exclude=/backup.tar.bz2 --exclude=/mnt --exclude=/home/joey/Music --exclude=/home/joey/Videos --exclude=/home/joey/Pictures --exclude=/home/joey/ISO's --exclude=/home/joey/Photos --exclude=/sys /
>


As you can see, after I entered the command, it outputted ">".

To get out of this little carrot-mode as I like to think of it, I entered a hyphen and pressed enter. This is the response that I received:

tar: Cowardly refusing to create an empty archive
Try `tar --help' or `tar --usage' for more information.


I tried tar --help and couldn't figure it out. It sounds like the command is trying to create an archive and then fill it which tar doesn't seem to like. How would I go about fixing this? :confused:

Much thanks,
M4rotku

[M4rotku]

bump, anyone?

[scorp123]

... message deleted: accidental double-post ...

Sorry about that. ;)

[scorp123]

As you can see, after I entered the command, it outputted ">". Of course. It only takes 1 second to spot your mistake:

joey@joey-laptop:/$ tar cvpjf backup.tar.bz2 --exclude=/proc \
--exclude=/lost+found --exclude=/backup.tar.bz2 \
--exclude=/mnt --exclude=/home/joey/Music \
--exclude=/home/joey/Videos \
--exclude=/home/joey/Pictures \
--exclude=/home/joey/ISO's \
--exclude=/home/joey/Photos --exclude=/sys /

The apostrophe << ' >> has a special meaning to the shell. You should definitely not use it in file names.

You have to escape it (= UNIX/Linux lingo for "how to use special characters without the stupid shell thinking that it has to interpret that special character") e.g. by putting a backslash in front of it, e.g. ... ISO\'s ...

Small example involving the "mkdir" command:
mkdir ISO's
> => Same behaviour as "tar" in your example above. The shell stops because it thinks you're about to input a string of some sorts. This is obviously not what we wanted.

With the backslash:
mkdir ISO\'s => Command terminates without error. A "ls -al" shows that I now have this new directory: drwxr-xr-x 2 adm adm 4096 2008-08-16 10:14 ISO's ... So this is what you want.

But it would be better to avoid such special characters altogether and just name the directory in question "ISOs" ;)

[user11]

I just use remastersys

[satir.satir]

add this in the thirth party softwear
deb http://www.remastersys.klikit-linux.com/repository remastersys/
sudo apt-get install remastersys
to create Live CD/DVD :)
sudo remastersys backup
or
sudo remastersys dist
System>Administration>Remastersys Backup
http://www.remastersys.klikit-linux.com/
:guitar:

[sprechkaese]

Hi there

got one really wired error there:

root@linksys_01:/# tar cvpzf backup.tgz --exclude=/backup.tgztar: Anlegen eines leeren Archivs wird feige verweigert.
„tar --help“ oder „tar --usage“ gibt weitere Informationen.


what the heck does he want to tell me with that??? :confused:

[scorp123]

Hi there got one really wired error there That's pretty far away from a "weird" error. :) As a matter of fact, all Linux error messages are pretty clear. For weird error messages: Use Windows and get some hexadecimal bluescreens.

root@linksys_01:/# tar cvpzf backup.tgz --exclude=/backup.tgztar: Anlegen eines leeren Archivs wird feige verweigert.
„tar --help“ oder „tar --usage“ gibt weitere Informationen.
what the heck does he want to tell me with that??? :confused: That you should check your syntax maybe? ;) You see: You are not specifying at all *WHAT* you want inside that archive, you only specified what you do not want inside of it. The error message is thus correct: You *ARE* trying to create an empty archive ... which does not make sense at all. Hence it refuses to do that because the program suspects an error in your syntax.

[birkopf]

2: Restoring
Once again, make sure you are root and that you and the backup file are in the root of the filesystem.

One of the beautiful things of Linux is that This'll work even on a running system; no need to screw around with boot-cd's or anything. Of course, if you've rendered your system unbootable you might have no choice but to use a live-cd, but the results are the same. You can even remove every single file of a Linux system while it is running with one command. I'm not giving you that command though! ;-)


Three questions:

1) I am a pretty new user and I am quite good at destroying my Kubuntu. 6 reinstalls within 3 days. If I will do it again (like last time) that after turning on laptop, there is just black screen... even GRUB doesn't want to load

- whats the next step from there ?
Should I put linux installation disk and do sth
or is there any other way to get into command line and execute commands you written in howto ?

2) I have two partitions ( one for / and one for /usr) and I have moved backup'ed file to second one. Do I need to move it back to / to start system restore ? I am still talking about situation from point one.

3) How I can "thank you"? I mean - forget about drink :lolflag: but there should be some option for this, but I can't find it...

[scorp123]

that after turning on laptop, there is just black screen... even GRUB doesn't want to load You should open a separate thread about that error and supply more information. Maybe your harddisk is defective ... or maybe it's something that can easily be fixed.

whats the next step from there ?
Should I put linux installation disk and do sth or is there any other way to get into command line and execute commands you written in howto ? Bingo. You got it. If your laptop does not want to boot anymore no matter what: Boot the live CD (the Ubuntu "Desktop" CD) and try your luck there.

2) I have two partitions ( one for / and one for /usr) That doesn't make much sense to be honest. If you want an elaborate partitioning scheme (with separate /boot, /, /usr, /opt, /var and /home partitions) then you should do that instead. If you want to keep things simple then you should at least go for a separate / and /home.

I am still talking about situation from point one. You don't even know what the issue is. Did you check the filesystem etc.? You should do a forum search on how to do that from the live system.

3) How I can "thank you"? There should be that "medal-like" icon ... that's the "thank you" button.

[AceRimmer]

Great thread guys. I've got about 400gb of data I want to backup, a lot of video and music along with other stuff. I did a tar as the initial post detailed and it worked, but it took at least 10 hours. Is there any faster method and what about incremental so the next backup isn't so long? I realize its a lot of data so maybe 10 hours is just what to expect.

And are there any good GUI programs available?

[birkopf]

Thanks for answer.

I used it first time and it created files about 1.7GB. When I used it again today it created files 27GB and I have 3% free space on my partition.

Where this change came from? I didn't install anything within' few days and I just copy 3GB to my /home, so it shouldn't be so big.

Any ideas what to do next?

[rugbert]

Ok so it looks like the backup worked successfully but the restore part not so much. after it untars if i try to get a directory listing it kirks out.

I was able to cd to / but if I hit ls I get this:

[14644.xxxxxx] request_module: runaway loop modprobe binfmt-464c

then when I try to boot to a live CD and jump into grub to reinstall i just keep getting Error 15: file not found

[scorp123]

I've got about 400gb of data I want to backup, a lot of video and music along with other stuff. I did a tar ... That does not make much sense. Videos and music files such as MP3 are already compressed, trying to compress them further is a waste of time and CPU power and won't do much good. It would be better to simply copy the contents of that file system to another location, e.g. external USB disk of appropriate size. In the stores in my area I can see that 2 TB USB disk drives and NAS's have gotten really really cheap lately, so if you can afford it I'd go for one of those really big disks.

[AceRimmer]

That does not make much sense. Videos and music files such as MP3 are already compressed, trying to compress them further is a waste of time and CPU power and won't do much good. It would be better to simply copy the contents of that file system to another location, e.g. external USB disk of appropriate size. In the stores in my area I can see that 2 TB USB disk drives and NAS's have gotten really really cheap lately, so if you can afford it I'd go for one of those really big disks.

Good point. What I want to do is backup my 750gb main disk to the backup 750gp disk in the same system. I want it to be automated and incremental. I don't need any more hard drives since the other 750gb disk is just there for backing up this system, and some data off my other computers. Some of the files should be compressed but as you said not all need it but I don't want to have to mess around with multiple operations. Any suggestions?

[tigerplug]

Thanks for the info!

Must give it a go later on.

[scorp123]

I used it first time and it created files about 1.7GB. When I used it again today it created files 27GB and I have 3% free space on my partition. Well, I guess that depends on what you packed into that archive? :)

Where this change came from? My crystal ball is malfunctioning at the moment ;-) .... But I'd suspect you simply mistyped something into the command line. There's your change. And please: If you're a beginner please write down the stuff you type in!! And when you ask for help: Show us what you typed to get that weird result! Nobody can help you if you don't give any specific details on the commands you executed.

Any ideas what to do next? Buy a new harddisk? Or free up some space? Or check that command line again so you can be sure it really does what it is supposed to do?

[birkopf]

My crystal ball is malfunctioning at the moment ;-) .... But I'd suspect you simply mistyped something into the command line. There's your change. And please: If you're a beginner please write down the stuff you type in!! And when you ask for help: Show us what you typed to get that weird result! Nobody can help you if you don't give any specific details on the commands you executed.

I copied command line straight into Konsole, so it's not that.

When I was doing it 4th time - I noticed something interesting - all stuff that I had deleted from disk, was in root folder in trash. It seems that in Kubuntu - after deleting files as user, to bin, and emptying bin - all files are still not deleted byt transferred to second trash which belongs to root.
- Am I right ?

[rugbert]

Ok so it looks like the backup worked successfully but the restore part not so much. after it untars if i try to get a directory listing it kirks out.

I was able to cd to / but if I hit ls I get this:

[14644.xxxxxx] request_module: runaway loop modprobe binfmt-464c

then when I try to boot to a live CD and jump into grub to reinstall i just keep getting Error 15: file not found

Bump please!

also - might I suggest starting a new thread for organizational reasons. the thread is getting kinda long.

[scorp123]

I copied command line straight into Konsole, so it's not that. Well, you have to watch out ... maybe the thread author's disk setup was/is different from your's and hence might produce different results on your setup?

all files are still not deleted byt transferred to second trash which belongs to root. Am I right ? Nope. There is something wrong there. Sounds like a corrupted file system to me. ReiserFS used to do such weird things from time to time (you could bring back long dead files with a simple filesystem check! Zombie files, Eeeeek ... :D ) but I imagine you stayed with the defaults and use Ext3?

Can you please copy & paste the content of this file here: /etc/fstab (make notes of the "/dev/sdxxxx" something entries, you will need those - see below).

Best thing would be if you boot into a Live CD and then issue this command: sudo fsck.ext3 -f /dev/sdxxxxx__whatever ... You will have to replace "sdxxxx_whatever" with whatever "/etc/fstab" says about you having Ext3 filesystems. Example output:
# /dev/sda1
UUID=foo-bar-123 /boot ext3 relatime 0 2 ... So that's an Ext3 filesystem there (my /boot partition ... don't worry if you don't have that ... I'm "oldschool" ;) ). You should be able to identify those entries from your system and then have the filesystems checked with that command above.

[scorp123]

Ok so it looks like the backup worked successfully but the restore part not so much. after it untars if i try to get a directory listing it kirks out.

I was able to cd to / but if I hit ls I get this:

[14644.xxxxxx] request_module: runaway loop modprobe binfmt-464c

then when I try to boot to a live CD and jump into grub to reinstall i just keep getting Error 15: file not found No idea about that kernel module. Sorry but you will have to use Google to find out what it is used for and why it misbehaves. As for the rest: Sounds like a badly corrupted disk or at least a badly corrupted boot loader. Did you try the "Super GRUB Disk" (sometimes called 'SGD' for short). You will have to use Google to find it. It's a very small live CD specialised in bringing corrupted boot loaders back and booting "unbootable" systems again. I'd highly recommend it. It does a very good job at guessing what you want. So you just start it, you select something like "Boot Linux, first disk" (not sure what exactly it is called), and voila, you're back in your formerly dead OS and can reinstall the proper boot loader from there (e.g. sudo update-grub).

[gaixixon]

hi, I've been diggin' around and I don't know if I miss something or not. Now straightly to my question. I've made a full backup and save it in /fullbak.tgz now I want to make an incremental backup, of course with tar console not with Acronis or something Window$$..
any help?
thnx.. :confused:

[Aiman]

I'm facing a problem here,i made a backup of my system couple days back and decided to use it today on a fresh install,all went fine except for the part where grub loaded
i have kernel 2.6.24.19 in the backup,when i try to load it from grub,it will show ubuntu loading and stay for ever
when i load kernel 2.6.24.16,it works and gets to the desktop with all setting restored fine,i thought it was just from grub,when i checked within ubuntu,it was really loaded on kernel 2.6.24.16,any idea ??

[rugbert]

No idea about that kernel module. Sorry but you will have to use Google to find out what it is used for and why it misbehaves. As for the rest: Sounds like a badly corrupted disk or at least a badly corrupted boot loader. Did you try the "Super GRUB Disk" (sometimes called 'SGD' for short). You will have to use Google to find it. It's a very small live CD specialised in bringing corrupted boot loaders back and booting "unbootable" systems again. I'd highly recommend it. It does a very good job at guessing what you want. So you just start it, you select something like "Boot Linux, first disk" (not sure what exactly it is called), and voila, you're back in your formerly dead OS and can reinstall the proper boot loader from there (e.g. sudo update-grub).

oh well turns out im dumb and was trying to do this on a 32bit machine fro ma 64bit one. its working better now. Im just having grub and apache issues. Im sure I can work that out...hopefully!

[Buttons840]

I'm having a problem with this system.

I created a backup, but screwed up and didn't exclude the backup.tgz, so I backed-up the backup. No problem, I deleted it, which sent it to the trash, and I then deleted it from the trash.

Every time I make a back up now, SOMEHOW the backup process locates a old backup file. I've searched my hard drive as well as I know how (I'm new to linux) using the find files option, and also doing a search from the root.

The file is located at /root/.local/share/Trash/files/backup.tgz

What's with that ".local"? I can't find that in the root with the file browser/navigator.

I have the original backup up inside the 2nd backup, which is inside the 3rd backup, which is inside the 4th backup, which is inside the 5th backup... you get the point. Needless to say, each backup is more and more bloated.

I'm using ext2. I cannot use ext3. I receive an error when attempting to install using ext3 outlined in this post http://ubuntuforums.org/showthread.php?t=314940 as far as I can tell nobody has solved this issue. I spent days on this issue until I found the first truly helpful post which said "Use ext2 instead of ext3."

Any help with my bloated backups is appreciated.

[broken tibula]

I, ah, messed something up. Kind of. Actually, I just wasn't thinking.

I made the backup, but it ended up being to big for the partition, so it crashed or something... Anyway, I got Ubuntu back up and running, only the graphics have taken a serious hit and I got some error message logging in that said that I didn't have enough harddrive to run Nalitus (I didn't see it long enough to write it down). I'm logged in right now, and the easiest solution seems to be to delete the backup file and then expand my partition and then try again, but when I try to delete the file, it won't let me. It says "Permission denied".

What can I do? I'm a complete newbie, I really shouldn't have been messing with this stuff at all, and now I have and I've messed it all up. :(

ETA: Double crap. I can't run any administrative tasks. I get the message:

Failed to run /usr/sbin/synaptic as user root.

Unable to copy the user's Xauthorization file.

This might possibly be related to the "permission denied" issue...

[scorp123]

now I want to make an incremental backup, of course with tar console ... any help? Open a terminal and read the manual: man tar (navigate up and down with the cursor keys; press "Q" to quit).

There are various parameters there (e.g. "-u" and ) which can do what you want; you'd just need to figure out how exactly you want to do it.

I'd copy the existing backup script (let's call it "fullbackup.sh") to another location and call that one "incremental_backup.sh"; then edit that file and just change the parameters so that it says "tar -uvf ..." instead of the previous parameters (with the "-c" parameter) which would create a new archive file from scratch.

[scorp123]

The file is located at /root/.local/share/Trash/files/backup.tgz You logged into the desktop as root??? Or why else would the file now be in root's trash folder?

What's with that ".local"? I can't find that in the root with the file browser/navigator. Files and folders with a dot in front of them are "hidden", a file browser will not show them per default.

As a normal user you should not be able to access root's home folder /root and as "root" you should not use graphical programs such as file browsers because too many things can go wrong!


To get rid of the file: sudo rm -i /root/.local/share/Trash/files/backup.tgz

[scorp123]

ETA: Double crap. I can't run any administrative tasks. Probably your file system is 100% full (based on your descriptions) ... UNIX-like OS really really don't like it when their file systems get 100% full and act weird when it happens. The problem here is that stuff like switching between user accounts (e.g. via "sudo") and opening programs requires write access to /tmp and /var ... but with the filesystem being 100% full no write accesses are possible and lots of stuff will fail; other things not requiring any write access might still work though. As I said ... acting weird.

What I recommend: Reboot into "recovery" mode and then as "root" try to free up some space by removing that *tar.gz file which has gotten too big. Then reboot.

Or: You could boot a desktop live CD and then mount your disk partition(s) and then do the same from there in a graphical way if you're more comfortable with that. Try to free up some space by removing those backup files (and nothing else please!!) and then reboot, see if your system behaves normally again.

[MadJestyr]

Though I realize that this is an ongoing discussion, I have a question if you do not mind, heliode. As with most things, I am teaching myself Linux/Unix the hard way, by trial and error. I have had to reboot my system more than once in the last two weeks because I was playing around. Your back-up system seems to work fine, but i was wondering why you excluded the /sys folder? Does it reload when you re-install with the live-cd? Is there any reason that you wouldn't exclude it? Such as a change of information? And what would constitute such a change? Anyone out there with the answer feel free to educate me.


Thank you for your time.
Always,
MadJestyr

[MadJestyr]

Ok, so, Doh, i'm a retard, found the answer in the first page of the thread. I should read a little farther before I open my big mouth.


Thanks anyway,
MadJestyr

[Gabix]

I ran into problems with this method because of my noobness and everything was resolved in this thread: http://ubuntuforums.org/showthread.php?t=911678
I still have some things that look strange on boot, but everything looks to be working fine...

THANKS Heliode!!

[Kow]

Just to add my 2 cents: Instead of having to include all of these --exclude commands for other partitions/filesystems why not use tar's "--one-file-system" option? Not sure how well it works but someone should try it. Would save a lot of troubleshooting for a few people who forget an exclude due to some bind mount or loopback they setup a while ago and forgot about.

[scorp123]

Instead of having to include all of these --exclude commands for other partitions/filesystems why not use tar's "--one-file-system" option? "--one-file-system" in the case of the "/" (root) filesystem would ignore external mount points, yes. But it would not ignore things like /tmp (which would be a waste to backup) or virtual filesystems such as /proc and /sys ... because those things are still on "/" even though those files are virtual.

You most likely have to use both options "--one-file-system" + "--exclude" in conjunction depending on how your system was setup.

[coldcoffee]

I hope I am not stepping on any toes by posting this. I sure am not meaning too. But I personally use a program called PING (http://ping.windowsdream.com/) for backing up my system. I do have a second hard drive though which makes it easy for me. You can also make bootable recovery disks with it. It is a program I find very useful.

[escapedturkey]

Can this be done with rsync as well or does it require tar?

Tar is nice but multiples can take up lots of space, whereas rsync simply updates files and can be done daily.

Thank you. :)

[raynard]

I'm unable to boot up the kernel anymore, but I still have a tar backup made by this guide.
Now, when I try to extract the content of this tarball with the live-cd, it refuses to write to the disk (it can only read from it). I suppose this has to do with permissions, but I don't know how to solve it. Any help would be appreciated.
(By the way, the backup is stored on an external hd, formatted in FAT32. Can this cause any problems, because I've read somewhere in this thread the file permissions could be gone ?)

[gaixixon]

have you tried this:
at the console:
sudo su
cd /
tar xvpf {your archieve}

[gopher38]

Hello. I've got a problem. I've used the commands on the first post of this thread in the past to restore an HP notebook with Ubuntu:
Archive
tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /
Restore
tar xvpfz backup.tgz -C /

Worked great. But I've got a problem now on a different. I've got an eee, on which I put eeeUbuntu. I've an archive on disk that I want to restore, but it keeps dying saying that there's not enough space to copy a bunch of files and then completely giving up the ghost. Will not allow me to boot back in either.

Here's the deal. Those eee's don't have much space. There's only 4G. I think that the system occupied about 2G, which went down to 800M when tar'ed. If the restore just out and out clobbered the old files, it should fit back on, right? It seems as though tar isn't strictly overwriting the files, but somehow keeps them (temporarily?), effectively trying to put two OS's onto the machine, which runs out of room. I've tried looking through the tar man page, and tried to boot with LiveCD and delete all the files and then run tar, but that didn't work either. Same problem. I'm not sure that I succeeded in deleting the files either.

Anyone have any good ideas for me?

Thanks.

[forger]

try extracting using --overwrite
--overwrite overwrite existing files when extracting


tar xvpfz backup.tgz --overwrite -C /
By the way, are you using the above command *exactly* as it is written or..?

[davethewave83]

Hmm I backed up my system, formatted in LVM Encrypted mode, restored and now grub won't boot the system, I type Boot at "Grub" command prompt and it says the kernel must be loaded first. :confused: it's supposed to ask me for the LVM password

*edit* another problem with this method is that it does not keep the permissions, I am going in circles trying to figure out what to chmod all my files now because the permissions changed after extracting them from the backup.

[RavUn]

I'm planning on reinstalling everything on my laptop and want to know if this method still works? I see it's over 3 years old.

[scorp123]

I'm planning on reinstalling everything on my laptop and want to know if this method still works? I see it's over 3 years old.

http://ubuntuforums.org/showpost.php?p=5016635&postcount=577

[otheracco]

I've made a backup and everything seemed to go OK. I did get the error right at the end but, like the guide says, I ignored it.

I'm having a lot of problems restoring onto an external USB disc with the following directive -
tar xvpfz backup.tgz -C /mnt/ext

This is being run in a root shell, but I'm getting the following errors;
tar: cannot mkdir: no such file or directory
tar: cannot mkdir: input output error

Most of the I/O errors are in the /var/cache folder, but I haven't gotten past the folder yet, because it takes several minutes to timeout with the error on almost every file and I've ended up killing the process.
Since this is run as root, what can the problem be?

[scorp123]

I'm having a lot of problems restoring onto an external USB disc And that external USB disk has been formatted with a Linux filesystem so it is capable to recreate the permissions and other special UNIX-style attributes of each file?

[otheracco]

And that external USB disk has been formatted with a Linux filesystem so it is capable to recreate the permissions and other special UNIX-style attributes of each file?

It was a bit by bit copy done with HDCLONE. It was showing as ext3 under gparted.

I rewrote ext3 to it anyway and now it's working great!
Thanks for the suggestion, but strange problem anyway.

[RavUn]

I can't seem to get this to work. I backed everything up and tried restoring with

tar xvpfz backup.tgz --overwrite -C /

I made sure I ran it as root and had the backup file stored in the root dir. It would start unpacking then my screen flashes a few times then gets stuck at a screen that says "checking battery state... [OK], running scripts... [OK]" I reboot and it copied some files over because I see the kernel I was using in the GRUB menu but it doesn't boot up. It just goes to a command prompt with (initdram).

I tried it without the --overwrite also and it doesn't flash the screen or anything and it completes, but when I reboot I still get the same error with it not booting and going to the command prompt.

Any ideas what's up?

[scorp123]

It would start unpacking then my screen flashes a few times then gets stuck at a screen that says "checking battery state... [OK], running scripts... [OK]" I reboot and it copied some files over because I see the kernel I was using in the GRUB menu but it doesn't boot up. It just goes to a command prompt with (initdram). Your PC crashes when you unpack a tar.gz file?

[RavUn]

Only when I unpack this tar file. It crashed if I used --overwrite... I assume because it's overwriting the files I'm currently using. If I don't use --overwrite then everything unpacks successfully but nothing I click on works... I was unable to click "log out" or connect to the internet... so I restart and try to go back to ubuntu and it loads for a while and I get to the command prompt with (initdram).

[scorp123]

Only when I unpack this tar file. It crashed if I used --overwrite... I assume because it's overwriting the files I'm currently using. If I don't use --overwrite then everything unpacks successfully but nothing I click on works... I was unable to click "log out" or connect to the internet... so I restart and try to go back to ubuntu and it loads for a while and I get to the command prompt with (initdram). You just learned how not to restore stuff!!

I don't mean to sound mean or rude ... But: You can't repair a car while it's still driving 130 km/h down the highway, right? ... if anyone tried anyway then the most likely result would be a crash. Well, guess what: Same with Operating Systems. You can't do major surgery or major repairs on an Operating System while it's still running .....

It's one thing if you just restore one single lost file ... that should be OK. But overwrite an entire installation while it's still in use and loaded into memory?? That's got to end in a crash :D I am not aware of any OS that would not react in such a way if you do this.

So I'd suggest you reinstall if necessary (your filesystem is now most likely corrupted anyway) and the next time you restore something please make sure you only restore the files that need to be restored. If you need to restore the complete OS then please reboot your PC into a Live CD, mount your computer's harddisk and restore from here.

While I am at it you could maybe also read this .... I once wrote this for the Linux Mint forum:

http://www.linuxmint.com/forum/viewtopic.php?f=42&t=3969&start=0&st=0&sk=t&sd=a

[RavUn]

hmmm... I was going off of the OP's comments

One of the beautiful things of Linux is that This'll work even on a running system; no need to screw around with boot-cd's or anything. Of course, if you've rendered your system unbootable you might have no choice but to use a live-cd, but the results are the same. You can even remove every single file of a Linux system while it is running with one command. I'm not giving you that command though!

Anyway, how could I do it from a livecd? I tried mounting the HDD and going to the root of /media/disk/ (which has all of my ubuntu files that were installed on my ubuntu partition) and untar it but it says it runs out of room (so I asume I'm untaring it to the livecd somehow?)

[scorp123]

Anyway, how could I do it from a livecd? Did you check the link I gave you?

[scorp123]

hmmm... I was going off of the OP's comments: "One of the beautiful things of Linux is that This'll work even on a running system; ... " Making backups, yes. "tar" will just take whatever is stored on the disk in the moment it makes a backup. Hence why I use the "sync" command a lot in my own "Backup How-To", just to make sure I really get the latest version and all buffers are written to disk. In some occasions (e.g. high-load web servers, databases, etc.) it might even be necessary to stop all processes and go into "single-user" mode before doing a backup. But yes, you can make a backup while the system is fully running.

And yes, you can restore single files. Especially if you just deleted them accidentally.

But the entire OS? While it's still running? Nope. You have to know what you do here, e.g. if you are going to overwrite binaries then it would be a wise idea to stop those first, or if necessary use "kill", "pkill", "killall" and similar such commands just to make sure that the programs are really gone from the system's memory before you overwrite them ...

For a complete restore it would be best to be in a Live CD session, that way you won't be accidentally corrupting any running programs on your system.

[derekshaw]

For the record:

Regardless of what OS you are using, your chances of success in backup and recovery are hugely greater when the system is quiescent. If you don't have a live CD, then a "recovery mode" boot will give you a pretty good chance.

If you have any but the simplest needs, this book is invaluable:
Unix Backup and Recovery (by W. Curtis Preston)
http://safari.oreilly.com/1565926420

[Green_Star]

I am not sure about this, but I did restored many many times when Ubuntu is running(with GUI Gnome). Of course, I closed all applications which I can see on my screen but not the back ground services. I restored without any problems.

But anyway, if you have problems when Ubuntu GUI running then try to boot into maintenance mode(I do not remember the exact name but it should be just below of your normal selection of GRUB screen), select admin prompt and try restoring again. It may work flaw less/reduce your problems because GUI part is not running.

On second thought I think I didn't used '--overwrite'. I will check when I go home.

You just learned how not to restore stuff!!

I don't mean to sound mean or rude ... But: You can't repair a car while it's still driving 130 km/h down the highway, right? ... if anyone tried anyway then the most likely result would be a crash. Well, guess what: Same with Operating Systems. You can't do major surgery or major repairs on an Operating System while it's still running .....

It's one thing if you just restore one single lost file ... that should be OK. But overwrite an entire installation while it's still in use and loaded into memory?? That's got to end in a crash :D I am not aware of any OS that would not react in such a way if you do this.

So I'd suggest you reinstall if necessary (your filesystem is now most likely corrupted anyway) and the next time you restore something please make sure you only restore the files that need to be restored. If you need to restore the complete OS then please reboot your PC into a Live CD, mount your computer's harddisk and restore from here.

While I am at it you could maybe also read this .... I once wrote this for the Linux Mint forum:

http://www.linuxmint.com/forum/viewtopic.php?f=42&t=3969&start=0&st=0&sk=t&sd=a

[noremac]

Just wanted to someone to assure me that I did a proper backup before I burn it to a DVD. Here is the command I did:

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys --exclude/dev --exclude=/media /

Essentially the same thing as posted, plus excluded /media and /dev

Ended up with a 3.8GB file.

Sound about right?

[Green_Star]

Just wanted to someone to assure me that I did a proper backup before I burn it to a DVD. Here is the command I did:

tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys --exclude/dev --exclude=/media /

Essentially the same thing as posted, plus excluded /media and /dev

Ended up with a 3.8GB file.

Sound about right?

I think so, mine is around 2.8GB, I do not have anything in my home folder. I store everything into my external drive. By the way why you are backing up to dvd? any specific reason??

[noremac]

I think so, mine is around 2.8GB, I do not have anything in my home folder. I store everything into my external drive. By the way why you are backing up to dvd? any specific reason??

I figure why not since it will fit on one. Perhaps give it to someone else in case something terrible were to happen. However, if something THAT bad were to happen, getting my computer back up and running is probably the last thing on my mind.

More so though, I just love the fact that it will fit on one DVD.

[Green_Star]

I figure why not since it will fit on one. Perhaps give it to someone else in case something terrible were to happen. However, if something THAT bad were to happen, getting my computer back up and running is probably the last thing on my mind.

More so though, I just love the fact that it will fit on one DVD.

I am not sure how frequent you do backup. I made a script, it will backup every week to my other drive using cron and give that backup name as backup-'date'.tgz and delete the old backups which are older than two weeks, i.e., at any time I will have two previous backups.

[escapedturkey]

What'd be nice to do is:

Hook up USB stick - Reboot - Grub - boots to a clonezilla install - runs a script that creates image and copies to USB stick - when finished reboots to Ubuntu.

Is that possible?

[Green_Star]

I never heard of clonezilla, I just googled it, looks like it makes a disk/partition copy. Thanks for letting me know about that product.

I think you may have to write a script in clonezilla to make a image of your required disk directly to your flash drive, and reboot after the process.

To boot directly from USB you may have to change your BOIS boot sequence. Even though if you are not at the computer when it is rebooting after the process then it will boot again to your USB.

As of now, I can say that you can not automate the process, it requires some manual work.

OR

Install clonezilla into one of your usb.

IN UBUNTU

Make a copy of your menu.list as menu.clonezilla
Modify the menu.clonezilla so that there is only one entry that will boot directly into your clonezilla
Create a script, it will move your menu.list to menu.backup and menu.clonezilla to menu.list, then it reboots the system.
Make sure this script runs as root
Schedule this script using corn


In CLONEZILLA

Make a script to create a disk image into your USB, after this process it will move the ubuntu partition menu.list to menu.clonezilla and menu.backup to menu.list, reboot the system.


I am not sure how we can make an GRUB entry so that it will boot from USB. The problem with this method is, when this ubuntu schedule kicks in the USB drive should be plugged in. And if some reason machine is not able to boot from USB then you may have to restore your GRUB with some other methods. So instead of loading clonezilla into your USB I guess it is better to install in any other drive available which is not going to backup by clonezilla. Because I think clonezilla wants the disk should be unmounted before making a clone, so that means if you want to clone a disk from where clonezilla is booted then it may not possible. Check with clonezilla documentation.

What'd be nice to do is:

Hook up USB stick - Reboot - Grub - boots to a clonezilla install - runs a script that creates image and copies to USB stick - when finished reboots to Ubuntu.

Is that possible?

[escapedturkey]

I was thinking more along the lines of if you could install CloneZilla in a partition on the HD, then hook up a USB stick or insert a writeable cd/dvd, reboot, Grub, it boots CloneZilla off your HD, than a script automatically creates image, copies, and reboots. Would that be easier? :)

[lmicu]

Hi, and welcome to the Heliode guide to successful backing-up and restoring of a Linux system!

Most of you have probably used Windows before you started using Ubuntu. During that time you might have needed to backup and restore your system. For Windows you would need proprietary software for which you would have to reboot your machine and boot into a special environment in which you could perform the backing-up/restoring (programs like Norton Ghost).
During that time you might have wondered why it wasn't possible to just add the whole c:\ to a big zip-file. This is impossible because in Windows, there are lots of files you can't copy or overwrite while they are being used, and therefore you needed specialized software to handle this.

Well, I'm here to tell you that those things, just like rebooting (http://www.ubuntuforums.org/showthread.php?t=34629), are Windows CrazyThings (tm). There's no need to use programs like Ghost to create backups of your Ubuntu system (or any Linux system, for that matter). In fact; using Ghost might be a very bad idea if you are using anything but ext2. Ext3, the default Ubuntu partition, is seen by Ghost as a damaged ext2 partition and does a very good job at screwing up your data.

1: Backing-up

"What should I use to backup my system then?" might you ask. Easy; the same thing you use to backup/compress everything else; TAR. Unlike Windows, Linux doesn't restrict root access to anything, so you can just throw every single file on a partition in a TAR file!

To do this, become root with

sudo su

and go to the root of your filesystem (we use this in our example, but you can go anywhere you want your backup to end up, including remote or removable drives.)

cd /


Now, below is the full command I would use to make a backup of my system:


tar cvpzf backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

Now, lets explain this a little bit.
The 'tar' part is, obviously, the program we're going to use.

'cvpfz' are the options we give to tar, like 'create archive' (obviously),
'preserve permissions'(to keep the same permissions on everything the same), and 'gzip' to keep the size down.

Next, the name the archive is going to get. backup.tgz in our example.

Next comes the root of the directory we want to backup. Since we want to backup everything; /

Now come the directories we want to exclude. We don't want to backup everything since some dirs aren't very useful to include. Also make sure you don't include the file itself, or else you'll get weird results.
You might also not want to include the /mnt folder if you have other partitions mounted there or you'll end up backing those up too. Also make sure you don't have anything mounted in /media (i.e. don't have any cd's or removable media mounted). Either that or exclude /media.

EDIT : kvidell suggests below we also exclude the /dev directory. I have other evidence that says it is very unwise to do so though.

Well, if the command agrees with you, hit enter (or return, whatever) and sit back&relax. This might take a while.

Afterwards you'll have a file called backup.tgz in the root of your filessytem, which is probably pretty large. Now you can burn it to DVD or move it to another machine, whatever you like!

EDIT2:
At the end of the process you might get a message along the lines of 'tar: Error exit delayed from previous errors' or something, but in most cases you can just ignore that.

Alternatively, you can use Bzip2 to compress your backup. This means higher compression but lower speed. If compression is important to you, just substitute
the 'z' in the command with 'j', and give the backup the right extension.
That would make the command look like this:


tar cvpjf backup.tar.bz2 --exclude=/proc --exclude=/lost+found --exclude=/backup.tar.bz2 --exclude=/mnt --exclude=/sys /


2: Restoring

Warning: Please, for goodness sake, be careful here. If you don't understand what you are doing here you might end up overwriting stuff that is important to you, so please take care!

Well, we'll just continue with our example from the previous chapter; the file backup.tgz in the root of the partition.

Once again, make sure you are root and that you and the backup file are in the root of the filesystem.

One of the beautiful things of Linux is that This'll work even on a running system; no need to screw around with boot-cd's or anything. Of course, if you've rendered your system unbootable you might have no choice but to use a live-cd, but the results are the same. You can even remove every single file of a Linux system while it is running with one command. I'm not giving you that command though! ;-)

Well, back on-topic.
This is the command that I would use:


tar xvpfz backup.tgz -C /


Or if you used bz2;


tar xvpfj backup.tar.bz2 -C /


WARNING: this will overwrite every single file on your partition with the one in the archive!

Just hit enter/return/your brother/whatever and watch the fireworks. Again, this might take a while. When it is done, you have a fully restored Ubuntu system! Just make sure that, before you do anything else, you re-create the directories you excluded:

mkdir proc
mkdir lost+found
mkdir mnt
mkdir sys
etc...


And when you reboot, everything should be the way it was when you made the backup!

2.1: GRUB restore
Now, if you want to move your system to a new harddisk or if you did something nasty to your GRUB (like, say, install Windows), You'll also need to reinstall GRUB.
There are several very good howto's on how to do that here on this forum, so i'm not going to reinvent the wheel. Instead, take a look here:

http://www.ubuntuforums.org/showthread.php?t=24113&highlight=grub+restore

There are a couple of methods proposed. I personally recommend the second one, posted by remmelt, since that has always worked for me.


Well that's it! I hope it was helpful!
As always, any feedback is appreciated!

I followed the instructions here and I also excluded the /dev. I was backing up on a external hard drive and I received an error like this:

bzip2: I/O or other error, bailing out. Possible reason follows.
bzip2: No space left on device
Input file = (stdin), output file = (stdout)
root@lucian-laptop:/media/My Book#

is this normal, I think that there were more things to back up, my backup file is only 6 G, it should be around 20 or more, space is not an issue I have around 300 free on this hdd

thanks

[escapedturkey]

Can any of this be done with rsync -- to a usb stick, dvd, or remote server?

[steveydoteu]

I would suggest using rsync with cron jobs, but it is indeed possible, as this howto is only aiding you in creating the back up files which can then be moved to a remote location.

You would need a script to create the back up, and one to use rsync for uploading/moving. Although they could probably be combined. This then needs to be passed to cron in the form of a cron job.

My knowledge of cron/rsync is limited though, so I could be totally wrong.

[commonplace]

Great tutorial. Due to disk space constraints, I needed to export the tar to an external drive that is fat32 formated. If this hasn't been posted already, thought I would share how to get around the 4GB size restraint using the split command.

If you take Heliode's command and alter it like this

tar cvpzf - --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/media --exclude=/sys / | split -a 2 -b 2000m - /media/ieee1394disk/backup.tgz


It will create your backup.tar in 2000MB chunks at /media/ieee1394/

Restore like

gzcat /media/ieee1394disk/backup.tgz* | tar xvpfz - -C /


I've done the backup portion using this, but haven't actually tested a restore. So use at your own risk :)


The gzcat command isn't on my system (Ubuntu 8.10) and I can't find it in the repos. Can I use gunzip somehow, or how do I get gzcat? I've got my backup but no way to restore it at the moment. :)

Edit: Figured out gzcat is the same as gunzip -c

/Kevin

[brandroid]

I'm not sure if this has been asked (or what to search for if it has), so just point me to the answer if it's already out there: I want to backup my existing Ubuntu 8.1 installation and save it on my ex-HD. Then I want to install Ubuntu on a virtual machine (under WinXP), and take the data I backed up and restore it there.

Is it possible? Are there other steps I should do?

Sort of second question: Would backing up just home do what I need? My biggest worry is losing my programs/settings/etc and having to go through and install (and set them up) all over again.

Thanks in advance for any help.

[gnome_mm]

Sort of second question: Would backing up just home do what I need? My biggest worry is losing my programs/settings/etc and having to go through and install (and set them up) all over again.

Concerning your second question:
As a user, the only directory you have write access to is your home directory (apart from some other directories such as /tmp, removable drives under /media and so on). And since you will most likely execute your programs as a user (and not root), the settings of that program can only be saved in your home directory, e.g. .gimp, .mplayer and so on. Even your panel and gnome-specific settings are saved there, e.g. in .gnome2. This is the reason why it is often regarded sufficient to backup your home partition and do a fresh install of your system to keep your settings and files.

However, be advised that this might not hold true if a different release of Ubuntu is installed as some programs and both the way they store their settings (xml instead of a simple text file) and the location where they are stored (e.g. liferea --> liferea_1.4) might have gone through a major change.

Keeping your home directory along with a list of installed packages (e.g. have synaptic spit out a text file with all your packages and use that in a single apt-get command) should often be sufficient as a backup solution if you didn't mess around as root too much.

For convenience though, you can create a first backup of your system using the offered solution.

Now that I think about it. Can't you just install Ubuntu on your VM, copy over your home directory and install whatever package was left out during the installation by using the log created from synaptic? This way, you keep both your programs and settings.

Regards

EDIT: Thanks for the useful hints in this thread. For my personal needs, I prefer the TAR backup solution over all others that are available. There is, however, a disadvantage to it and it has been pointed out in this thread several times: files are only overwritten and none of the additional files created after the last backup are deleted. Shouldn't it be simply
a) boot from a live CD,
b) mount the partitions to be restored,
c) wipe everything on those partitions (via the useful rm -rf#*!§$ you know :) and
d) just extract the tar backup to the mounted partition
to overcome this disadvantage? Or am I missing an important point here?

At the moment, I can't test this idea, so I would be grateful if someone could post their procedure or a few hints on this and why it might or might not work.

[Green_Star]

Concerning your second question:
As a user, the only directory you have write access to is your home directory (apart from some other directories such as /tmp, removable drives under /media and so on). And since you will most likely execute your programs as a user (and not root), the settings of that program can only be saved in your home directory, e.g. .gimp, .mplayer and so on. Even your panel and gnome-specific settings are saved there, e.g. in .gnome2. This is the reason why it is often regarded sufficient to backup your home partition and do a fresh install of your system to keep your settings and files.

However, be advised that this might not hold true if a different release of Ubuntu is installed as some programs and both the way they store their settings (xml instead of a simple text file) and the location where they are stored (e.g. liferea --> liferea_1.4) might have gone through a major change.

Keeping your home directory along with a list of installed packages (e.g. have synaptic spit out a text file with all your packages and use that in a single apt-get command) should often be sufficient as a backup solution if you didn't mess around as root too much.

For convenience though, you can create a first backup of your system using the offered solution.

Now that I think about it. Can't you just install Ubuntu on your VM, copy over your home directory and install whatever package was left out during the installation by using the log created from synaptic? This way, you keep both your programs and settings.

Regards

EDIT: Thanks for the useful hints in this thread. For my personal needs, I prefer the TAR backup solution over all others that are available. There is, however, a disadvantage to it and it has been pointed out in this thread several times: files are only overwritten and none of the additional files created after the last backup are deleted. Shouldn't it be simply
a) boot from a live CD,
b) mount the partitions to be restored,
c) wipe everything on those partitions (via the useful rm -rf#*!§$ you know :) and
d) just extract the tar backup to the mounted partition
to overcome this disadvantage? Or am I missing an important point here?

At the moment, I can't test this idea, so I would be grateful if someone could post their procedure or a few hints on this and why it might or might not work.

That should work, I will try this weekend and let you know. Any way you can also test this in Virtual Box. I do all kinds of experiments in Vbox. Just install ubuntu once in VBox, clone the vdi image and save it for future. In case if you messed up your actual vid then re-clone from the previous clone. This way I avoid lots of time reinstalling ubuntu in VBox.

[162019444]

My ubuntu has broken several times...

Make a mark to learn later.

Thanks a lot

[gnome_mm]

That should work, I will try this weekend and let you know. Any way you can also test this in Virtual Box. I do all kinds of experiments in Vbox. Just install ubuntu once in VBox, clone the vdi image and save it for future. In case if you messed up your actual vid then re-clone from the previous clone. This way I avoid lots of time reinstalling ubuntu in VBox.

I just had the opportunity to try out what I posted above. Here is a rough outline of what I did:

(1) install Ubuntu
(2) perform backup
(3) boot into live CD environment
(4) mount disk to be restored (done automatically usually, see /media)
(5) remove everything on the mounted disk that is to be restored with the contents of the tar file(s) (in this test, the complete disk was wiped clean)
(6) restore
(7) reboot and remove live CD

I didn't experience any problems during or after the restore and the procedure resulted in a system identical to the one just after the backup.

Note: Everything was tested in virtualbox. I assumed a single disk with a single partition in the description above, but it also worked for multiple partitions on my laptop. Just take care of extracting to the right locations and creating the directories excluded in the backup process or you might end up with a non-existant home folder and things like like that.

Conclusion: The backup is really simple and time-saving. The restore is somewhat of a pain if you desire a system identical to the one that was backupped, i.e. without any additional and orphaned files. But then again, one will most likely perform many more backups than restores, so I'm really content with this approach for my home desktop or laptop environment.

[gargamel1]

Hi, I am a complete newbie to ubuntu and the linux world, and I like to know:
if I make a tarbal of all the content on my harddrive with a GUI like xarchive or Archive Manager, and later on, when my system won't boot up anymore. If I then extract the content of that tarball over the files on my harddrive with the GUI interface of the live cd, will the system work again ? because I've read something about permissions which would be incorrect.

Thanks in advance !

[sionghua]

Just one quick question to the original post, won't the restoration only overwrites existing files that are backed up, so new files created after the backup will not be affected, so if the problem relates to new files - eg. a certain configuration file in /home, then this backup method can't solve it?

[gaixixon]

@gargamel1: Yes, it works like a charm. :guitar:
I did more than once restoring my system via liveCD enviroment. if I have my full backup name: fullbak.tgz located right at the /
to restore, I launch liveCD, fire console and issue following command:
first I need to mount my hard drive to (example) /mnt/old_root_system
then
cd /mnt/old_root_system
sudo tar xpzvf fullbak.tgz
and voila.. reboot and enjoy your the-good-old-day :lolflag::lolflag::lolflag:

@sionghua: Sadly it still keeps newer files added after the backup.. I'm trying to figure out how to do.. My current solution is to start live cd and wipe out my old system by rm -Rf /* :guitar:

[mozkill]

What'd be nice to do is:

Hook up USB stick - Reboot - Grub - boots to a clonezilla install - runs a script that creates image and copies to USB stick - when finished reboots to Ubuntu.

Is that possible?


You cant create a bootable Thumbdrive recovery with Clonezilla but you can create a bootable recovery CD. You can also boot with Clonezilla and recover from a backup stored on a Thumbdrive.

Clonezilla rocks. Dont waste time with anything else. Worry about other things.

[kooldino]

So to back up a 500G drive (that's not even full) to a tar file on another 500G drive (both SATA), it took four days with this procedure. What gives?

[scorp123]

So to back up a 500G drive (that's not even full) to a tar file on another 500G drive (both SATA), it took four days with this procedure. Something is definitely broken then in your case.

I just completely wiped my laptop. It has a 250 GB disk. After around two hours and a completely fresh installation of Ubuntu 8.10 I had all my stuff back.

Based on experience I'd say that you need around 30 minutes per 100 GB for making a backup and about twice that if you include the restore (this is assuming that you have fairly modern equipment and enough RAM, e.g. modern IDE and/or SATA disks and at least 1 GB RAM).

You can lose dramatic amounts of time if you try to compress stuff which is already compressed, e.g. packing tons of MP3 music files and compressed videos (XVID, DIVX, MPEG, MP4) into a "tar.gz" archive is a bad idea ... you'd be better off to copy such files straight to your backup volume (e.g. rsync -av --progress /path/to/my/music /path/to/backups/ ).

But four days?? Something is seriously wrong. Sorry if this sounds stupid, but you were not by any chance doing something funny to your computer while it was making backups?? e.g. compressing videos, rendering 3D scenes, running a database with volumes of traffic day and night, compiling stuff, running several BitTorrent downloads, etc. ?? If you take backups it's best if very little or even nothing is running, this greately increases the chances of taking proper snapshots of everything. Plus compressing files needs plenty of CPU power and RAM ... if something else is running which needs plenty of CPU% and RAM too then this will drastically reduce the performance.

The only other thing coming to my mind is that one or both your harddisks are broken or at least not running at full speed (Google "hdparm" and do some tests!) or that your disk controller isn't properly supported and thus not running at full speeds.

Backing up 500 GB should take 4 hours max. .... not 4 days!
.
.

[greenwom]

Well, I have to admit prior to my post that didn't read all 11 pages of this...

I'm going to purchase a Linksys NAS that mirrors two hard drives in it's enclosure.

I want to write a script that will create the backup in a directory on this network drive.

And I want to make this a background event that runs in the middle of the night.

I could even go as far as making a little GUI for this with a restore option or a lightweight live CD with the restore function.

I'm not that worried about my Ubuntu install (I've never had a problem since warty that I didn't create). I more worried about my cheap OLD hardware.

Would I be recreating another NAS backp project?

[scorp123]

Would I be recreating another NAS backp project? Could be.
http://www.freenas.org/

[suffer1989]

Will his work for my problem - Will be able to restore Ubuntu As-is with this method. Even onto a different sized partition ?

http://ubuntuforums.org/showthread.php?t=986711

[tenzinmonlam]

Hi,
Thanks for the great post. I was wondering if your howto applies if I go head and backup my current OS which is Ubuntu 8.4 and restore it on the 8.10. I just configured Conky and others stuffs. I don't want to redo all again.

Thanks..

[steveydoteu]

I would suggest that you back up things such as your conky configuration and then reimpliment them in your installation of 8.10, packages and such will have changed between versions, and things may break.

[padams10001]

I'm currently running 7.04 and since it's no longer supported, I'm looking to upgrade. Would copying and pasting my home folder onto an external drive, and then copying it back into a fresh install work, or do i need to use the methods mentioned in here?

[scorp123]

Would copying and pasting my home folder onto an external drive, and then copying it back into a fresh install work Yes, if you only care about your settings and mails and stuff and don't mind to install and add all (extra?) applications again as needed (which in fact is easy if you know the apps you want ... just open Synaptic and have them installed again).

Just make sure you copy your /home in such a way that it would keep the file permissions intact and not miss out on the "dot" files. Best thing would be if you copied your /home to another drive (e.g. USB harddisk?) containing a Linux file system ... that should make sure that file permissions etc. are carried over correctly and can also be copied back correctly. I'd recommend "rsync" for this, e.g. sudo rsync -av --progress /home /path/to/backup/target/location/

Later, once everything is installed again you can copy your old settings back (this is assuming that you use the same username that you have used before): sudo rsync -av --progress /where/the/backup/is/stored/home/* /home/

[cK-judic]

!@#!@# So this plan sounded great, until I actually tried to restore. Here's my situation. I had a drive die about 2 weeks ago. It was a pity because the only thing I was worried about was my code and that's put on my server's SVN (except the one project I needed most). What's worse is I just didn't have to time buy the external HD and configure everything for backups. So, I've actually RMA'd the HD because it's a 150gig raptor and I'm not about to let that go. In the meantime, I setup my computer on a different SATA drive. I setup weekly cronjobs on both my server and desktop to backup and then move those backup files to the external harddrive.

Here is what my desktop backup looks like which is run from root's crontab:

#!/bin/sh

#backup_tar.sh
#self-explanatory
#backs up the server through tar

DATE=`date '+%m_%d_%y'`

tar cvpzf /home/jax/$DATE.tgz --exclude=/proc --exclude=/lost+found --exclude=/mnt --exclude=/sys --exclude=/media --exclude=/home/jax/mnt --exclude=/home/jax/$DATE.desktop.log --exclude=/home/jax/$DATE.tgz --exclude=/root/.local/share/Trash --exclude=/home/jax/.local/share/Trash / >> /home/jax/$DATE.desktop.log

sudo -u jax mv /home/jax/$DATE.desktop.log /home/jax/mnt/serverdejax/media/backupdrive/backups/logs/
sudo -u jax mv /home/jax/$DATE.tgz /home/jax/mnt/serverdejax/media/backupdrive/backups/systems/jax/desktop/tar/



I just received the new raptor HD in the mail 2 days ago. The restore process does not seem to work for me. I've formatted the HD to ext3, booted with the Live CD and issued the command as represented in the first post. First, I grab the tarball from my external backup drive. Then I do

sudo su
tar xvpfz backup.tgz -C /media/disk/


This process runs smoothly and puts all the files in place. I then created the folders which I skipped in the tar process. Next, I restore grub since the raptor HD is different from the temporary drive:

grub
root (hd0,0)
setup (hd0)
quit


I'm at work and I can't remember exactly what these commands returned, but I do remember something along the lines of "successful." I do know that I was using the incorrect numbers at first and get errors until I used hd0.

Reboot! :( The screen remains at the boot screen for quite some time with the orange loading bar. Then after a minute or so, the screen goes to an initdram prompt which has limited commands available.

So I'm at a loss for what to try next as I belive I've faithfully followed the commands for both backup and restore. Any ideas from someone would be helpful, or you could point out my stupidity at some point in this process. Thanks.

[scorp123]

!@#!@# So this plan sounded great, until I actually tried to restore ...

Reboot! :( The screen remains at the boot screen for quite some time with the orange loading bar. Then after a minute or so, the screen goes to an initdram prompt which has limited commands available. Sounds to me like you forgot to edit /etc/fstab ....

The new disk's UUID is very very likely different from your old UUID. But if you restored your old /etc/fstab the entries in there would still point to your previous disk's ID's ... and thus your OS waits infinitely for your old partitions to show up.

I'd recommend to boot with a live CD, mount your partition(s) and then check the few following things:
sudo vol_id /dev/sda1 => obviously you'd have to replace "/dev/sda1" above with the correct device name of your "/" partition.

And now compare that with your "fstab" file: do the UUID's really match? My guess is they don't and that's why your computer is hanging at boot.

Example output:
> sudo vol_id /dev/sda1
ID_FS_USAGE=filesystem
ID_FS_TYPE=ext3
ID_FS_VERSION=1.0
ID_FS_UUID=c3a47a5e-4321-40a8-92c8-778c766dafc3
ID_FS_UUID_ENC=c3a47a5e-4321-40a8-92c8-778c766dafc3
ID_FS_LABEL=
ID_FS_LABEL_ENC=
ID_FS_LABEL_SAFE=

> cat /etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
# /dev/sda1
UUID=c3a47a5e-4321-40a8-92c8-778c766dafc3 /boot ext3 relatime 0 2
...
... rest removed ...
...


As you can see, /dev/sda1 would be "/boot" in my case. The actual UUID of the disk has to match with what's in /etc/fstab and I guess that's where your system is hanging.

[cK-judic]

Makes sense. I was under the impression that issuing the grub commands would ameliorate that problem. Sounds like a relatively easy thing to fix. Thanks for your help, I'll give that a shot once I get home. Since I've only changed my HD, and you've given me the fix that issue, do you see any other problems with what I've done? Getting everything setup for a development environment is time consuming. I'd rather not spend more time than I have to.

[scorp123]

I'd rather not spend more time than I have to. I use this method (similar if not identical to what's posted here):
http://www.linuxmint.com/forum/viewtopic.php?f=42&t=3969&start=0&st=0&sk=t&sd=a

When I want things to be done as fast as possible and if I don't have to bother about saving the /home folders (e.g. because they're on a remote location anyway) then I skip everything and just backup these things:

/etc/apt/* (so I get to keep my repository data)
The package selections
very few hand-selected config files from /etc (so I don't have to write those config files manually again)
.... And then I only backup and restore my package selections (e.g. very useful for cloning or if I _presto!!_ need a 2nd machine which is identically setup to an existing system):
On the system which will serve as "master image":
sudo dpkg --get-selections > packagelist.txt
and then on the clones:
sudo dpkg --set-selections < packagelist.txt
sudo apt-get dselect-upgrade Result: the computer will automagically download and install the same packages that the "master image" had. The nice thing here is you can do this pretty quickly and with multiple machines if necessary. All machines would then software-wise be identical to the first machine.

I guess this could come in handy for you if you have to test e.g. the same software but with different setups ... or if something goes wrong and you need to reproduce a problem and debug stuff that refuses to work ... and so on.

[cK-judic]

I use this method (similar if not identical to what's posted here):
http://www.linuxmint.com/forum/viewtopic.php?f=42&t=3969&start=0&st=0&sk=t&sd=a


Thanks for the help. I've marked that page so that I can make some changes to my backups. That piece will come in handy when I buy a new laptop for transferring the majority of my current configuration.

[M4rotku]

I just backed up my system so that I can upgrade to 8.10 without worrying.

However, it kinda made me worry at the end of the backup. The last thing that the tar program emitted was "tar: Error exit delayed from previous errors"

Here is the output from when the errors started:

/tmp/.exchange-joey/
/tmp/orbit-joey/
tar: /tmp/orbit-joey/linc-1d41-0-5234111bd5797: socket ignored
tar: /tmp/orbit-joey/linc-241f-0-55dc1e37c996a: socket ignored
tar: /tmp/orbit-joey/linc-1d90-0-7f1bb783193b1: socket ignored
tar: /tmp/orbit-joey/linc-1daf-0-2dc3b39de800b: socket ignored
tar: /tmp/orbit-joey/linc-1d05-0-28ff1d4e70dfc: socket ignored
tar: /tmp/orbit-joey/linc-1d1c-0-331abd201548e: socket ignored
tar: /tmp/orbit-joey/linc-1cda-0-65f1b800e80fa: socket ignored
tar: /tmp/orbit-joey/linc-20c0-0-35728f277fca8: socket ignored
tar: /tmp/orbit-joey/linc-212a-0-fc6cc8dd42e9: socket ignored
tar: /tmp/orbit-joey/linc-1da2-0-1b0626e1c3a0b: socket ignored
tar: /tmp/orbit-joey/linc-24c8-0-3c3105774cf1: socket ignored
tar: /tmp/orbit-joey/linc-1dd1-0-7a951052b2f23: socket ignored
tar: /tmp/orbit-joey/linc-1d6d-0-389765e0a0eb0: socket ignored
tar: /tmp/orbit-joey/linc-1da7-0-490eea964d47c: socket ignored
tar: /tmp/orbit-joey/linc-1cb6-0-5effb66adca85: socket ignored
tar: /tmp/orbit-joey/linc-1a5a-0-5d9cb67da0d30: socket ignored
tar: /tmp/orbit-joey/linc-20c5-0-233f2f52d69f9: socket ignored
tar: /tmp/orbit-joey/linc-1dac-0-1b0626e17871b: socket ignored
tar: /tmp/orbit-joey/linc-1b42-0-681da7c411bdb: socket ignored
/tmp/orbit-joey/bonobo-activation-register.lock
/tmp/orbit-joey/bonobo-activation-server-ior
tar: /tmp/orbit-joey/linc-1a8f-0-7a37e9e026620: socket ignored
tar: /tmp/orbit-joey/linc-1db5-0-1b0626e16c48c: socket ignored
tar: /tmp/orbit-joey/linc-1d11-0-776fe6164c3e3: socket ignored
tar: /tmp/orbit-joey/linc-1d9f-0-276e6c585238a: socket ignored
tar: /tmp/orbit-joey/linc-1cbd-0-3da72e7d73ec9: socket ignored
tar: /tmp/orbit-joey/linc-1d3d-0-18c5cdeae2d32: socket ignored
tar: /tmp/orbit-joey/linc-1d0b-0-2a51ca51d0d86: socket ignored
tar: /tmp/orbit-joey/linc-1d1b-0-438f317338cd4: socket ignored
tar: /tmp/orbit-joey/linc-1da9-0-1b0626e160f98: socket ignored
tar: /tmp/orbit-joey/linc-1a8f-0-3cac1a14c12d1: socket ignored
tar: /tmp/orbit-joey/linc-1cff-0-4874a6d62728e: socket ignored
tar: /tmp/orbit-joey/linc-1cc0-0-3da72e7d7e6b9: socket ignored
tar: /tmp/orbit-joey/linc-1db2-0-f0254744fd9: socket ignored
tar: /tmp/orbit-joey/linc-1d06-0-28ff1d4e72cde: socket ignored
tar: /tmp/orbit-joey/linc-1d3f-0-11193cfbe2d17: socket ignored
/tmp/gconfd-joey/
/tmp/gconfd-joey/lock/
/tmp/gconfd-joey/lock/ior
/tmp/.gdm231RKU
/tmp/.ICE-unix/
tar: /tmp/.ICE-unix/6799: socket ignored
/tmp/virtual-joey.hfbiIm/
/vmlinuz.old
/initrd/
/initrd.img
tar: Error exit delayed from previous errors
joey@joey-laptop:~$


The command that I used to back up the system was:

sudo tar cvpjf pre-upgrade.tar.bz2 --exclude=/proc --exclude=/lost+found --exclude=/home/joey/pre-upgrade.tar.bz2 --exclude=/mnt --exclude=/sys --exclude=/home/joey/Music --exclude=/home/joey/Videos --exclude=/home/joey/Pictures --exclude=/home/joey/ISO\'s --exclude=/home/joey/.VirtualBox/VDI --exclude=/media /

Can anyone tell me if this still created a valid back-up. Also, if I upgrade and things go wrong and I need to go back to 8.04, do I merely restore the backup and then create the extra folders. It seems too easy.

[El_Belgicano]

I just backed up my system so that I can upgrade to 8.10 without worrying.

However, it kinda made me worry at the end of the backup. The last thing that the tar program emitted was "tar: Error exit delayed from previous errors"

Here is the output from when the errors started:



The command that I used to back up the system was:



Can anyone tell me if this still created a valid back-up. Also, if I upgrade and things go wrong and I need to go back to 8.04, do I merely restore the backup and then create the extra folders. It seems too easy.

For the "socket ignored" errors, you don't have to be scared, when backing up I got some, for gnutella, transmission and some 4 others, but the reinstalled system works fine.
About the other errors, I don't know.
But I exclude my tmp folder...

[kooldino]

Something is definitely broken then in your case.

I just completely wiped my laptop. It has a 250 GB disk. After around two hours and a completely fresh installation of Ubuntu 8.10 I had all my stuff back.

Based on experience I'd say that you need around 30 minutes per 100 GB for making a backup and about twice that if you include the restore (this is assuming that you have fairly modern equipment and enough RAM, e.g. modern IDE and/or SATA disks and at least 1 GB RAM).

I'm running a Core 2 Duo @ 3Ghz, 2GB of RAM, and all SATA and IDE HDDs.

You can lose dramatic amounts of time if you try to compress stuff which is already compressed, e.g. packing tons of MP3 music files and compressed videos (XVID, DIVX, MPEG, MP4) into a "tar.gz" archive is a bad idea ... you'd be better off to copy such files straight to your backup volume (e.g. rsync -av --progress /path/to/my/music /path/to/backups/ ).

After the first 12 hours, I killed the operation and got rid of the compression and just had it make an uncompressed tarball, since I figured it was choking on the compressed formats as you mentioned.

But four days?? Something is seriously wrong. Sorry if this sounds stupid, but you were not by any chance doing something funny to your computer while it was making backups?? e.g. compressing videos, rendering 3D scenes, running a database with volumes of traffic day and night, compiling stuff, running several BitTorrent downloads, etc. ?? If you take backups it's best if very little or even nothing is running, this greately increases the chances of taking proper snapshots of everything. Plus compressing files needs plenty of CPU power and RAM ... if something else is running which needs plenty of CPU% and RAM too then this will drastically reduce the performance.

Nope. I was usually not even on the machine, and when I was, I was simply browsing the web.

The only other thing coming to my mind is that one or both your harddisks are broken or at least not running at full speed (Google "hdparm" and do some tests!) or that your disk controller isn't properly supported and thus not running at full speeds.

Started playing with this:

http://ubuntuforums.org/showthread.php?p=6231551#post6231551

Also, is it possible that this is so slow because I'm reading from an NTFS drive and writing to another NTFS drive?

[scorp123]

Also, is it possible that this is so slow because I'm reading from an NTFS drive and writing to another NTFS drive? You should have mentioned that. Yes, this is the most likely reason.

NTFS is a proprietary Microsoft file format not native to Linux and the drivers that enable this compatibility run in so called "user space" (= outside of the kernel's control where processes enjoy higher priviledges) which already makes things a bit slower; plus those drivers probably have to do tons of extra checks just to make sure they don't ruin your NTFS disks. Until a few years ago writing to NTFS disks with Linux would have been a seriously bad idea because the NTFS drivers were too shaky and Microsoft obviously did not want to release any proper documentation ...

Why do you use Linux to backup things from and to NTFS disks? It would be better to leave that to Windows as NTFS is its native filesystem.

[scorp123]

"tar: Error exit delayed from previous errors" Probably harmless if that error appeared at the end. It most likely just means that "tar" hit the end of your files and it therefore quits because it finds it has nothing more to do.

Can anyone tell me if this still created a valid back-up. Why don't you simply test if your backup is good? Unpack a few sample files somewhere and then have md5sum run a check against what you unpacked from the archive and against what's still on the disk (for the manual: man md5sum ... press "q" to quit the manual again). If you get the same checksums then the backup was OK.

[kooldino]

You should have mentioned that.

I honestly forgot until I looked into it.

Yes, this is the most likely reason.[/QUOTE]

I thought it might be.


NTFS is a proprietary Microsoft file format not native to Linux and the drivers that enable this compatibility run in so called "user space" (= outside of the kernel's control where processes enjoy higher priviledges) which already makes things a bit slower; plus those drivers probably have to do tons of extra checks just to make sure they don't ruin your NTFS disks.

Yup.

Until a few years ago writing to NTFS disks with Linux would have been a seriously bad idea because the NTFS drivers were too shaky and Microsoft obviously did not want to release any proper documentation ...

I recall. NTFS support was something I always kept in mind when considering the switch to linux.

Why do you use Linux to backup things from and to NTFS disks? It would be better to leave that to Windows as NTFS is its native filesystem.

Well, here's the thing...I used to run Windows, then I moved to a dual-boot, and now I run strictly linux on this machine. So when I originally set up my data disk, it was set up on a Windows Machine years ago. Ditto for my backup disk.

FAT32 won't do what I need it to do (4GB max file size is not enough for me), so that's not an option. NTFS works, but it's slow under linux. I'd go with ext3, but if I ever need a Windows machine to read the drive, I'd imagine that would be a bit of a pain.

Any advice?

Perhaps I could get away with leaving my data drive as NTFS and making my backup drive ext3? I'd imagine that reading NTFS is a hell of a lot faster than writing it.

[scorp123]

Any advice? I am somewhat in a similar situation .... I have multiple disks (all in the 350 - 500 GB range) and they are all Ext3 ... And now I want them to be XFS. XFS really shines with lots of big files and that's precisely what I am confronted with. Gigabytes of really large files ... going with Ext3 was a mistake in this case. I should have known better. Those disks are filled to the last bit with data, and I have no spare disk to offload the data on to ...

So right now I am considering to buy one or two additional disks ... at least 500 GB, maybe even 1 TB or 2 TB. Fact is that those huge disks cost next to nothing these days.

You can never have too much disk space .... :)

[kooldino]

I am somewhat in a similar situation .... I have multiple disks (all in the 350 - 500 GB range) and they are all Ext3 ... And now I want them to be XFS. XFS really shines with lots of big files and that's precisely what I am confronted with. Gigabytes of really large files ... going with Ext3 was a mistake in this case. I should have known better. Those disks are filled to the last bit with data, and I have no spare disk to offload the data on to ...

So right now I am considering to buy one or two additional disks ... at least 500 GB, maybe even 1 TB or 2 TB. Fact is that those huge disks cost next to nothing these days.

You can never have too much disk space .... :)

I feel your pain. In your case, I'd advise getting a 1TB disk and being done with it.

As for my situation, what do you think I should do? I can completely wipe one of my drives (as it's simply a backup-only disk), so I can reformat it...

[Loan_Refi]

I need assistance with backing up my computer.

I have followed the instructions listed on the 1st page of this thread. I successfully created a backup and restored my computer but when I rebooted my computer I got the following error;

Starting up ...
Loading, please wait...
Check root: bootarg cat /proc/cmdline
or missing modules, devices: cat /proc/modules 1s /dev
ALERT! /dev/disk/by-uuid/4b11a1b-f056-4fb1-a8ea-4649c0d55637 does not exist. Dropping to a shell!

BusyBox v1.1.3 (Debian 1:1.1.3-5ubuntu12) Built-in shell (ash)
Enter 'help' for a list of built-in commands.
(initramfs)

The reason for backing up my data was to replace my existing 20 gig hard drive with a 250 gig on the same computer.
I still have my 20 gig hard drive with my date and is working fine.
I want to avoid having to reconfigure everything by backing up my data. I have several programs & tweaks set up already including VMWare, wine, windows apps etc. and I want to save time and learn to to backup and restore my data to a new hard drive.

Can someone help me with this? I would greatly appreciate it!
Thanks!

[scorp123]

ALERT! /dev/disk/by-uuid/4b11a1b-f056-4fb1-a8ea-4649c0d55637 does not exist. Dropping to a shell! Just follow the last few postings on the one or two last pages before this posting here. Chances are you forgot to change the UUID's. As the name implies they are unique. When you format a disk the UUID changes, but in the backup there will still be records of your previous old UUID. Hence when you restore and forget to adapt your config to the new UUID the boot process will fail because it's waiting to see a disk which according to the config files it is supposed to have but which obviously will never ever show up again (because any formatting etc. will cause your disks to get new UUID's).

[afd]

I created a backup tar file last night just as I'd got a fresh install of Ibex to how I like it (with frustrating things like Wifi all up and running).

I've come to restore the machine today as when I turned it on the Wifi wasn't recognised (not for the first time) and although I restore as per your guide this doesn't fix my wifi. Note that when I did the restore I hadn't performed it from the live disk - instead just straight from the root of the file system while running the installed OS.

I'm not Ubuntu-savvy yet, but from what I understand the /sys folder that gets excluded in the backup process would contain certain things used by/referred to by the OS. If this folder had the drivers in it and I excluded it from the backup then I'm basically screwed again...

At this point I got slightly peaved with the way ubuntu help documentation works. I remember seeing the manual somewhere - and obviously it's best to RTFM before you start bitching - but now I can't see it anywhere!

So far the Ubuntu documentation seems to span these areas;

Ubuntuforums.org
http://manpages.ubuntu.com/
The manual I found previously that came with the install.
The help icon on the top desktop panel.


None of these avenues have allowed me to pose the question of "What does the sys folder get used for?"

If anyone can answer that I'd be grateful. More specifically whether the sys folder has any drivers in it (in relation to the restore not bringing hardware functionality back with it) OR a half-decent walk-through for how to get my STA driver back.

Also, just to stop this happening again, has anyone got an idea as to why my wifi dissappears from time-to-time?

[mo0osah]

I have been trying to backup my system using the instructions but every time I try to make a tar, it says "file changed as we tried to read it" at random files and then it just stops there. I don't even use my system when it is making the tar. How can I fix this?

[scorp123]

and although I restore as per your guide this doesn't fix my wifi. Of course not. WiFi if anything has very little to do with what's in your backup or not.

Note that when I did the restore I hadn't performed it from the live disk - instead just straight from the root of the file system while running the installed OS. Not a very good idea. That's like trying to fix a car while it's driving 130 km/h down the motorway ...

I understand the /sys folder that gets excluded in the backup process would contain certain things used by/referred to by the OS. /sys, /proc and /dev are virtual filesystems. Backing them up is pointless, restoring them is pointless too. The contents of those locations gets regenerated everytime a system boots.

http://en.wikipedia.org/wiki//sys
http://en.wikipedia.org/wiki/Procfs

If this folder had the drivers ... "drivers" is Windows-talk. Here on Linux we have "modules", and they are all located under /lib/modules

I suspect you have flaky hardware and backing up and restoring an entire system isn't going to do much about that.

"What does the sys folder get used for?" And how about entering something simple as "/sys" into Wikipedia?? Or how do you think did I find that URL that I posted above? :D

[scorp123]

every time I try to make a tar, it says "file changed as we tried to read it" at random files and then it just stops there. I don't even use my system when it is making the tar. How old is your hard drive? :confused:

[cygnis1]

Thanks for the howto. It is just what I needed. I upgraded to Ibex, and hated it. I used the recovery disk to go back to Hardy, and an having to reconfigure my laptop. this will be nice to have a completed system already configured if I decide to upgrade again, or install another distro.
Cygnis1

[kooldino]

I feel your pain. In your case, I'd advise getting a 1TB disk and being done with it.

As for my situation, what do you think I should do? I can completely wipe one of my drives (as it's simply a backup-only disk), so I can reformat it...


Well, here's what I did. I left my DATA drive as NTFS and I reformatted my backup drive as ext3. The backup completed in a matter of hours, as predicted.

Looks like reading from the NTFS drive isn't any slower than writing to the ext3 drive, so it all works out well.

[mo0osah]

How old is your hard drive? :confused:

It's not that old. I just got this laptop 8 months ago. Why would an old harddrive have that effect anyway?

[scorp123]

Why would an old harddrive have that effect anyway? Well ... you said you were not doing anything when you created your backup ... and yet "tar" was complaining a lot that files had randomly changed? Obviously something is changing the bits and bytes that are stored on your platters ... If it isn't your OS and any of its processes and if it isn't you ... well, then it is the harddisk itself. You see, harddisks being mechanical things (e.g. they have rotating and moving parts) in nature will fail one day. It's just a question of "when".

If in your case one of the heads or the surface itself is damaged then it's very well possible that the bit patterns of any stored information might flip around all of a sudden and change.

But maybe I am exaggerating ...? Hard to tell as you did not include anything in your posting but a vague description. And to me it sounded like your harddrive had started to fail ...

Would it be possible for you to add more details to your description? e.g. copy and paste the output here of whatever "tar" is saying, and so on? ;)

[scorp123]

I feel your pain. In your case, I'd advise getting a 1TB disk and being done with it. I had to buy a new computer ... my old system started to die. USB is no longer working (seems the controller on the motherboard got fried??), firewire only sees half the capacity of any disk I connect to it (and if I tried to write to such a disk it would totally screw up the filesystem of such a disk ... very funny), and "dmesg" was giving me tons of funny messages I have never ever seen before, not since I started using Linux back in 1996 :D ....

So now I have a brand-new PC from HP, a Pavilion m9480.ch ... Quad-Core CPU's, 8 GB of RAM, 2 x 640 GB harddisks ... plus I bought an external 1 TB disk for my backup needs.

HP also offers those "Personal Media" drives. But basically they're just USB disks and rather expensive. The only real advantage they have is that I could insert them directly into the two special drive bays my new PC has (one for the 2.5" "HP Pocket Media Drive Bay", and the bigger one for the 3.5" "Media Drive" ...). Not really necessary I guess? Any ordinary external USB harddisk costs less and does the same job too. :)