Disk cloning

[meanburrito920]

How exactly would B be disk cloning? I want to move all the files (including the operating system, everything) from one hard drive to another hard drive. I'm not so stupid that I can't copy paste files.

The difference is as such: in A, you copy byte for byte every bit of data to the other drive, so it is for all intents and purposes an exact copy of the drive. In B, the files and everything are moved to the other drive, but they are not an 'exact' image. both methods can be used as a full system backup in that you can copy everything back and it will work perfectly. That is a great advantage of linux. you dont need a disk image to recover from, you only need the files. For your case I would suggest option B, because you can tarball (the equivalent of zip) your entire disk to the backup disk, which saves space, plus is fully recoverable from (just unzip it as /).

[TheSlipstream]

Alright, I'll try archiving onto the disk, assuming it's actually possible to do that while the files run, something I know Windows could never do. Should I archive the filesystem, or what? I remind you, I'm not after a backup here. I'm actually planning to use the drive I'm copying into, and format the one I'm copying from. It needs to work as a full disk, as in, you can boot from it.

Edit: Eeek. I guess I never said I was going to use the new drive. Sorry. :/

[Return Privacy]

Hi all,
I also need to do a clone of my hard drive. 1=1, bit for bit, whatever you want to call it. It is running Ubuntu 8.04. It is working fine, but the drive is starting to make noise. I want to clone it to another hard drive before it fails, and then just switch the hard drives out. I am new to Ubuntu, but have done this same thing in windoze machines using ghost before.

I have the system drive connected by ide cable to mobo. I can put the "new" drive that I want to put the clone on, on the secondary ide plug on the same cable. Or I can plug the "new" drive on the secondary ide plug on the cable that goes to the dvd/cdr burner.

This ubuntu setup is running well, with a lot of customization, and I don't want to loose it. I didn't set it up.
Thank you in advance for you help.

[ByteJuggler]

OK, if you want to use/boot the other (cloned) disk as a direct replacement for the original, then you do need to make a bit for bit clone. I'll explain the process below, but you may also reference this thread for background.

Note however as a word of warning, that if your source disk is having any kinds of issues (e.g. having trouble reading certain parts of the disk) then the below procedure may run into trouble/may take substantially longer than normal, as the cloning process will read every byte on the disk, whether its used or not. Note also that, if your OS is still booting that does not neccesarily mean that the disk free from having problems. If your source disk does have significant problems impeding the below process, then an alternate process using a tool called "dd_rescue" would be needed, which I'll not cover now unless it turns out to be needed.

OK, so on to the process. Ideally you should perform this operation while booted from a LiveCD as you don't want open files on the disk being cloned causing an inconsistent state/problems on the clone.

So firstly you have to identify which is the source disk and which is the target disk. You can usually identify this using the output from e.g.

sudo fdisk -lu

The result should be two disk device identifiers, e.g. firstly the source drive, say (for IDE):

/dev/hda

or (SATA)

/dev/sda

And secondly the desination drive (probably empty with no partitions) e.g. :

/dev/hdb

or (SATA)

/dev/sdb

This is only an example, if your current source disk is not the first disk in the system then the device identifier for the source will obviously be something else e.g. /dev/hdb or /dev/sdb etc.

Once you've established which is the source drive and which is the target drive, you can then proceed to cloning the disk using the "dd" command as follows:

sudo dd if=/dev/hda of=/dev/hdb

The "if" above stands for "Input file" and the "of" above stands for "Output File." It means that "dd" will read from "if" and write to "of", thus copying the entire disk from one to the other. So, you need to substitute "/dev/hda" above with your source/original disk being cloned, and "/dev/hdb" with the target where you're cloning to.

A word of warning: If you get the above mixed up/the wrong way round, you will destroy your existing data! Be sure to have backups of anything you can't afford to lose before you start this, and double check you've got the above command exactly right before you issue it!!!

Note: The command will run for a substantial while, copying the entire disk with no feedback about what its doing. This is normal. You can stop the copy if you want by pressing ctrl-c.

If you'd like some feedback while the copy happens, you can instead perform the copy as follows:

sudo -s
dd if=/dev/hda of=/dev/hdb & pid=$!
while kill -USR1 $pid; do sleep 1; done

The above version of the dd command will put the copy into the background, and the following "while" line will then send a signal to the background copy to ask it report progress once every second. To stop the monitoring you may press ctrl-c, and then to stop the background copy you can then issue "fg" to bring the background copy job to the foreground, followed by ctrl-c again to stop it.

Finally, after copying, exit the root shell (started on the first line with the "sudo -s") by entering:

exit

This is very important, to prevent accidental damage by the use of the root shell. (I used it above since the & symbol does not work well in conjunction with sudo on the same command.)

Once the copy has finished, you may shut down the machine, remove the original and place the clone drive in the place of the original. You should find you can now boot off the clone as though it's the original disk.

Edit: After successful cloning, you can at your leisure resize the partitions on the cloned disk to make use of the full space on the target disk, or you can of course just create a new partition to make use of any space left over after the copy.

[TheSlipstream]

Hm. After some testing, I'm nearly 100% sure that my External Drive Holder is defective. I'm getting a new one, and when I do, I'll make sure to post if I was successful with your method. Have a gold star for effort.

[Return Privacy]

Bytejuggler,
I followed your post tonight. I used the commands to clone to a brand new WD 320 gb ide hd. The original is a WD 40 gb ide hd. It cloned with no problems, and I typed exit when it was done. I removed the live cd, and put in the new drive.
It attempted to start and it says DISK BOOT FAILURE, INSTALL SYSTEM DISK AND PRESS ENTER.
I tried several times.
Like I said this new drive is right out of the box, I didn't do anything but clone to it. Is there some way I was supposed to prepare it before cloning?

[ByteJuggler]

Bytejuggler,
I followed your post tonight. I used the commands to clone to a brand new WD 320 gb ide hd. The original is a WD 40 gb ide hd. It cloned with no problems, and I typed exit when it was done. I removed the live cd, and put in the new drive.
It attempted to start and it says DISK BOOT FAILURE, INSTALL SYSTEM DISK AND PRESS ENTER.
I tried several times.
Like I said this new drive is right out of the box, I didn't do anything but clone to it. Is there some way I was supposed to prepare it before cloning?

No, doing a disk-to-disk clone copies *everything*, including partition table/master boot record and all the partitions on the original disk. After that, it's generally putting it in the place of the original, which includes checking/setting up the BIOS boot order etc. It sounds like either the clone operation was done somehow incorrectly somewhere or your BIOS boot order is somehow wrong currently. How long did the copy process take?

Can you again connect both drives, then boot up with an Ubuntu LiveCD, then from a terminal window post the output of the following command:

sudo fdisk -lu

So we can see what partitions currently exist on all the disks.

If you don't know how to copy text from a terminal, then you may use the following commands instead, to get the text into an easy to copy from text editor. Enter the following command into a terminal:

sudo fdisk -lu >/tmp/fdisk.txt; gedit /tmp/fdisk.txt

[Return Privacy]

Hi bytejuggler,
Thanks for responding quickly. I tried connecting both drives, and booting from the live cd. It boots, but the fdisk -lu brings up nothing. I then disconnected the "new" drive, and connected only the "old" drive and let it boot normally. I got the following from the fdisk -lu command. I attached the text file.
When I connected only the "new" drive and booted from the live cd, the fdisk -lu does nothing. So, I guess the cloning didn't work. One strange thing though. I noticed when I booted from the "old" drive, it now has what appears to be another drive on the desktop that is named "media 2.0gb".
That was never there before. And I think it explains partially what happened? When I did the cloning, it said it completed and it had cloned 2.1gb. I had just assumed that was the size of the actual ubuntu install from the "old" drive and it cloned it to the new drive?
Remember when I first started to clone, you had me do a command that showed the two drives as /dev/hda (40gb) and /dev/hdb (320gb). I even wrote it down. When I did the cloning I specified
if=/dev/hda and of=/dev/hdb, and it did do the cloning and finished? Maybe you can figure it out more than I can. I just know that the new drive doesn't show up at all when connected by itself and booting from the live cd? Oh, and if it matters, I did jumper the "new" drive as slave when I connected it to the "secondary" plug on the ide cable when I did the cloning. Now when I try connecting just the "new" drive I have tried with jumper, and without, as a master, and it doesn't show up. Hope this all makes sense to you, I'm stumped..

[ByteJuggler]

OK, something odd is then going on. OK firstly, when you had both disks connected, did you run the "fdisk -lu" with "sudo"? I ask, because fdisk behaves like (producing no or sometimes incomplete [depending on permissions] output) when you run it without "sudo" so I'm wondering if perhaps you left out the sudo that time. Just a guess. Anyway, when you have both disks connected, you really should see entries for both disks, even if the new disk contains no partition table yet, in the output of "sudo fdisk -lu".

Also note for reference, because a cloning operation copies the *entire* disk regardless of what's on it, it should report that 40GB was copied if everything goes according to plan. So, clearly the copy didn't work as planned for some (at present) unkown reason. I'm guessing the problem crept in during your identification phase somehow. I notice the file you attached reference /dev/sda (which is normally indicative of SATA disks), whereas your post references /dev/hda, /dev/hdb and jumpers on a hard disk, which implies IDE disks. So, please clarify exactly where you're copying from and to (and the disk types.)

Anyway, I suggest we use an alternate method to do the disk identification for source/target: Use "gparted", the partition editor. Normally it's accesible from System->Administration->Partition Editor, if it's installed. If it's not installed, then first install it, for example using Synaptic. Go System->Administration->Synaptic Package manager, then click "Search", enter "gparted", click it and click "Mark for Installation", then click "Apply" and wait for it to be installed. (You can even do this while booted from the liveCD if needed, can't offhand remember if it has it installed or not.)

Once you go into the Partition editor, it will list a screen something like this:

Notice on the Gparted menu, there a submenu listing all the disks (devices) in the system. Those are the identifiers you need for the cloning process. In my system snapshotted above, there's a 320GB disk and an 8GB USB key (showing up as slightly less due to the differences between how manufacturers measure a GB vs. how a computer does.)

So, get back to me on the drive id's and then we can try the cloning again.

[Return Privacy]

Hi again bytejuggler,
When I did the fdisk -lu, I did use sudo in front of it. And I tried it several times to make sure. With both the "old" and "new" drives, booted from the Live cd, it returned nothing at all. Not even the "old" drive information?
But, remember when I first connected the "new" drive and the old and new drives and did the sudo fdisk -lu, the old drive is /dev/hda 40gb, and the new drive is /dev/hdb 320gb.
This IS correct, I checked it and wrote it down. (this was before I did the clone)
Now, when I try to do the gpart with the machine booted from the Live cd, it returns nothing at all. It is blank. This is with both drives connected to the ide cable.
So, I had to boot the machine with the old original drive connected, and with the new drive connected with the jumper on slave. Yes, to answer your question, the drives are both ide drives, not sata.
Here is the result of the gpart scan window like you told me to do. (it took me a while to figure out how to do a screen capture) I just don't know how to paste it in here, so I'll attach the image.
As you may have assumed, I paid a guy to set up this ubuntu system on this original drive. It works well, but I want to clone this to the new drive. And I want to learn to do this myself. I really do appreciate all your help in this.