Page 2 of 6 FirstFirst 1234 ... LastLast
Results 11 to 20 of 56

Thread: using rsync for backups

  1. #11
    Join Date
    Nov 2008
    Beans
    50

    Re: using rsync for backups

    Wise men. Very helpful people

  2. #12
    Join Date
    Nov 2008
    Beans
    50

    Re: using rsync for backups

    Quote Originally Posted by LightningCrash View Post
    I can boot an rsync'ed root partition in a VM in minutes. What exactly is horrible about that?

    rsync over to a new directory every time and have fslint dedup with hard links. throw in a copy of sfdisk's output and you have what you need to put it back down.
    (BackupPC would work better though)
    This sounds interesting. Point us in the right direction please. Looks like something i will definately look at as a solution to other stuff

  3. #13
    Join Date
    Jun 2007
    Location
    Oklahoma City, OK
    Beans
    200
    Distro
    Kubuntu 10.04 Lucid Lynx

    Re: using rsync for backups

    Quote Originally Posted by djanie78 View Post
    This sounds interesting. Point us in the right direction please. Looks like something i will definately look at as a solution to other stuff
    I would start with something like LINUXexplo
    http://www.unix-consultants.co.uk/ex...inux-explorer/

    Have it run in cron and copy to NFS or email the results somewhere. The script needs to be modified to exclude process trees in /proc, though.

    Then I would just run BackupPC.

    To restore you will need the explorer output
    1. lvcreate a volume the same size as the original
    2. use your sfdisk output to partition it
    3. use kpartx to map the partitions
    4. mkfs, mkswap on the partitions
    5. mount the partitions in /mnt/restore/ as needed
    6. have BackupPC lay the filesystem back down relative to /mnt/restore
    7. pull the UUIDs of the kpartx partitions, then go into /mnt/restore/etc/ and modify the fstab to reflect the new UUIDs. (or if you're using xen you can use xvda1, xvdb1, etc)
    8. check the networking information in the networking directories
    9. umount the kpartx mapped partitions
    10. use kpartx to unmap the partitions
    11. make a xen config file or make a vmdk file for virtualbox,vmware
    12. boot the VM


    I have scripts that do most of this on my behalf. It's actually very similar to how I provision VMs when I don't use virt-install
    Last edited by LightningCrash; September 23rd, 2010 at 04:27 PM.
    Te audire no possum. Musa sapientum fixa est in aure.

  4. #14
    Join Date
    Jun 2007
    Location
    Oklahoma City, OK
    Beans
    200
    Distro
    Kubuntu 10.04 Lucid Lynx

    Re: using rsync for backups

    I will add that one advantage to using image files (like dd files or qcow2, etc) in production over direct LVM assignment is that you don't have to screw with rsync and kpartx and sfdisk, etc. You just copy the image file and you're pretty good to go.
    You get a lot of duplicated blocks that way, though.
    Te audire no possum. Musa sapientum fixa est in aure.

  5. #15
    Join Date
    Feb 2007
    Location
    Seattle
    Beans
    226
    Distro
    Ubuntu Karmic Koala (testing)

    Re: using rsync for backups

    This is the script I use for home backups, I have this machine and a couple others back up to /var/backup/ on my home file server for quick access. Once it's done with that, it mounts a USB drive, and then dumps /var over to the USB drive, since I want some other stuff in /var dumped over to it. Since I don't want to backup all the binaries, just the configs, I dump out the list of installed packages, so I can easily feed that list back into apt-get and have it reinstall everything, and since I have /etc backed up, I can then easily restore the configs for anything I need.

    Code:
    #!/bin/bash
    /usr/bin/dpkg --get-selections > /etc/installed-software
    /usr/bin/rsync -avrt --exclude 'mtab' /etc /var/backups/hal9000/
    /bin/mount /mnt/backup
    /usr/bin/rsync -avrt --exclude 'cache/apt' --exclude 'cache/apt-cacher-ng/' --exclude 'cache/man'  /var/ /mnt/backup/var/
    /usr/bin/rsync -avrt --exclude 'smb/videos/' --exclude 'smb/downloads' --exclude 'smb/backup/Linux/' --exclude 'smb/backup/images/' /home/ /mnt/backup/
    /bin/umount /mnt/backup
    When I lost the hard disk in my home DNS/SSH/DHCP server, it took me just over an hour to replace the disk, reinstall debian (I use apt-cacher-ng as an apt proxy, so I had a copy of all packages sitting on my server), import the package list, bring over the config files, and get it up and running again.

    Unless you use the --delete flag with rsync, it shouldn't be deleting files in the destination folder that no longer exist in the source folder.

    If anyone is looking for a real managed backup solution, that can fully backup windows and linux clients, plus the OS files, I recommend bacula. I use it at work to backup our servers, but it's way overkill for a few home machines.
    "I was dead long before you were born, and I'll be dead long after you're dead."

  6. #16
    Join Date
    Nov 2008
    Location
    Metro Boston
    Beans
    9,041
    Distro
    Kubuntu 14.04 Trusty Tahr

    Re: using rsync for backups

    Quote Originally Posted by Plecebo View Post
    I think the idea is that rsync can protect you from hardware failure, since you can restore to your most recent rsync, however if you delete a file and run rsync you still don't have a backup of the file.
    Unless you use the one of the various --delete options, rsync won't remove files from the backup directory just because they don't exist on the source.

    Sorry, I just noticed that paulisdead made this point as well.

  7. #17
    Join Date
    Aug 2006
    Location
    Canada
    Beans
    389
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: using rsync for backups

    Quote Originally Posted by SeijiSensei View Post
    Unless you use the one of the various --delete options, rsync won't remove files from the backup directory just because they don't exist on the source.
    What about if you inadvertently change a file (or it gets corrupted/malware etc) and you need to go back to an earlier version?

    "rsnapshot" is another good option that will efficiently keep snapshots using rsync and hardlinks so you can recover older versions of files.

  8. #18
    Join Date
    Nov 2008
    Beans
    50

    Re: using rsync for backups

    Quote Originally Posted by paulisdead View Post
    This is the script I use for home backups, I have this machine and a couple others back up to /var/backup/ on my home file server for quick access. Once it's done with that, it mounts a USB drive, and then dumps /var over to the USB drive, since I want some other stuff in /var dumped over to it. Since I don't want to backup all the binaries, just the configs, I dump out the list of installed packages, so I can easily feed that list back into apt-get and have it reinstall everything, and since I have /etc backed up, I can then easily restore the configs for anything I need.

    Code:
    #!/bin/bash
    /usr/bin/dpkg --get-selections > /etc/installed-software
    /usr/bin/rsync -avrt --exclude 'mtab' /etc /var/backups/hal9000/
    /bin/mount /mnt/backup
    /usr/bin/rsync -avrt --exclude 'cache/apt' --exclude 'cache/apt-cacher-ng/' --exclude 'cache/man'  /var/ /mnt/backup/var/
    /usr/bin/rsync -avrt --exclude 'smb/videos/' --exclude 'smb/downloads' --exclude 'smb/backup/Linux/' --exclude 'smb/backup/images/' /home/ /mnt/backup/
    /bin/umount /mnt/backup
    When I lost the hard disk in my home DNS/SSH/DHCP server, it took me just over an hour to replace the disk, reinstall debian (I use apt-cacher-ng as an apt proxy, so I had a copy of all packages sitting on my server), import the package list, bring over the config files, and get it up and running again.

    Unless you use the --delete flag with rsync, it shouldn't be deleting files in the destination folder that no longer exist in the source folder.

    If anyone is looking for a real managed backup solution, that can fully backup windows and linux clients, plus the OS files, I recommend bacula. I use it at work to backup our servers, but it's way overkill for a few home machines.
    Damn there as some clever people in this joint. You all make the world go round

  9. #19
    Join Date
    Sep 2010
    Beans
    37

    Re: using rsync for backups

    rsync sure seems like the best way to do backups for anyone who can write a script, or even just modify one that somebody else has written. Certainly no more difficult than figuring out all the settings in a black box like Acronis, and then you have a script you understand thoroughly.

    I've got a Python script running as a cron job, keeping a fully-synchronized copy of all my program and data files on an external disk. This script runs every night, and it also makes copies so I can keep snapshots of the system 1, 2, 4, and 8 days ago. Everything is now backed up except the /proc and /sys directories. I'm assuming those can be left as-is when I restore all the old files to a new hard disk with a system built from my original Ubuntu CD. Is that a safe assumption? Is there a better way to do this?

    Here is the core of my script. /media is the external drive.

    Code:
    src = '/'
    excludes = "--exclude '/media' --exclude '/proc' --exclude '/sys'"
    command = "rsync -a --delete %s %s %s"  %  (excludes, src, dest)
    Last edited by david.macquigg; November 6th, 2010 at 01:27 AM. Reason: simplification of script

  10. #20
    Join Date
    Feb 2007
    Location
    Seattle
    Beans
    226
    Distro
    Ubuntu Karmic Koala (testing)

    Re: using rsync for backups

    When you do the restore, the /proc and /sys folders need to exist, but don't need anything in them. They're not actually folders on the disk, but a representation of the running kernel in memory.

    If I've got a full backup including the binaries, or just want to transfer an existing install to another drive, I wouldn't do a full reinstall. In the past, I took a livecd, partition the drive, mount all the partitions into the running livecd's os to somewhere like /mnt/ubuntu. Then I copy all the files back into place. It's very important to make sure your backups preserve permissions, and when you restore that you keep the original permissions. A 'cp -Rp' will do it (note the capital R, so it picks up hidden files/folders), as will 'rsync -ar' work. Once the files are copied in place, you need to fix up the fstab. Use blkid to determine the new UUIDs of the partitions, and then put them into the new fstab. You then need to mount /proc and /dev from the livecd's folders into the copied folder for the next step. You can do it like this 'mount -o bind /proc /mnt/ubuntu/proc'. The final step is to reinstall the bootloader by cd'ing into the new install and chroot into it, 'chroot /mnt/ubuntu /bin/bash' then a grub-setup && grub-install should do it, and you can exit and unmount everything and reboot.
    "I was dead long before you were born, and I'll be dead long after you're dead."

Page 2 of 6 FirstFirst 1234 ... LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •