Page 2 of 4 FirstFirst 1234 LastLast
Results 11 to 20 of 31

Thread: Backup plan

  1. #11
    Join Date
    Feb 2019
    Location
    Virginia
    Beans
    98
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: Backup plan

    I cleaned up my script and it runs without any errors. I have a separate rdiff-backup for each LVM partition. I also have backups of my boot partitions (non-LVM). This is a bare metal setup with OS on SSD.

    In my mind the worst case scenario is my root partition crashes and the system won’t boot up. I want to practice restoring my root partition. My concern is I’ve read that rdiff-backup restore command could wipe my mounted HDD backup drives if not careful, unless I indicate read-only.

    I’m also trying to figure out the best environment to boot up to in order to run the restore. I have looked at SystemRescueCD but they do not integrate rdiff anymore. Perhaps an Ubuntu Server on a USB pendrive is another option. Basically, I don’t feel that I have a test environment or a method figured out yet. If I set up a VirtualBox test environment I am assuming I will need eight partitions matching my current set up in order to properly test? I know I have achieved nothing yet until I have a restore plan...that much I know.

  2. #12
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    19,169
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Backup plan

    I use a standard Ubuntu-Mate desktop install flash drive. Really, any Ubuntu of the same release should be close enough.

    I've already made my suggestions against having separate rdiff-backup commands run for each LV. I think that is a bad idea and complicates things that don't need to be complicated and breaks data homogeneity. Additionally, I wouldn't store stuff to be backed up in /tmp/. Some of that data is sensitive. I put it into /root/backup/. /root has 700 permissions, so non-root users can't see it.

    Don't confuse partitions and LVs. I suggested merging them which would also simplify the restore. As long as the files "appear" to be in the correct places under /, the back-end storage used doesn't matter. You'll just need to fix the fstab as needed. fstab stuff is really easy ... plus you'll have examples from backing up /etc/.

    But it is your system. You do what makes sense to you. I know my restores take 30-45 minutes.
    Last edited by TheFu; August 18th, 2019 at 03:46 PM.

  3. #13
    Join Date
    Feb 2019
    Location
    Virginia
    Beans
    98
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: Backup plan

    I consolidated my logical volumes because in my use case as home server they are not large and more complexity is not needed. I'm not serving all the employees at IBM

    Here's my latest, working script for using rdiff-backup to backup my server OS. I could probably carve out more from my backups but since these are increments they aren't taking up much space. Thanks again for the coding tips and suggestions throughout this thread

    Code:
    #!/bin/bash
    ##########################
    #
    # Creates LVM snapshot & mounts it.
    # Runs rdiff-backup to separate local HDD, then unmounts & removes the snapshot.
    # Removes backups older than 180 days old.
    # Will run using a cron job.
    #
    ##########################
    
    RDIFF=/usr/bin/rdiff-backup
    DAYS=180D
    SRC_TOP=/mnt/snapshots
    TGT_TOP=/media/WD3tbHDD2/backup-serverOS
    
    # backup package lists
    /usr/bin/apt-mark showauto > /root/backup/pkgs_auto.lst
    /usr/bin/apt-mark showmanual > /root/backup/pkgs_manual.lst
    
    ########## ROOT ##########
    /sbin/lvcreate -L6G -s -n root_snap /dev/LVG/root
    sleep 2
    /bin/mount /dev/LVG/root_snap /mnt/snapshots/root_snap
    
    $RDIFF -v5 --print-statistics --exclude-special-files \
      --exclude /mnt/snapshots/root_snap/proc/'**' \
      --exclude /mnt/snapshots/root_snap/dev/'**' \
      --exclude /mnt/snapshots/root_snap/sys/'**' \
      --exclude /mnt/snapshots/root_snap/mnt/snapshots/root_snap/'**' \
      --exclude /mnt/snapshots/root_snap/media/WD3tbHDD1/projects-mount/'**' \
      --exclude /mnt/snapshots/root_snap/media/WD3tbHDD1/media-mount/'**' \
      --exclude /mnt/snapshots/root_snap/media/WD3tbHDD1/cloudy-mount/'**' \
      --exclude /mnt/snapshots/root_snap/media/WD3tbHDD1/dfay-mount/'**' \
      --exclude /mnt/snapshots/root_snap/media/WD3tbHDD2/backup-serverOS/'**' \
      --exclude /mnt/snapshots/root_snap/media/WD3tbHDD2/backup-data/'**' \
      $SRC_TOP/root_snap \
      $TGT_TOP/root
    
    /bin/umount /mnt/snapshots/root_snap
    /sbin/lvremove -f /dev/LVG/root_snap
    sleep 2
    ########## BOOT ##########
    $RDIFF -v4 --print-statistics --exclude-special-files \
      --exclude /boot/efi \
      /boot \
      $TGT_TOP/boot-sda2
    sleep 2
    ########## BOOT-EFI ##########
    $RDIFF -v4 --print-statistics --exclude-special-files \
      /boot/efi \
      $TGT_TOP/boot-efi-sda1
    ########## REMOVE BACKUPS OLDER THAN 180 DAYS OLD ##########
    $RDIFF --remove-older-than $DAYS $TGT_TOP/root
    $RDIFF --remove-older-than $DAYS $TGT_TOP/boot-sda2
    $RDIFF --remove-older-than $DAYS $TGT_TOP/boot-efi-sda1
    ########## END ##########

  4. #14
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    19,169
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Backup plan

    aljames2,

    Just curious, how is this working? Did the restore go as smooth as you'd like?

    A results report would be helpful to others.
    -TF

  5. #15
    Join Date
    Mar 2007
    Location
    Denver, CO
    Beans
    7,832
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Backup plan

    One thing I've started doing, in addition to your scripts is having copies of common things -- like .zshrc, .z10 and a basic installed package list uploaded to a git repository. This isn't really for backup purposes but rather useful when provisioning new machines. I suppose it could be used for backup of some non-sensitive files.

    I prefer borg over rdiff-backup. You can read about the differences in various places:https://www.reddit.com/r/linux/comme..._solution_and/ https://blog.mdda.net/oss/2019/01/09/borg-backup. Its probably most useful if you need encrypted backups. Vorta is a nice GUI that can run borg on the backend, or you can do it the old fashioned way with either scripts and combination of crontabs or systemd timers. I'm really into systemd timers lately so I prefer that method. I don't like splitting things between cron and systemd timers since I forget what I put under which.

  6. #16
    Join Date
    Feb 2019
    Location
    Virginia
    Beans
    98
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: Backup plan

    I am working to figure out a restore sequence. The scenario I’m testing is a complete restore to a separate blank HDD. I get stuck at boot-up giving me the GRUB2 command line. The issue I suspect is the UUIDs are different and therefore the grub.cfg needs updating. I already updated the /etc/fstab with the new UUIDs.

    I tried a Live USB boot and mounted my newly restored system partitions to it at /mnt:
    Code:
    sda1   /boot/efi
    sda2   /boot
    sda3   /
    However, when I tried to install & update grub via chroot /mnt I keep getting failure with chroot not finding /bin/bash when it is really there.

    I was able to rdiff-backup restore my backups to the partitions but of course, it’s not a solution until I get it to boot.

    Still working on it. I feel that I’m close..
    Last edited by aljames2; February 24th, 2020 at 04:44 AM.

  7. #17
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    19,169
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Backup plan

    Which is part of the reason that I just do a fresh install, restore data, /home/, specific system settings (not FSTAB or BOOT anything!), then feed a list of manually installed programs (apt-mark) in the apt to be installed. As the programs install, they see the prior system settings and data, don't touch any of it then bring up each program. For under 15G of data, that is done in 30-45 min from the decision to reinstall. When all done, it has all the data, settings, programs, that was on the prior-to-failure system, so it effectively **is** the same system.

    During the installation, I can change almost any of the hardware or storage layout or add/remove LUKS encryption, since the backups aren't tightly coupled to the storage layout. Most important, I don't care about the /boot/ area at all. The fresh install will address that.
    Last edited by TheFu; February 24th, 2020 at 06:45 AM.

  8. #18
    Join Date
    Mar 2007
    Location
    Denver, CO
    Beans
    7,832
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Backup plan

    Can you post the error exacty?

    I think what you are trying to do is going to work. Once you changed the uuids, did you rebuild the initramfs with mkinitcpio?

    I did something kind of similar to what you are doing -- albeit with arch linux and zfs on root. I was working with snapshots and basically trying to copy the snapshots to a new installation. I really wasn't concerned as you are with backups -- I was trying to use this as a base installation for a VM which then I could go onto modify -- I basically wanted to use this copy several times so I didn't have to reinstall everything from scratch over and over again.

    My link is here:https://bbs.archlinux.org/viewtopic.php?id=252587. It's probably not going to be totally relevant to you however you'll see the steps are similar. You might need to rename the host and might need to modify things for networking since the MAC address of the system is different -- idk.
    Last edited by kevdog; February 24th, 2020 at 06:35 AM.

  9. #19
    Join Date
    Feb 2019
    Location
    Virginia
    Beans
    98
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: Backup plan

    @TheFu, lesson learned. Given that a fresh server install on the new media is somewhat trivial and fast, it doesn’t make sense with my setup to try to glue all the pieces back together to rebuild the server. I will work on an rdiff-backup —include method to backup the essentials including apt-mark references. Thanks!

    Thanks @kevdog, I may keep tinkering with it some in my test environment but I’m losing faith in it...especially from a time investment standpoint. In my case I “only” changed the HDD. If the server blew up and I introduced more than just a new HDD, such as a different motherboard and CPU, the complexity restoring (if possible) from my current backups just multiplied, and so did the risk.
    Last edited by aljames2; February 24th, 2020 at 08:18 PM.

  10. #20
    Join Date
    Mar 2007
    Location
    Denver, CO
    Beans
    7,832
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Backup plan

    FWIW I'm using systemd-boot (aka gummiboot) which intergrates really well with zfs. It really simplifies the boot process. I know you're using grub. I know I'm like making your job a lot more difficult, but it is really useful to be able to do a zfs send/receive on your root partition, put the received partition into a new VM, chroot into installation, install kernels and regenerate initramfs, and boom your good to go.

Page 2 of 4 FirstFirst 1234 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •