How to do system backup

[tomislav.stasina]

Hi everyone, Ubuntu newbie here!

I installed Ubuntu 20.04.2 LTS and I'm starting to dig OS. I installed programs I need to do some work, mounted additional SSDs, personalized system a bit etc., and now I feel comfortable using it.
I would like to have a local backup in case I mess something up while learning all the stuff.

What are my options for creating system backup and what will backup affect?

Thank you!

[tea for one]

For backup of the Operating System, I use https://clonezilla.org/

For backup of personal data, have a look at grsync (gui for rsync) - available via the repositories.

I'm sure other users will chime in with their alternative suggestions.

[GhX6GZMB]

I use Timeshift for system backup. Hasn't failed me yet, I do a backup before fiddling with something. Also rescued me after a total fail (my fault) and format/reinstall.
Remember to include the hidden files in /home in your backups (tick-box option).

For /home backup I use Back in Time.

[monkeybrain20122]

or fsarchiver, it has many features and can do live backup (i.e when you are using the system providing you don't alter the file system like installing or removing software)
There is a gui called qt-fsarchiver, it is easy to use but has less options then the cli (e.g, it doesn't have the option to exclude directories)

[T6&sfpER35%]

i use luckybackup to just backup my home (to external drive)
and i use timeshift to backup/restore points on my local hard drive

[TheFu]

There are 3 major types of backups

• images
• mirrored files
• versioned files

There are pros and cons for each.

If you are just trying to prevent bonehead mistakes from losing a few files, then the mirrored files could be sufficient and relatively simple. That would be 1 copy only. Use rsync for that. No downtime should be necessary. Restore of 1 file or entire directories is fairly easy. After the first mirror is created, subsequent updates to the mirror are just a few minutes - only the updates are done.

Images use lots of storage and generally require downtime to get a clean backup. They can be restored (the full image, not parts) onto a replacement HDD that is the same size or larger and uses the same block size (512b or 4Kb), but you are stuck with the same hardware. A full restore is 1 command, which can be good at 3am. There isn't any "updating" a backup. Each one is 100% new and takes the same long time to create. If you want 5 copies, then you'll need 5x the storage. With an "image" backup, that is typically created for an entire partition or the entire HDD.

Versioned backups with files are a little more complex, but provide all the capabilities that normal file-based mirrors provide, but are even more efficient with backup times and storage. Specific areas to be backed up and excluded files can make this very efficient.

I don't backup the entire OS. I see little reason to backup the 4-10G of an OS when a fresh install of the core OS is 10-15 minutes. I do backup lists of installed packages, system and personal configuration files and all data (personal and system). The restore steps are basically, do a fresh OS install from a Live-Boot Ubuntu installer, then restore my data, system data, my settings, system settings, and lastly feed the list of manually installed packages into the package manager to be reinstalled. The order matters. At re-install time, applications look for prior settings and prior data. When it sees those, it things - "ah, this is a reinstall, use the existing settings and existing data". In about 30-45 minutes, my system is back and it is MY SYSTEM with my configurations and all my programs just as they were.
Because my restore starts with a new OS install, I can change computers and storage layouts completely. The new system ends up being like my old one. Images can't do that. I've gone from encrypted storage to unencrypted storage. I've gone from a 32-bit OS to a 64-bit OS as part of an upgrade. Once had a laptop stolen in an airport. When I got to the client location, picked up a cheap replacement laptop at the local "big box" electronics store, then restored a backup to it over the internet over night. It was "my system" the next morning.

Additionally, versioned backups help fight not only hardware failures and stupid user tricks (I do something dumb about once a week), but these versions mean a corrupted file gets more time to be noticed. Once had to go back 37 days in my versioned backups to restore a corrupted database. Nobody would keep 37 images. They might keep 4 - say one a week. With malware, versioned backups will make it clear when thousands of files become crypt-locked. Chances are that the size of the backups will become 2x the current size. I get daily reports about system backups and 2x a size increase for most systems would jump out. To be more effective against malware, it would be better to have a different backup server that "pulls" the backups. Just ensure that no clients have direct, unauthenticated, access to the backup server. The backup server would create the connection and pull the files to be backed up. This is a more advanced method.
Https://ubuntuforums.org/showthread....9#post14049979 has a very small backup example that is good for 1 user's HOME directory. It is a start.
For a full system backup, something like:

$ sudo rdiff-backup
       --exclude-special-files \
       --include /usr/local --include /etc --include /home --include /root \
       --exclude '**'        /         /Backups
$ sudo rdiff-backup   --remove-older-than 90d --force /Backups

would handle many needs. Just be certain to put the list of manually installed packages in a text file that gets included in the backups. The 2nd command is what limits the number of versions to 90 days. rdiff-backup will use perhaps, 1.4x the original storage to provide 90days of daily, versioned, backups.

Remove the sudo, put those commands into root's crontab and you can have daily, automatic, versioned, backups for most systems.

Do a little test with just /etc/ in the includes. Do the first run, change a few times in /etc/ and run it again. Go look at what happens in the /Backups directory. The last backup looks like a mirror.

Run

rdiff-backup --list-increments /Backups

and

rdiff-backup  --list-increment-sizes   /Backups

If you have multiple systems to backup, put each into a different directory. I use /Backups/{hostname}/ to keep them all straight.

Restores can be just a simple copy using any tool you like. If I need the most recent backup restored, I can use rsync, but I'll typically use rdiff-backup --restore-as-of now /Backups / . Note how the source is now the backup area and the target is where I want stuff placed?

Of course, the commands can be more complex with all sorts of special included files, excluded file and it might be good to generate some updated system information before each daily backup to be included. Here's what I capture.

# ls /root/backup
apt-mark.auto    cpan.list   crontab.root  gem.list    mcpan.list    lshw.list  new-manual
apt-mark.manual  crontab.thefu  dpkg.list     installed-pkgs  new-auto   parted.txt
lvs    vgs   pvs

I may not need the information, but it could be very important too. Just depends on what the failure was which is causing a restore.

[GhX6GZMB]

or fsarchiver, it has many features and can do live backup (i.e when you are using the system providing you don't alter the file system like installing or removing software)

So if you install or remove software, your backup doesn't work anymore? Seems pretty senseless to me. Timeshift (rsync-based) doesn't have that issue.

[TheFu]

So if you install or remove software, your backup doesn't work anymore? Seems pretty senseless to me. Timeshift (rsync-based) doesn't have that issue.

All backups have a time homogeneity mandate. For image based backups there are ways around this issue - typically people will boot from alternate media to make the images, but if the underlying volume managers supports snapshots, then those can be used as well to freeze the blocks to be backed up and let the system keep running.

No backup solution is perfect. They all have issues, caveats, and all can be corrupted either the entire backup or a single file, if steps to avoid the corruption are not expressly taken. Just because rsync seems to never have a problem, doesn't mean that it doesn't ... for example, rsync with hardlinks for versioning only retains the last owner, group, ACLs and xattrs. If a file was corrupted 37 days ago, but the owner/group were modified just 15 days ago, the restore of the data from 37 days ago will end up with the wrong owner/group. Lots of backup tools make that mistake.

Some backup tools are tied into volume management snapshots, which can be a good thing ... provided the backup tool or the volume manager doesn't become confused.

[GhX6GZMB]

@TheFu:
The original question was simple and from a "newbie".
You are scaring people away here!
We all appreciate your knowledge on this, but really... you can take things too far. Just a friendly hint.

[monkeybrain20122]

So if you install or remove software, your backup doesn't work anymore? Seems pretty senseless to me. Timeshift (rsync-based) doesn't have that issue.

Well yeah you can either manually backup regularly or make a corn job. But the advantage of fsarchiver is that it basically copies the file system along with the meta data but is quite insensitive to the underlying drive configuration and it is a lot more flexible than say, clonzezila, and it is very easy to use and non intrusive so you can continue to read or browse the web when backup is happening in the background (the gui is especially convenient for new users even though it doesn't allow the fine grain control of the cli). restoration is also very simple (might need to reinstall grub though)

I don't know about timeshift but by reading the manual it seems you need to have a dedicate formatted drive for it and I don't see how it can keep track of installing and removing software without gap unless it is constantly working to sync the system. In that case wouldn't that affect performance?

My need for system backup is basically 1) to restore system should a bad update mess up the system so basically you can created snapshots with fsarchiver and restore system to the point before the bad update (which merely making a list of installed apps for reinstalling doesn't help since you'll get the same defective updated version of foo that messes you up in the first place) and 2) making an identical system in another machine say in case of hardware failure. In that clonezilla is picky about disk size and detailed drive configuration but fsarchiver doesn't care, again the flexibility beats other tools, I don't know if timeshift works in that scenario.

In fact I used to "upgrade" my system by creating an installation of the new Ubuntu release in an external hard drive, test it and tweaking configurations to optimise performance and run it for a few months to make sure everything I need is working with no nasty surprises, then I use fsarchiver to make a clone of it and restore it to my internal hard drive. Not sure if timeshift would work for that.