Backup, Backup, Backup...Please Help

[wyattwhiteeagle]

Only backup your data, since there are copies of Linux at every university and ISP, you don't need to keep one too. Each time you reinstall Linux, it is easier and then you have the latest version. In general with Linux there are no license keys to safeguard, so saving program settings is usually not helpful either.

As for your data, unless you are wealthy enough to have lots of storage, use a differential backup system. You can easily keep three versions of all changed files, for very little more space than a single version. It can be very simple: https://www.aeronetworks.ca/2019/08/...de-simple.html

A "differential backup system"
Thank you for that

I get in one month what you probably get in one week.
I'm nowhere near "wealthy"

Only backup my data. Explain this to me please.
My personal data as in pics songs household records, or my system data which would include custom settings?
Other people are saying whatever I don't want to lose, but that answer doesn't really clarify.

So, why are other people saying that saving their custom settings are getting included into their backups?

https://ubuntuforums.org/showthread.php?t=2441366
Alot of info at that closed thread

You suggested that same link to me in the above closed thread
https://ubuntuforums.org/showthread....1#post13949311

I had some questions in that thread concerning the info at your link
https://ubuntuforums.org/showthread....7#post13949327

You never responded.
You suggested something, and then completely disappeared.
That's not helping, really.

Are you gonna bail on this one too?

[TheFu]

Seems you may be reaching beyond your current skill level. Unix skills build on each other. This is why many people dislike it. It requires knowledge that grows to get to a certain level before all the pieces start clicking. Prior to that point, people's eye gloss over and fairly simple things seem difficult.

I tend to avoid helping beginners because I'm terrible at answering "simple" questions and we both get frustrated. That doesn't help them and it doesn't help me.

You've listed a number of general knowledge items that are part of what I consider basic Linux/Unix skills. This thread is about backups. Herman and I have provided links to fairly simple answers.

If you don't understand Unix permissions, that is the first skill to be learned. It isn't hard and there is a simple elegance to how it works. There's no use avoiding this because you will waste hours, days, weeks, months, years until it is learned. Takes about 45 minutes of concentration to learn 90% of it.

If you don't understand cron and crontabs, that is how things get automated on Unix systems. It isn't hard either. But that topic really belongs in a separate thread ... depending on what you want.

For how Unix system files are laid out, Google "Linux File system hierarchy." There is a wikipedia article with a table. That is pretty much all that 99% of us need to know for which types of files go where. From that table, you'll know what should be backed up. Plus there are lots of threads here where people list what they backup. Typical end-users need $HOME, /etc/ and a list of installed packages. I'd bet that Herman's script gets that and I know that the one you pasted above does.

As for how to read a script ... well ... you actually need to read it. My scripts are commented. If you ask about 1 line you don't understand, someone will probably explain it. For bash scripting, you'll want to understand a little bash. https://tldp.org/LDP/abs/html/ ... this says "Advanced", but it starts from the beginning, so don't be fearful.

For normal, daily use, command line stuff, https://linuxcommand.org/tlcl.php is an easy introduction.

In the beginning, the learning curve is steep.

When we don't answer a specific question, usually that is because it is already answered OR we don't have experience in that area. I've never used timeshift, but I have read about it. It operates in 2 very different modes depending on the file system. Everyone I know who uses it, has another backup tool they need to use for stuff that timeshift doesn't handle. Should we all have 10 backup tools to get 1 system backup? I say no. 1 tool that actually works, is what someone should use.

When I say not to use rsync for backups, that is because I used it for over a decade and found where it works great and where it fails completely. For a remote server running in a VPS, using rsync is probably fine. The number of files and size of the total backup are relatively tiny compared to what a desktop would have. Most of my server backups are ~5G. That's nothing. For a desktop, 50G would be a typical "backup" if not 500G or 5TB. rsync becomes impractical as the number of files increases. There are a number of other reasons rsync isn't the best tool, but everyone has different needs and different skills.

Almost all Linux backup tools are based on librsync, which is the library that rsync uses. I'm not anti-rsync. Heck, I use it 10x almost every day. What I'm against is having to manually do things for backups that rsync isn't good at - like versioning. rdiff-backup does the versioning of backups for us. There is little extra effort needed to have 1 mirror (like rsync does) or 200 versions, which is what rdiff-backup can easily support. The most recent rdiff-backup is just like an rsync mirror, so we keep that simplicity. The second and later backups run much faster with rdiff-backup and use a tiny amount of additional storage, not a full copy of yet another mirror. rdiff-backup isn't hard to script. See above. There are 2 commands. One creates a new backup version and the other trims really old versions off the end. If you only want 3 versions, there's an option to do that, but if your backup storage has room, wouldn't more versions be better? We never know when a file got corrupted until we try to use it. I had a DB corrupted on a system due to a failed software upgrade. Most of the software was working, but not everything. Then I needed to do some once-a-quarter work and found that the file had been corrupted 37 days prior. If I didn't have 90 days of versioned backups, I'd have been screwed. Do you see why more backup sets are useful? What if I'd not noticed for the entire quarter - 90+ days?
And I can't say how many times I do dumb things on my systems and reach for the backup files to fix it. Usually it is a simple data files - like a spreadsheet. I know that every night a backup gets made automatically. Getting the file back from yesterday is just a copy out of the backup storage to the working storage area.

As for testing the restore process ... I use virtual machines for that. You would use another USB stick.

[QIII]

@wyattwhiteeagle

You never responded.
You suggested something, and then completely disappeared.
That's not helping, really.

Are you gonna bail on this one too?

We all get frustrated at times. However, I would suggest that you remember that all of us here are volunteers giving our time as we have it to give. We are not employees of Canonical, we are not paid to be here, we do not work on schedules. People do have other things to do in life. In the case of the thread you cited, oldfred stepped in to answer some of your questions.

Do not demand that specific community members snap to attention and, forsaking their own lives, immediately and fully divert their attention to address your questions. This is a community: If someone has to be elsewhere, another can step in.

Curb your indignation and cool your jets, please.

[GhX6GZMB]

@wyattwhiteeagle

Every user's backup needs are different as most people have pointed out.

Let me describe my scenario and why I backup the way I do as an example.

I use older HP business laptops (~2008, 2009, 2010) because they are cheap, perform extremely well and have all the hardware ports/interfaces that modern laptops miss. The keyboards are excellent, as opposed to the "designer" types today, and the screens are matte, allowing outdoor work. Upgrading RAM (2 GB and 4 GB work well) and using SSD is no issue. It just works. And Lubuntu gives them 4x the performance of the earlier Vista OS. They're really fast

I split my hard drive as follows:

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 223,6G  0 disk 
├─sda1   8:1    0  29,3G  0 part /
├─sda2   8:2    0 188,3G  0 part /home
└─sda3   8:3    0     6G  0 part [SWAP]
sr0     11:0    1  1024M  0 rom

Because I want to be able to reinstall/reconfigure my system without having to worry about my documents/pictures/videos etc. They are all on /home

Now for backup, there are two scenarios in my life:

1: backing up my data in /home. This I do on a weekly/daily basis, depending on activity. For this I use BackInTime, which was designed for this purpose. It can do system backups as well, but that's a bit kludgy.

2: backing up my system. Now, the argument that this is not necessary, beacuse I can reinstall it all again I regard as nonsense. Over time, you do so many installations, tweaks, modifications etc. that a reinstall/reconfiguration of it all will take at least a day. I do system backups when I plan on changing something on the system: new installs, tweaking a setting, fiddling with system files etc. It gives me a good feeling, and has saved my behind several times, when I've done something stupid, even to the point of a complete reinstall.
For this I use Timeshift, which is optimized for this task.

So for me, it's two completely different situations. Using two backup programs may seem quirky ("one or the other can do both jobs"), but it's a question of ease-of-operation. Once set up, each is just a couple of clicks.

Both programs are rsync-based incremental backups, so only the initial backup takes some time, every subsequent one takes a minute or two.

I don't know your situation, but this works for me.

Cheers.

[wyattwhiteeagle]

You never responded.
You suggested something, and then completely disappeared.
That's not helping, really.


Are you gonna bail on this one too?

@wyattwhiteeagle


We all get frustrated at times. However, I would suggest that you remember that all of us here are volunteers giving our time as we have it to give. We are not employees of Canonical, we are not paid to be here, we do not work on schedules. People do have other things to do in life. In the case of the thread you cited, oldfred stepped in to answer some of your questions.


Do not demand that specific community members snap to attention and, forsaking their own lives, immediately and fully divert their attention to address your questions. This is a community: If someone has to be elsewhere, another can step in.


Curb your indignation and cool your jets, please.

I'll curb and cool.

[wyattwhiteeagle]

Thanks to all for the patient guidance. It is very much appreciated.

I believe no matter what I choose, it is only very temporary given my hardware setup and skill level.

128gb usb as my hard drive
2-6gb swap area
4gb RAM

It's looking to me as though my prioritized choices need to be...

Timeshift
BackInTime
Bash Script Backup

Timeshift and BackInTime while learning Bash.

Rsync scripting may become an option, but I fear it will conflict with Bash. Especially during the learning process.

The link within the post here where I was indignant with hot jets presented to me that rsync uses the cloud, I realize now that using the cloud is only an option and not a requirement.
I know the cloud is a good thing, but I'm not a major cloud user.

I tried just opening root's crontabs folder, it said permission denied.
A bit of a revelation to myself that I need to be realistic about my skill level and not try to learn too much too soon.

The 1.0 TB internal hard drive suddenly not working sent me into a panic where all options of recovering, including backup, was catching my attention all at once.

[TheFu]

Back-in-Time does rsync+hardlink backups. I think Timeshift does the same thing on non-BTRFS systems. You don't need both. Back-in-Time can make full and selected backups.

rsync has a target and a source. Where those are located are up to you. If you have a cloud provider that supports rsync, then you can point it there, but it isn't necessary. rsync can be used for 100% local, same computer, backups. I think using cloudy services is a bad thing for anything you don't expect to share with everyone on the internet. "Cloud computing is careless computing." This has been proven true over and over. Don't put anything onto cloudy servers you don't want to share with the entire world.

I tried just opening root's crontabs folder, it said permission denied.
A bit of a revelation to myself that I need to be realistic about my skill level and not try to learn too much too soon.

Probably a good idea, since you shouldn't be using any GUI to attempt this specific thing. I'm afraid to provide any guidance, since you could end up harming your system doing this with your current level of knowledge.

Download a copy of that book I linked above. Ensure you don't skip over the early chapters that explain how different userids run with different euid and egid values. That basic understanding leads into Unix permissions, which is the core of the entire OS security.

[wyattwhiteeagle]

1: backing up my data in /home. This I do on a weekly/daily basis, depending on activity. For this I use BackInTime, which was designed for this purpose. It can do system backups as well, but that's a bit kludgy.

2: backing up my system. Now, the argument that this is not necessary, beacuse I can reinstall it all again I regard as nonsense. Over time, you do so many installations, tweaks, modifications etc. that a reinstall/reconfiguration of it all will take at least a day. I do system backups when I plan on changing something on the system: new installs, tweaking a setting, fiddling with system files etc. It gives me a good feeling, and has saved my behind several times, when I've done something stupid, even to the point of a complete reinstall.
For this I use Timeshift, which is optimized for this task.

So for me, it's two completely different situations. Using two backup programs may seem quirky ("one or the other can do both jobs"), but it's a question of ease-of-operation. Once set up, each is just a couple of clicks.

Both programs are rsync-based incremental backups, so only the initial backup takes some time, every subsequent one takes a minute or two.

I don't know your situation, but this works for me.

Cheers.

Timeshift vs BackInTime

Per the quoted guidance, I will try one and if it doesn't work for my USB install, I will try the other.

I seen a post where Timeshift was able to be configured to do what I'm wanting.
( https://ubuntuforums.org/showthread....1#post14011521 )

I will try Timeshift first.

[HermanAB]

The problem was that you were asking very basic questions about file paths and mounts, which means that you still had a lot to become familiar with. What I see in this thread, is that you are rapidly learning from TheFU and therefore I don’t need to say anything more - you’ll be a graybeard UNIX guru in another week or so! 😱

[GhX6GZMB]

Timeshift vs BackInTime

Per the quoted guidance, I will try one and if it doesn't work for my USB install, I will try the other.

I seen a post where Timeshift was able to be configured to do what I'm wanting.

I will try Timeshift first.

I think they're already in the repository, just run muon to install either.