Copying system setup from on machine to another?

[GhX6GZMB]

I have two HW-wise identical laptops (HDD size and RAM size differ).

I'm trying to make two identical machines SW-wise. One is already running perfectly, the other has a base 20.04 install which boots just fine.
The machines have different UUIDs and users, and that's how it's supposed to be.

My idea was copying the / file system from the running machine to the new one, but excluding the following files:

/etc/default/grub
/etc/fstab
/etc/passwd
/etc/nsswitch.conf

This doesn't work, unfortunately. After doing the copy, login is no longer possible (password not accepted).

Two questions:
1: Is this idea insane?
2: If not, what else do I need to exclude from the copy to make it work?

Thank You.

[oldfred]

You should just need to copy /home.
That has all your user settings.

If you have installed a lot of software you can export list of installed apps and use that to reinstall everything on 2nd system.

My backup includes this also. The dpkg list is a very long text list of all applications and the dependencies.
If upgrading, you may want to edit it to remove obsolete, old kernels or others. It will not re-install anything already installed.
https://help.ubuntu.com/community/Re...ngSamePackages
from lovinglinux - use dpkg to list installed apps
http://ubuntuforums.org/showpost.php...75&postcount=5
http://kevin.vanzonneveld.net/techbl...selectupgrade/

[GhX6GZMB]

Apparently, I expressed myself badly.
I have /home fully under control and I do not want to copy it (it's on a separate partition, BTW).

I want to copy / to a new machine to create an identical software setup, but with other users. A machine that's a copy of the first when it comes to installed programs, setups etc.

[guiverc]

I have no recent experience of any value sorry.

I volunteered at a recycler (https://www.computerbank.org.au/) years ago where we removed drives of donated hardware, dban was run on drives, then the customized Ubuntu was cloned onto the drives and they were just stacked according to drive capacity.

During the 'build' phase of the process a drive of wanted capacity was grabbed, put into the box and the system booted & verified that everything was working as expected (minor things like audio tweaked, boot time stop watched, graphics tested and drivers added if required etc) & once checklist was completed, form was completed, box priced and it was was moved to the 'shop'.

We straight cloned drives...

I do note you mention `/etc/passwd` but not the secondary password file (`/etc/shadow`). Was that an error?, as I can't imagine not ensuring both are in sync. (since you're changing usernames; if you're missing that file that could create problems for you! I forget what the effects are if not in sync, but I'd expect problems)

[GhX6GZMB]

@guiverc, 1000 Thanks.
I was not aware of /etc/shadow and it seems very likely that that's part of my problem.

I'll try that tonight.

[TheFu]

Two questions:
1: Is this idea insane?
2: If not, what else do I need to exclude from the copy to make it work?

1: No.
2: Too many things to list and we'd have to assume all sorts of things that haven't been said which may not be correct. Identical hardware seldom is truly identical, even when contractually mandated during procurement. Versions of chips get modified in the supply chain, for example.

fstab, networking, users, groups will all be different, assuming everything really is identical.

It is not possible to safely clone a running OS. Open files prevent that from working correctly. You can mirror it using an rsync running as root for a first pass of each partition, but you'll need to boot from alternative media, connect 2 storage devices/partitions under that other OS, then re-run rsync (as root again) to get clean copies of all the files that weren't possible to get before.

BTW, when copying files, you'll need to exclude all FIFOs, Queues, Sockets and Named pipe files from the copy. After all, reading from /dev/random will never complete - not ever.

Then you'll need to go back into the target file system, modify the passwd, group, shadow, gshadow files to be consistent. Don't forget to move whatever the old HOME directories in passwd were to the new names. That isn't strictly mandator, but username "joe" would love to cd ~joe and end up in /home/joe, not /home/user1. This won't harm anything except the human's brain. HOME directories don't need to map to the username, humans just prefer that.

Ok, so you can do that stuff and deal with the other little things that will come up, if any.
OR
you can use your backup/restore method to move everything to a new system. I've written about what to backup here 20+ times and I've written what needs to be restored, in what order once. https://ubuntuforums.org/showthread....6#post13757986

There are lots of things possible when we understand the userid/username/uid and groupid/groupname/gid relationships and can manually manipulate those. At restore time, it is really easy to handle those manipulations.

Another key idea is to place system stuff only in /etc/ and personal stuff only in HOME directories. Some tools will put localized settings into /lib/ or /var/lib/ ... which really isn't following the Linux file system hierarchy standards. For example, ufw commands get put into /lib/ufw/, so if you have used those and have them persistent between boots, then you'll need to backup /lib/ufw/ in addition to /etc/. I specifically DON'T use ufw commands directly to avoid that. UFW settings belong in /etc/ufw/ ... where is unimportant to me, since I backup all of /etc/ for my sanity. Same for systemd-unit files that I've customized. I force those to be under /etc/ so backups get them, regardless of where systemd wants them.

As you can see, it is easy for each system to be a little different and the way that each admin actually performs their admin duties can drastically impact which areas need to be backed up. On a desktop, I backup:

my $backup_sources  = "--include /root
--include /etc
--include /var/www
--include /var/log/nginx
--include /var/lib/awstats
--include /usr/local
--include /home";

That is directly from the backup script.

On a VM server, I backup:

my $backup_sources  = "--include /root
--include /etc
--include /usr/local
--include /home";

I don't backup each VM - they are responsible for their own backups. Anyways, you get the idea.

[GhX6GZMB]

First, let me report success!!!

The scenario was to set up a "new" HW-identical laptop as an existing one, with new UUIDs, users etc., but to avoid reinstalling programs and program settings. This sort of ruled out the backup/restore scenario.

I copied the complete / file system fron the existing/running machine onto the new one (freshly installed with a new root user), but excluded the following files:

/etc/fstab/
/etc/passwd
/etc/passwd-
/etc/shadow
/etc/shadow-
/etc/group
/etc/group-
/etc/gshadow
/etc/gshadow-
/etc/subgid
/etc/subgid-
/etc/subuid
/etc/subuid-
/etc/hosts
/etc/hostname
/etc/default/grub
/etc_nsswitch.conf

The reference from @guiverc to the /etc/shadow file was gold, but I also found that the new /etc/group needs to be left undisturbed.

@TheFu: you're the guru here on this kind of thing, but your own setups are so complicated and involved that I get lost. This is no criticism, quite the opposite. Your expertise is needed.
I only have personal stuff and settings under /home. /etc and /usr and such are strictly reserved for the system.
I understand your reference to HW not being always the same, but in this case it's not so.

I Thank You both.

Cheers.

EDIT: I've added a couple of files to my list that should also NOT be copied to the new system (in red).

[scorp123]

... to create an identical software setup ...

Then why not just tell "dpkg" to do that for you? Let the packet manager handle that.

On the original installation you do this:

• copy /etc/apt/sources.list to the target computer ... this file needs to be identical on the target
• copy /etc/apt/sources.list.d/* to the target computer ... if there's anything here. These files also need to be on the target.

Then tell "dpkg" to write down what's installed into a file:

sudo dpkg --get-selections > mypackagelist.txt

Copy that file "mypackagelist.txt" to the target too.

Then, on the target computer, do:

sudo dpkg --set-selections < mypackagelist.txt

The target computer will mark all the packages the original computer had as packages it now wants too.

Then you just trigger the re-installation of everything:

sudo apt-get dselect-upgrade

The result should now be that the target computer has the exact same packages the original computer had, independent of any user accounts that might or might not be present, network configuration, etc. This just "clones" what software was installed on the original and tells a target computer to go and install the same stuff, it doesn't mess with anything else.

[TheFu]

dpkg --get-selections/--set-selections leaves packages inside APT with less-than-great tags for the installed packages. They all get marked as manually installed, so dependencies which would be removed later should another package be deleted from the system won't be removed. Just something to consider.

But apt-mark showmanual can make lists of manually installed packages which can be used similar to the technique provided. By only installing the manually installed packages and letting the dependency engine solve that aspect, we have correct dependencies on the new box. Just something to consider.

[him610]

Something I do; maybe it will help you.
Perhaps you have some unique aliases you have defined in .bash_aliases file. A new installation does not come with .bash_aliases installed in its home directory. You can immediately use your previously defined aliases after you scp the file from your origin system to your clone system.