var directory full, new Ubuntu 22.04 installation, no further apps permitted

[julius-nepos]

It may be more efficient at this point to re-partition and install 22.04 It seems that a separate partition for /usr is very antiquated & has to do with a read-only partition.

It appears that fsck must be run at boot and even after clearing any errors I'll still have the merge problem to deal with. Since a live install is apparently only 999 userid and I guess I need 1000 or more I see no way to get the permissions necessary to merge the partitions. Of course I'll lose my OpenFOAM installation but I can always re-install.

The only concern is how the /usr partition got in there. I wrote down the partitions I was making on a page and it did not include /usr.

[1fallen]

Linux is always a multi-user OS, regardless of what we may believe. We need to treat it like there are 5000 different users on the system.

+1

[TheFu]

It may be more efficient at this point to re-partition and install 22.04 It seems that a separate partition for /usr is very antiquated & has to do with a read-only partition.

Well, the reason for having a /usr separate is from the 1990s when HDD storage was much, much more expensive. Similar operating systems running on similar CPUs would NFS mount the same /usr across many different systems.
Storage is cheap and /usr **is** part of the OS, so it makes sense to have it on the same partition as other static OS files. Linux is changing and it is still unusual for /usr to be in a separate file systems or to be mounted read-only today, though some distros which want to be highly secure do run that way. ChromeOS on chromebooks is one example. ChromeOS is 2 full OS installs - usually an "A" install and a "B" install with ROOT-A and ROOT-B being partitioned separately. Actually, ChromeOS has 11 partitions, last time I checked. About 50% of those are for "A" and the other 50% are for "B". The 1 left over is for user data and gets mounted regardless of which OS is being run (A or B).

It appears that fsck must be run at boot and even after clearing any errors I'll still have the merge problem to deal with.

This is true and not really hard. It is a basic skill necessary for all Linux admins.

Since a live install is apparently only 999 userid and I guess I need 1000 or more .... I see no way to get the permissions necessary to merge the partitions. Of course I'll lose my OpenFOAM installation but I can always re-install.

This is NOT true. It is a misinterpretation. Userid values are just sequential to be unique. There's nothing more implied. Higher isn't better. It is just different. Standard Unix permissions apply. Learn about those and these things become obvious.

I see no way to get the permissions necessary to merge the partitions. Of course I'll lose my OpenFOAM installation but I can always re-install.

I posted the commands AND pointed you at them already, but those commands will not fill in lacking knowledge. Learn the basics of Unix file and directory permissions and all will become clear.

Learning file permissions well takes about 45 minutes. I suspect a fresh install, reinstall of your data and reinstall of any non-standard applications will require hours. Not understanding Unix file permissions will lead to frustration for hours, weeks, months and years until you spend the time/effort to just learn them. Why delay? You have a need, now.

Of course, it is your choice which way to head.

[julius-nepos]

I've spent all day yesterday on this until 11:30 pm at night. Just learned that I can move or merge things as I don't have permission, drives are Read-Only, and I don't have the full syntax for the mount commands. I'll try a bit more but basically this issue has held up all computer operations for days.

I''' do a fsck which apparently needs to be done from a live CD. Or else by putting some fsck file in some boot directlry and then booting normally.

[TheFu]

I've spent all day yesterday on this until 11:30 pm at night. Just learned that I can move or merge things as I don't have permission, drives are Read-Only, and I don't have the full syntax for the mount commands. I'll try a bit more but basically this issue has held up all computer operations for days.

I''' do a fsck which apparently needs to be done from a live CD. Or else by putting some fsck file in some boot directlry and then booting normally.

a) check the mount to see if it is actually read-only. the command is 'mount' and the options for each file system (including fake ones) will be displayed.
"rw" means read-write.
"ro" means read-only.
For example, here's one of mine ... I picked the shortest one

/dev/md2 on /raid2 type ext4 (rw,noatime,errors=remount-ro)

It is mounted rw currently.

b) To provide the best mount commands, we need to know the file system and where you want it mounted .... temporarily and permanently. The command that I'd use for this is:

lsblk -e 7 -o name,type,fstype,size,FSAVAIL,FSUSE%,label,mountpoint

I have an alias for it, so I don't have to type it all the time. There are a number of Linux how-to mount stuff tutorials. For temporary mounts, I'd use the sudo mount .... version. For permanent mounts, I'd setup the /etc/fstab . Lots of tutorials on setting up the /etc/fstab. If I google for "ubuntu mount /etc/fstab", this link is the first result: https://help.ubuntu.com/community/Fstab In general, stuff you find at websites with "ubuntu" in the DNS part of the name are reputable. The DNS part of that link is: "help.ubuntu.com", just to be clear.

c) fsck can only be run on un-mounted file systems. So, for some file systems that aren't mounted, it can be run without anything too special. Point it at the device, not the directory. Using the output from my mount command above, the device is /dev/md2. It is the first part of the fstab lines that do mounts, assuming a device is used and not a UUID or LABEL instead. You can see which file systems are currently mounted using the 'df -Th' command. Using the same md2 example, that line from df is :

Filesystem                               Type  Size  Used Avail Use% Mounted on
/dev/md2                                 ext4  1.3T  384G  810G  33% /raid2

Each of these commands shows some of the same data, but some different data, which can be matched to get a complete picture.

Just because the current userid cannot write to an area on disk, that doesn't mean it is read-only.

The command to be used once /dev/md2 is umounted is:

sudo fsck -y /dev/md2

So ... what to do if the file system you need to run fsck is already mounted? Well, un-mount it. Of course, not all file systems on a running OS can be un-mounted. /usr is an example. This can only be fsck'd by booting into 'Rescue' mode (that's in the advanced Grub menu item at boot), providing a grub boot option either at boot time or by modifying the grub config file (I find these a hassle) or by booting into a Try Ubuntu environment from an Install ISO. That ISO can be a flash drive or optical media. I much prefer flash devices, since they are much faster than any optical drive and with a good flash drive, we can have 20 different ISO files and boot each by selecting in a menu.

In the recovery menu, I think there is an option to run fsck on all file systems. That should work for all Linux file system, but not for vfat/fat32/exfat or NTFS file systems.

Sadly, the old sudo touch /forcefsck method stopped working when systemd took over Linux around 2016. Not all changes are good. This is a perfect example, IMHO. Placing that file at the top level of a mount would automatically force an fsck on the file system at boot and delete the file when completed. It was simple, effective, could be run from halfway around the world and doesn't require a special console device to accomplish like the "new" systemd crap method does for systems not in the same room. It is very frustrating, since we had this standard capability for 20 yrs before it was removed. It was too easy and worked, so they couldn't leave it alone.

[julius-nepos]

Okay, at least I was able to do some reporting. When I mount the SSD as it's whole partition e.g. nvme0n1, I get errors.
When I mount each of it's sub partitions, e.g. nvme0n1p1 it seems to work fine.

sudo fdisk -l

Disk /dev/nvme0n1: 476.96 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: THNSN5512GPUK NVMe TOSHIBA 512GB 

Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 6A0A95FD-90DA-4D2A-B396-15B583BFA4AE


Device             Start        End   Sectors   Size Type

/dev/nvme0n1p1      2048    1968127   1966080   960M EFI System
/dev/nvme0n1p2   1968128  212905983 210937856 100.6G Linux filesystem
/dev/nvme0n1p3 212905984  466812927 253906944 121.1G Linux swap
/dev/nvme0n1p4 466812928 1000214527 533401600 254.4G Linux filesystem


Disk /dev/sda: 5.47 TiB, 6001175126016 bytes, 11721045168 sectors
Disk model: WDC WD6004FZWX-0

Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 1106BD65-71F7-43EE-956C-0706C2192522

Device     Start         End     Sectors  Size Type
/dev/sda1   2048 11721043967 11721041920  5.5T Linux filesystem

Results from this
ubuntu@ubuntu:~$ sudo fsck -y /dev/nvme0n1

fsck from util-linux 2.34

e2fsck 1.45.5 (07-Jan-2020)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/nvme0n1

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>
Found a gpt partition table in /dev/nvme0n1

ubuntu@ubuntu:~$ sudo fsck -y /dev/nvme0n1p1

fsck from util-linux 2.34
fsck.fat 4.1 (2017-01-24)
/dev/nvme0n1p1: 22 files, 13875/245276 clusters


ubuntu@ubuntu:~$ sudo fsck -y /dev/nvme0n1p2

fsck from util-linux 2.34
e2fsck 1.45.5 (07-Jan-2020)
/dev/nvme0n1p2: clean, 56244/6594560 files, 1923900/26367232 blocks

ubuntu@ubuntu:~$ # note p3 is a swap partition
ubuntu@ubuntu:~$ sudo fsck -y /dev/nvme0n1p3

fsck from util-linux 2.34

ubuntu@ubuntu:~$ sudo fsck -y /dev/nvme0n1p4

fsck from util-linux 2.34
e2fsck 1.45.5 (07-Jan-2020)
/dev/nvme0n1p4: clean, 292379/16670720 files, 3617722/66675200 blocks

ubuntu@ubuntu:~$ sudo fsck -y /dev/sda

fsck from util-linux 2.34
e2fsck 1.45.5 (07-Jan-2020)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/sda

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:

    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>
Found a gpt partition table in /dev/sda

ubuntu@ubuntu:~$ sudo fsck -y /dev/sda1

fsck from util-linux2.34
e2fsck 1.45.5 (07-Jan-2020)
/dev/sda1: clean,210380/183144448 files, 14390401/1465130240 blocks

ubuntu@ubuntu:~$ sudo mount -rw /dev/nvme0n1 /mnt/SSD
mount:/mnt/SSD: wrong fs type, bad option, bad superblock on /dev/nvme0n1,missing codepage or helper program, or other error.

ubuntu@ubuntu:~$sudo mount -rw /dev/sda /mnt/HDD
mount: /mnt/HDD: wrong fs type,bad option, bad superblock on /dev/sda, missing codepage or helperprogram, or other error.

All of these mounts seem to work tho:

ubuntu@ubuntu:~$ sudo mount -rw/dev/sda1 /mnt/HDD
ubuntu@ubuntu:~$ sudo mount -rw/dev/nvme0n1p1 /mnt/SSD/p1
ubuntu@ubuntu:~$ sudo mount -rw/dev/nvme0n1p2 /mnt/SSD/p2
ubuntu@ubuntu:~$ sudo mount -rw/dev/nvme0n1p4 /mnt/SSD/p4

ubuntu@ubuntu:~$sudo lsblk -e 7 -oname,type,fstype,size,FSAVAIL,FSUSE%,label,mountp oint
note that sdb is my thumb drive 16Gb flash

NAME        TYPE FSTYPE    SIZE FSAVAIL FSUSE% LABEL                    MOUNTPOINT
sda         disk           5.5T                                         
|_sda1      part ext4      5.5T    5.1T     0%                          /mnt/HDD

sdb         disk          14.6G                                         
|_sdb1      part vfat     14.6G   13.9G     5% JOHN K WO                /mnt/flash
sr0         rom  iso9660   2.6G       0   100% Ubuntu 20.04.1 LTS amd64 /cdrom

nvme0n1     disk           477G                                         
|_nvme0n1p1 part vfat      960M  903.9M     6%                          /mnt/SSD/p1
|_nvme0n1p2 part ext4    100.6G   88.2G     5%                          /mnt/SSD/p2
|_nvme0n1p3 part swap    121.1G                                         
|_nvme0n1p4 part ext4    254.4G  227.8G     4%                          /mnt/SSD/p4

So do I have bad superblocks, whatever those are?

[TheFu]

Running fsck on whole disk device files can be dangerous. Only run it on file systems. That might be a partition or it might be on a logical volume, depending on how you setup things.

/dev/sda is the whole disk. NEVER run fsck on that.
/dev/sdb is the whole disk. NEVER run fsck on that.
/dev/nvme0n1 is the whole disk. NEVER run fsck on that.

Things can end badly.

Also, fsck is a front-end program to the file-system-specific versions of fsck. Not all file systems have an fsck, so that won't work. The name of fsck.{file system type} is the usually convention.

$ fsck.{tab}{tab}
fsck.btrfs   fsck.ext2    fsck.ext4    fsck.fat     fsck.msdos   fsck.xfs     
fsck.cramfs  fsck.ext3    fsck.f2fs    fsck.minix   fsck.vfat    fsck.zfs

I think the btrfs, zfs, vfat/fat, and msdos versions above are likely placeholders to tell you NOT to to it. You may see people using the direct e2fsck program for ext2/3/4 file systems. I've only used that when things are REALLY bad. I can't remember any time when it actually helped.

BTW, whole disk devices cannot be mounted either. Only devices with a file system can be mounted. Now, it is possible to lay down a file system without any partition table on the device. This is a terrible idea. Usually people do it because they are setting up RAID5 or RAID1 arrays. It is a mistake for a number of reasons. Always, always, setup a partition table on any read-write storage device ... er ... except maybe a 1.44MB floppy.

[julius-nepos]

Hi & thanks for the clarification on fsck,

When I ran fsck on the whole disk, I got errors. When I ran it on the filesystem, it seemed to respond with no errors. Except for my SSD boot partition: nvme0n1p1.
This seems to be reported either as ext4 or as FAT32.

nvme0n1     disk           477G                                         
|_nvme0n1p1 part vfat      960M  903.9M     6%                          /mnt/SSD/p1

Running from the regular install, the system seems to work nicely. As you'll recall my only issue was getting rid of the /usr partition and merging or copying it back into /.

BTW, you've put a lot of time into helping me w/ this issue. Is there anything I can do in recompense? Tho not good with Linux, I can help a bit with html, CFD, or numismatics as you can tell from my avatar.

[TheFu]

You having a working solution is all I hope to see.