[HOWTO] compress /usr: faster safer system, also saves battery life

[synss]

This is not intended for the beginner, it is not difficult to do but will result in a broken install if done wrong.

The stock ubuntu kernel contains both squashfs and aufs, the former is support for a compressed (7z-style compression) read-only file system. The latter is a "union filesystem", it allows to stack directories and we will use it to keep our /usr read-write.

The idea comes from me on the gentoo forums, and other related forum discussion.

Here is what we will do:
compress /usr with squashfs and mount a read-write union on top of it to keep it writable.
It is safer because you keep a read-only backup of /usr
It is faster because searching through the squashfs image is faster than through the actual file system, see benchmarks
It saves battery life because it reduces physical head movements of the hard drive.

Now, if you do not feel comfortable with the command line and do not know what a file system is, this tip is not for you. /usr is crucial to your linux and messing with it is risky. It is risky if you do not know what you are doing. And I do not take responsibility if you break your ubuntu. OK?

Let's go.

Everything is done at the terminal.

Install squashfs-tools and aufs-tools (aufs-tools is not required, though)

sudo aptitude install squashfs-tools aufs-tools

Compress /usr, as root to avoid permission issue

cd ~
sudo mksquashfs /usr /.usr.sqfs -check_data

or

sudo mksquashfs /usr /.usr.sqfs -check_data -info -no-progress

Everything OK?

reboot, when grub starts (a few white lines telling you you can press ESC) press ESC. At the grub prompt, the entry on top should be selected, press e (edit) at the kernel line. Replace "quiet splash" with "S" at the end of the line. Press ENTER and b (boot).
After the kernel has been loaded, it will ask you if you want to "drop to a root shell" and yes, you want, select this option.

The system will finish booting and actually drop you to the root shell.

Move /usr out of the way

cd /
mv /usr /usr-old
mkdir /usr
mkdir -p /var/squashed/{ro,rw}

and edit /etc/fstab

nano -w /etc/fstab

add the following lines

/.usr.sqfs  /var/squashed/ro  squashfs  loop,ro  0 0
aufs  /usr  aufs  br:/var/squashed/rw=rw:/var/squashed/ro=ro  0 0

then you try it

mount -a

any error? you have made a mistake in /etc/fstab

mount | grep /usr

and have a look whether the filesystems are mounted correctly (two lines, one for the squashfs filesystem and one for the aufs union.)

touch /usr/test

if that works too, you should be all set. Reboot. If it does not go till the end, you may want to try to reboot and remove the quiet splash from the grub entry to see what fails.

Once back in gnome or KDE or... if all is working right, you can also remove /usr-old, you should probably make sure everything is mounted correctly again.

You can reconstitute the squashed image from time to time if you install a lot of things and /var/usr becomes big. You need to make it again also when you upgrade to the next version of Ubuntu.

If you are using LVM2, like me, you can even put the squashfs image in its own volume, there is nothing wrong with that. And having root on LVM2 is not a problem for this "how to" either, provided you keep /boot out of LVM.

[takmsdsm]

reboot, when grub starts (a few white lines telling you you can press ESC) press ESC. At the grub prompt, the entry on top should be selected, press e (edit) at the kernel line. Replace "quiet splash" with "S" at the end of the line. Press ENTER and b (boot).
After the kernel has been loaded, it will ask you if you want to "drop to a root shell" and yes, you want, select this option.

The system will finish booting and actually drop you to the root shell.

Adding replacing quiet splash with S causes my system to restart when i press B to boot, and it defaults back to the quiet splash variables when that happens.

This is with 8.04

[synss]

Adding replacing quiet splash with S causes my system to restart when i press B to boot, and it defaults back to the quiet splash variables when that happens.

This is with 8.04

An alternative way to accomplish that is to edit the line directly in the menu.lst file: mount /boot if it is on another partition ; then edit the file /boot/grub/menu.lst in the same manner (replace quiet splash with S) and reboot, after you are done moving /usr, reboot and revert your changes in menu.lst.

But going into single user mode from grub should work...

Also please note that I have changed the mount points in the how to to /var/squashed/rw and /var/squashed/to and that is much better! Please do this change too.

[Jayferd]

I'm running into some weird errors on Karmic UNR. It seems like aufs now needs /usr to be mounted before it will work. When I boot, my /.usr.sqfs gets mounted to /var/squashed/ro, but aufs does absolutely nothing. Same with mount -a. BUT when I first mount the thing to /usr with

mount -o loop /.usr.sqfs /usr

and THEN run mount -a, the contents of the rw folder appear in /usr.

So I tried changing the lines in my fstab to this:

/.usr.sqfs /usr squashfs loop,ro 0 0
aufs /usr aufs br:/var/squashed/rw=rw:/usr=ro 0 0

The idea is that by the time aufs has to do anything, the squashfs stuff is already mounted.

Now mount -a does the trick immediately, but somehow the mount doesn't happen at boot. init borks itself and drops me to a maintenance terminal, where I discover that not only is /usr empty, but also /var/squashed/ro. Even weirder is that as soon as I type mount -a, everything is mounted and happy. Is this specific to Karmic, I wonder, or has there been some change to aufs?

EDIT: I wonder if this has anything to do with this bug?

EDIT EDIT: Even stranger, when I run mountall -v it shows "mount /usr [1011] exited normally" even though the thing is empty.

[Jayferd]

Okay, now every time I boot it empties my fstab. Help please?

[Jayferd]

Wow, guys, the silence is deafening.

I've reinstalled on my eeepc, and I'm experimenting on a VM on my desktop. I've done this 3 times from a fresh install now, and basically have gotten nowhere new.

Basically, aufs refuses to combine the branches unless /usr is already there. Which files does it need? I've copied over com, paste, tee, and diff, plus the entire src directory.

[Labello]

Hi,

very nice guide! thanks for that! but there are two issues i would like to have resolved:
1. when i boot with the kernel-option "S" instead of "quiet splash" and select the entry "root" I am supposed to enter the root-password but as far as i know ubuntu disables the root-user which disables me to enter any password. the password of the administrator also doesn't work. I just did a normal boot, stopped all running services and used TTY1.

2. how do i update the squashed filesystem? do i have got to redo all steps or is the command that creates the filesystem enough? i mean this one:

sudo mksquashfs /usr /.usr.sqfs -check_data

but anyway: very nice and as is can confirm effective guide!

[Penguin Guy]

Compress /usr, as root to avoid permission issue

cd ~
sudo mksquashfs /usr /.usr.sqfs -check_data

or

sudo mksquashfs /usr /.usr.sqfs -check_data -info -no-progress

Why the cd ~, also when you say do this or do this - it would be nice to know the difference.

Perhaps you should get the reader to backup /usr at the start:

It's a good idea to backup our data unless we mess up:

cp /usr /usr~

.....


If everything went okay we can delete our backup:

rm /usr~

If something went wrong we can restore /usr:

mv /usr~ /usr

Other than that, great tutorial!

[Jayferd]

Hey guys, I'm glad this is working for you.

I've tried this about 6 times in a row in VBox, saving different files from /usr, and it consistently breaks. I think it's repeatable:

1. Install a fresh copy of UNR 9.10.
2. Follow the guide.
3. Reboot, it breaks.
4. Or, instead of typing "mount -a", use "mountall", then check out the contents of /usr. It will be empty.

Since this is repeatable on different hardware, I'm finding it hard to believe that I'm the only one experiencing this. Is the guide broken, or is there a secret step that I'm not supposed to know about?

At this point, I'm sort of frustrated, because even if you guys can't/won't help me, I'd at least have liked some acknowledgement that I exist; even just to tell me to go away.

Peace,
--Jay

P.S. I'm also not following the guide blindly; I do know my way around the command line, and I've been searching the web and the forum links provided here for answers. I simply haven't found a good explanation of why "mount -a" works and "mountall" does not.

[dude2425]

I believe the order in which things get mounted is no longer as logical as it was in the past. This guide was written in May 08, and since then we've got mountall and changes to the boot process to make it faster. I've experienced the same problem when adapting a guide for Gentoo to do the same. I think it's just the way mountall parses /etc/fstab or something.

I ultimately decided not to pursue this idea any more due to the fact that I live on the edge and am always updating software. If I kept the stuff required to get squashfs and aufs mounted in /usr, it would never get updated after the squashed image mounts over it. I would have to run an update, resquash everything, and either drop to recovery and run update over, or copy everything from the squashed image that I need back into /usr.

Currently, this is too much of a hassle for me. This is more suited for a system that never needs to receive updates, like gramma's laptop.