Making Ubuntu Insanely Fast using RAM

[terminator14]

Wow, finally got it to work . Had quite a lot of trouble too, but that's very likely because I'm using Jaunty with a self-compiled kernel (2.6.34). Things I ran into:

sudo apt-get install live-initramfs squashfs-tools

This broke apt for me (at least I think that's what started it). Running "sudo dpkg --configure -a", resulted in:

The fix, after some Googling, appeared to be to edit the file
"/usr/share/initramfs-tools/hooks/live", and change the line "copy_exec /usr/bin/udevinfo" into "copy_exec /sbin/udevadm".

When trying to boot into RAM, I was dropped into busybox, with:


Regular Jaunty creates Squashfs filesystem version 3.1. I tried installing the squashfs-tools from Karmic instead and recreated the squashfs image (version 4). That took care of it, but now a new error appeared:

When looking in the live.log, there was a complaint about a missing "wc" command (for busybox). This was fixed by adding a line "copy_exec /usr/bin/wc /bin" to the "/usr/share/initramfs-tools/hooks/live" file. After rebooting, the aufs error still appeared, so I tried installing squashfs-tools from Lucid and going through the process again, but still no luck.

So, back to Google, and it became clear that aufs-support isn't part of the mainline kernel; you'll need to patch it first (standard ubuntu kernels are already patched). So, compiled a kernel with aufs patches applied (it took several times because of kernel version conflicts), compiled the module, added module to /etc/modules (did forget to do that initially), and after going through the howto again, everything actually booted! Yay!

I must say, the speed difference is very noticable. I'm talking about application startup-time, something as simple as scrolling in firefox, etc. Very impressive. Boot time increased though (as expected).

I have a few more things to write (some recommendations as well), but I *really* should go to sleep now. Anyway, thanks again for the howto, I learned a lot.

Ya, playing around with such things can be a bit frustrating at times. Glad to hear it finally worked out for you. Thanks for sharing your experience so people know if they run into the same problems what they can try to fix them.

I'm having issues with my home directory. I created a new partition and added it to the fstab. However whenever I boot into ram it gives me errors saying that I need to create a home directory. I tried to copying my home directory to the new partition to no avail. Am I missing something?

What's your new partition formatted as? ext4/3? What device is it using (like /dev/?). And what does the line look like that you added to /etc/fstab for /home? Also, make sure your RAM session's /etc/fstab is the one that has been modified - if you simply modify your main OS's /etc/fstab and don't update the squashfs image, the RAM session's /etc/fstab won't change. To do this, you can't modify the RAM session's /etc/fstab directly either since it won't save after a reboot - you need to modify /var/squashfs/etc/fstab of your main OS and then rerun the command that creates the squashfs image.

Oh yeah and I forgot to mention in the howto (I will modify it now) that when you are in your main OS, run:

sudo cp -a /home/* /mnt/home/

to copy all your content of /home to the new partition you created. That is assuming you mounted the new partition on /mnt/home - change that to where you mount it.

[happyhamster]

I tested it on a fresh (unmodified) Jaunty installation (only 1.5GB of RAM), and it was much easier this time.

- Installing the necessary tools broke apt as before:

user@test:~$ sudo apt-get install live-initramfs squashfs-tools
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  busybox user-setup
Suggested packages:
  loop-aes-utils curlftpfs genext2fs httpfs2 mtd-tools squashfs-source
  lzma-source
The following NEW packages will be installed:
  busybox live-initramfs squashfs-tools user-setup
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 651kB of archives.
After this operation, 1683kB of additional disk space will be used.
Do you want to continue [Y/n]? 

Get:1 http://nl.archive.ubuntu.com jaunty/universe busybox 1:1.10.2-2ubuntu7 [304kB]
Get:2 http://nl.archive.ubuntu.com jaunty/main user-setup 1.23ubuntu20 [144kB]
Get:3 http://nl.archive.ubuntu.com jaunty/universe live-initramfs 1.154.8-1 [86.8kB]
Get:4 http://nl.archive.ubuntu.com jaunty/main squashfs-tools 1:3.3-7ubuntu2 [117kB]
Fetched 651kB in 0s (1025kB/s)   
Preconfiguring packages ...
Selecting previously deselected package busybox.
(Reading database ... 95750 files and directories currently installed.)
Unpacking busybox (from .../busybox_1%3a1.10.2-2ubuntu7_i386.deb) ...
Selecting previously deselected package user-setup.
Unpacking user-setup (from .../user-setup_1.23ubuntu20_all.deb) ...
Selecting previously deselected package live-initramfs.
Unpacking live-initramfs (from .../live-initramfs_1.154.8-1_all.deb) ...
Selecting previously deselected package squashfs-tools.
Unpacking squashfs-tools (from .../squashfs-tools_1%3a3.3-7ubuntu2_i386.deb) ...
Processing triggers for man-db ...
Processing triggers for initramfs-tools ...
update-initramfs: Generating /boot/initrd.img-2.6.28-19-generic
cpio: ./bin/udevinfo: Cannot stat: No such file or directory
update-initramfs: failed for /boot/initrd.img-2.6.28-19-generic
dpkg: subprocess post-installation script returned error exit status 1
E: Sub-process /usr/bin/dpkg returned an error code (2)

- And as before: changing the line "copy_exec /usr/bin/udevinfo /bin" into "copy_exec /sbin/udevadm /bin" (in the file "/usr/share/initramfs-tools/hooks/live") and then running "sudo dpkg --configure -a" fixed it. I'll see if there's a bugreport on this***.

- When trying to boot into RAM, there was a new error though: the command "du" was missing, which soon caused a zillion other errors. Adding "copy_exec /usr/bin/du /bin" to the file "/usr/share/initramfs-tools/hooks/live" fixed that.
- Same thing for the "wc" command. Rebuild the image again, booted, and it worked. So using a stock kernel does indeed make things easier on Jaunty.

- There were a few things I did not do: /home was already on a separate partition, so I just kept it as it was. And instead of using a device name, I used UUID's in menu.lst, but that's it I guess.

One suggestion:

4. copy your filesystem to temporary directory

Would be nice to add a little warning:

4. Copy your filesystem to temporary directory (this could take a few minutes)

Don't keep staring at the terminal output (especially if using a slow harddrive)! Go get yourself a cup of coffee or such

*** edit: found it: https://bugs.launchpad.net/ubuntu/+s...fs/+bug/343735
So, this appears fixed for Karmic and onwards.

[terminator14]

Installing the necessary tools broke apt as before

I wonder why it does that in Jaunty. It seemed to go ok in Lucid when I did that. I added a note about it to the howto in case anyone else runs into this problem.

One suggestion:

4. copy your filesystem to temporary directory

Would be nice to add a little warning:

4. Copy your filesystem to temporary directory (this could take a few minutes)

Don't keep staring at the terminal output (especially if using a slow harddrive)! Go get yourself a cup of coffee or such

Noted! haha

[CarloMagno]

Terminator14 thanks for this useful guide. I am writing now from my Ram Session in my htpc. I had planned to install a new SSD Drive but with your suggestion it seems I could live without it

I haven't set up a separate /home partition for Ram session (everything is in RAM as it was in the original post) and I will try to check what works and what doesn't (up the moment it seems mythtv frontend refuses to connect to the backend ... could be something to do with the home partition.. I will look for a solution)

Just in case anybody has the same problem I had, it seems that casper and live-initramfs cannot be installed simultaneously (at least from the repos) so you have to uninstall casper if you have it installed in your system to be able to follow Terminator14 instructions, I had it installed because I use Remastersys to backup my system).

Thanks again for the work,

[terminator14]

Terminator14 thanks for this useful guide. I am writing now from my Ram Session in my htpc. I had planned to install a new SSD Drive but with your suggestion it seems I could live without it

Awesome!

I haven't set up a separate /home partition for Ram session (everything is in RAM as it was in the original post)

That works great for a while but if you use your Ubuntu a lot, eventually you're going to want to save files to your desktop and to your home folder and be able to reboot the system without thinking twice about what you will lose. When you do, post back if you run into problems.

Just in case anybody has the same problem I had, it seems that casper and live-initramfs cannot be installed simultaneously (at least from the repos) so you have to uninstall casper if you have it installed in your system to be able to follow Terminator14 instructions, I had it installed because I use Remastersys to backup my system).

Thanks for the tip.

Thanks again for the work

Glad to help.

[mbsullivan]

Nice tutorial and good idea! Make sure to try and keep it as simple as possible!

Now let's get some benchmarks in here! How about running some of the Phoronix Test Suite on both the (1) RAM drive system, as well as the (2) harddrive system, if possible.

To do so:

(1) Install the test suite

sudo aptitude install phoronix-test-suite

(2) Choose the applicable benchmark suites. To see a list, you can run "phoronix-test-suite list-suites". For your purposes, the most important is definitely:

disk - "real-world disk and file-system tests"

However, to get a more well-rounded feel for Ubuntu desktop users, it might be nice to see results from a couple more suites, like:

gaming-free - "all game tests that are open-source"
workstation - "designed to test a system's workstation / server capabilities"

(3) Run all chosen benchmark suites (will also install them). For each chosen suite, run the following command:

phoronix-test-suite benchmark [suite name]

The benchmarking process takes a while, but should give a wealth of valuable info about when this makes a difference.

Mike

PS: Because of caching and the nature of demand paging under Linux, you should reboot right before running the test suite.

[sdennie]

@mbsullivan:

I'm not sure if this is the kind of change you can meaningfully benchmark without some sort of "desktop interactivity" benchmark. In which case, yes, having a fully populated cache is cheating. However, the purpose of this guide is to forcefully create a fully populated "cache" of the filesystem. So, it's not like comparing a fully cached benchmark with an uncached benchmark. You wouldn't be benchmarking apples and oranges. You'd be benchmarking apples that happen to be oranges and oranges.

[mbsullivan]

I'm not sure if this is the kind of change you can meaningfully benchmark without some sort of "desktop interactivity" benchmark. In which case, yes, having a fully populated cache is cheating. However, the purpose of this guide is to forcefully create a fully populated "cache" of the filesystem. So, it's not like comparing a fully cached benchmark with an uncached benchmark. You wouldn't be benchmarking apples and oranges. You'd be benchmarking apples that happen to be oranges and oranges.

That's true. I'd argue, though, that it'd be interesting to see all three: (1) Un-warmed up harddrive, (2) Nicely warmed up harddrive (perhaps through repeated runs of the benchmark), (3) RAM drive.

That'd give a good indication of how much the virtual page cache can approximate the gains bought through the full RAM drive approach.

Mike

[terminator14]

Now let's get some benchmarks in here! How about running some of the Phoronix Test Suite on both the (1) RAM drive system, as well as the (2) harddrive system, if possible.

Despite the apparent questionable usefulness of these tests in this case, I wanted to try them out as I have read a bit about this benchmark suite and was curious about it months ago but never got around to trying it out. I tried to run it on my RAM session with:

$ phoronix-test-suite benchmark disk

and after it downloaded the disk test, all it showed is:

SQLite:
sqlite
Test Run 1 of 21
Expected Trial Run Count: 3
Started Run 1 @ 21:16:51

It sat like that for hours, until I restarted it. Now it's back to the same screen. Is it supposed to take forever or is it stuck? I can't imaging a test involving SQL taking this long.

[sdennie]

That's true. I'd argue, though, that it'd be interesting to see all three: (1) Un-warmed up harddrive, (2) Nicely warmed up harddrive (perhaps through repeated runs of the benchmark), (3) RAM drive.

That'd give a good indication of how much the virtual page cache can approximate the gains bought through the full RAM drive approach.

Mike

*That* would be an excellent test actually.