Making Ubuntu Insanely Fast using RAM

[mbsullivan]

I have 6GB of RAM. I edited /usr/share/phoronix-test-suite/pts/test-suites/disk.xml to exclude tests that required too much space. This is what I ended up with.

You definitely get blazingly fast numbers on those benchmarks, especially the synthetic ones. Congrats!

Running the benchmark again in the RAM session, this time with 6.8GB of SWAP resulted in the completion of about 3 tests until the compilebench test ate up all the RAM and started using SWAP. The test ran for a while but eventually failed as the RAM ate up any room on the fs to save the test results to. All subsequent tests failed within a few seconds of starting as they had no room on the fs (/root/.phoronix...) to write temp data and results to.

Hmm... That's odd. Isn't the point of the swap space be to prevent the process from freezing when you run out of RAM? Pages should be pushed out to the disk, such that you can run programs as though you have a seemingly infinite memory space.

I'd think it'd be painfully slow when you have to swap, but not freeze.

Mike

[terminator14]

In our case, the filesystem (/) is located in RAM. When the space for the filesystem (saving files) starts to run low (RAM runs out), SWAP does not add to that space (AFAIK). It only replaces RAM when running applications need RAM.

[sdennie]

Preload is broken, doesn't work and is on nobody's fixit list.

And to make your system extra-insanely fast also do this:

write vm.swappiness=1 and vm.vfs_cache_pressure=50 on your /etc/sysctl.conf file

this would drastically reduce swap activity keeping more stuff in Ram.

Sort of. It depends on how you use your machine. Setting swappiness to a very low value can have some initial "feel good" results but, it may not be doing you as many favors as you think. A low swappiness value means that you are preferring basically abandoned RAM to disk cache. Even if you have a fast SSD, your hard drive by far the slowest part of your computer and the page cache does a nice job of hiding that fact. Setting a low swappiness value reduces the kernels ability to hide the fact that your disk is crazy slow.

Yes, setting swappiness to 0 is a known way to increase performance. But, it's not a benchmarked way to increase performance across the board. And at least 1 kernel developer runs with swappiness=100 because he understands what turning that knob means.

I think my signature is very relevant here. Don't try to make something "fast" until you are able to quantify "slow".

[terminator14]

Performing the exact same tests on my regular Ubuntu and comparing them to the previous results, it is clear the advantage running an OS in RAM has. Here are the results:

Regular OS running from HDD
RAM Session

The SQLite test appears in the regular OS benchmark but not in the RAM session presumably because it errored out in the RAM session for some reason (I must have missed the error) and was not included in the results by phoronix-test-suite.

I think my signature is very relevant here. Don't try to make something "fast" until you are able to quantify "slow".

I noticed your signature a while ago and thought it was very insightful and relevant to our topic. I guess I went about it the wrong way in first getting my system to work fast and only now benchmarking the speed of my old system (the regular HDD OS) and the new OS (RAM Session). I should have tested the system thoroughly with similar benchmarks first and focused on improving areas that I was not satisfied with, but I guess it was just easier to say "slow" is what I have now, "fast" is what I want it to be.

[mbsullivan]

In our case, the filesystem (/) is located in RAM. When the space for the filesystem (saving files) starts to run low (RAM runs out), SWAP does not add to that space (AFAIK). It only replaces RAM when running applications need RAM.

Is is possible to make the filesystem be larger than the available amount of RAM? If so, this (combined with playing with vm.swappiness) might make it possible to run the enterprise-level programs in this setup.

Performing the exact same tests on my regular Ubuntu and comparing them to the previous results, it is clear the advantage running an OS in RAM has. Here are the results:

I wonder why unpacking the linux kernel (the only non synthetic benchmark on there) actually got slightly worse on the RAM drive. The difference could be in the noise margin, indicating that the process is simply not disk bound. However, otherwise it makes no sense to me!

Mike

[terminator14]

Is is possible to make the filesystem be larger than the available amount of RAM? If so, this (combined with playing with vm.swappiness) might make it possible to run the enterprise-level programs in this setup.

The only way I know to make that happen is:

1. To extend RAM that applications have to use, add SWAP space
2. To extend fs in our case, figure out the biggest user of space in the fs (like /home) and edit /etc/fstab to mount that part somewhere on a disk. This would slow down operations considerably if you do it to a highly used portion of the fs (like /etc or /usr or something), but it would allow more space if you are running short on RAM.
3. Some sort of lvm setup that combines the storage or RAM and an HDD (I haven't played with lvm much). The problem with this is, even if it is possible, it would cause Linux to have no preference of where to write to and read from (AFAIK). This would mean, in the worst case, you could have totally empty RAM while your disk contained the OS, which would be no better in performance than having a simple OS install to the HDD.

I wonder why unpacking the linux kernel (the only non synthetic benchmark on there) actually got slightly worse on the RAM drive. The difference could be in the noise margin, indicating that the process is simply not disk bound. However, otherwise it makes no sense to me!

That is a weird one. I guess some things are already optimized to use RAM as efficiently as possible in Linux, so much so that putting them completely in RAM doesn't help. Not sure why it would hurt though.

[mbsullivan]

Quote:
Originally Posted by mbsullivan View Post
Is is possible to make the filesystem be larger than the available amount of RAM? If so, this (combined with playing with vm.swappiness) might make it possible to run the enterprise-level programs in this setup.
The only way I know to make that happen is:

1. To extend RAM that applications have to use, add SWAP space
2. To extend fs in our case, figure out the biggest user of space in the fs (like /home) and edit /etc/fstab to mount that part somewhere on a disk. This would slow down operations considerably if you do it to a highly used portion of the fs (like /etc or /usr or something), but it would allow more space if you are running short on RAM.
3. Some sort of lvm setup that combines the storage or RAM and an HDD (I haven't played with lvm much). The problem with this is, even if it is possible, it would cause Linux to have no preference of where to write to and read from (AFAIK). This would mean, in the worst case, you could have totally empty RAM while your disk contained the OS, which would be no better in performance than having a simple OS install to the HDD.

Hmmm... tmpfs can do this, albeit with a performance penalty. As a proof of concept, I created a 5G file in my workstation with 4G of RAM, 50% of which was already full. Note that tmpfs is a dynamically resizing ramdisk, such that the 8G is just a precaution given my situation:

sudo mkdir /media/tmpfstst/
sudo mount tmpfs /media/tmpfstst/ -t tmpfs -o size=8G
dd if=/dev/zero of=bigfile bs=1G count=5

By monitoring vmstat, I could see that it wrote to the swap when it ran out of memory. Unfortunately, it was also obvious that it was doing it in a very inefficient manner (copying only a small amount at a time). All in all, the performance was quite horrendous:

5368709120 bytes (5.4 GB) copied, 1084.94 s, 4.9 MB/s

Normally, I keep my vm.swappiness at 0, since I don't do much with my memory. Increasing it to 90 improved the performance considerably, though it is still a far cry from even a normal harddrive:

5368709120 bytes (5.4 GB) copied, 398.006 s, 13.5 MB/s

I think that the performance could be improved further with more tuning, but it's still likely to be as slow or slower than just a harddrive. My normal harddrive gets about 40MB/s. Also, I'm using a swap file instead of a swap drive, which might adversely impact performance a bit.

The test was meant to be a somewhat extreme case -- as you get closer to the size of the (free) RAM, performance increases considerably. Also, obviously, if the files fit in RAM, then performance is very good (though not as good as with a fixed size ramdisk).

I'd find it interesting if you could create a resizable ramdisk for the directories that might have large files in them (i.e. /home/). This should, I would think, allow the benchmarks to run to completion. It's clear that this makes memory system tuning (vm.swappiness and the like) much more important for the performance, though.

Mike

[terminator14]

I'd find it interesting if you could create a resizable ramdisk for the directories that might have large files in them (i.e. /home/). This should, I would think, allow the benchmarks to run to completion. It's clear that this makes memory system tuning (vm.swappiness and the like) much more important for the performance, though.

For the test to complete, this would need to be done to /root if running the benchmark as root like I am. I think I could use fstab to mount /root as a tmpfs filesystem allowing any overflow to use SWAP. That's a great idea.

Actually, now that I think about it, I don't need to do it at boot so I don't need fstab. Lets see if that works...

[gameguy]

I created a RAM boot thing and it said:
"could not update /home/matthew/.ICEauthority"
"There is a problem with the configuration server.
(/usr/lib/libgconf2-4/gconf-sanity-check-2 exited with status 256)"
"Nautilus could not create the following the required folders:
/home/matthew/Desktop, /home/matthew/.nautilus.
Before running nautilus, please create these folders, or set permissions such that nautilus can create them"
and it won't display panels (probably because nautilus won't load)

I then copied /home to the filesystem, and updated it. I already have a seperate data partition, which loads to /data (works fine on RAM and HD).
Now the last error message isn't displayed.
I can press alt-F2 and run something (only using terminal) or load terminal, and can run nautilus as sudo.
Also, every time I boot, it does the first time boot process (like creating users) because RAM doesn't save it.
And a lot of the config won't load, but I'm pretty sure that all this is to do with nautilus.

[terminator14]

I created a RAM boot thing and it said:
"could not update /home/matthew/.ICEauthority"
"There is a problem with the configuration server.
(/usr/lib/libgconf2-4/gconf-sanity-check-2 exited with status 256)"
"Nautilus could not create the following the required folders:
/home/matthew/Desktop, /home/matthew/.nautilus.
Before running nautilus, please create these folders, or set permissions such that nautilus can create them"
and it won't display panels (probably because nautilus won't load)

I then copied /home to the filesystem, and updated it. I already have a seperate data partition, which loads to /data (works fine on RAM and HD).
Now the last error message isn't displayed.
I can press alt-F2 and run something (only using terminal) or load terminal, and can run nautilus as sudo.
Also, every time I boot, it does the first time boot process (like creating users) because RAM doesn't save it.
And a lot of the config won't load, but I'm pretty sure that all this is to do with nautilus.

Hmm... It sounds like the first time around you didn't have a /home so nautilus had issues storing or retrieving config files. Once you copied your /home over, that was fixed as it now finds the config files it originally created. Even if you have a /data where you store all your data, you still need the /home because many GUI programs like firefox, panel settings, settings of any program you install are all stored in /home/.* which are hidden folders in /home that start with a "." (dot).

So what are the errors you are getting now that you copied /home over? Are you still getting the gconf-sanity-check-2 error?

it appears that this resolves it:

sudo chmod 755 /etc/gconf/gconf.xml.system

Also try:

mkdir -p /etc/gconf/gconf.xml.system; chmod 755 /etc/gconf/gconf.xml.system