Making Ubuntu Fast using RAM (updated and simplified)

[terminator14]

About the Project

Back in the summer of 2010, I wrote a HOWTO that can be found here. Later that year, I wrote a script that did the same thing, but was a lot easier to use. With over 55,000 views, and 33 pages of replies (at time of writing), it's apparent that people have been using it. While I've tried to maintain the script by doing quick tests on new versions of Ubuntu, the code has become old, and hard to manage, so I figured a rewrite was in order. Recently, I had some time, so I spent the past month working about 5 hours a day to rewrite this thing, basically from scratch.

Now, the script has cleaner code, should be easier to maintain, and gives the user as many options as possible, while having good default settings that should work for anyone.

What does it do?

So you've probably read this far, but have never heard of my previous HOWTO so you're clueless as to what the script actually does. The basic concept is simple. Many people today have plenty of RAM. Enough so in fact, to load an entire Operating System into it, and run the OS entirely from RAM. Why would you want to do this? Because RAM is fast. Really really fast. To quote myself from my old HOWTO,

Looking at the DDR2 page of wiki, it is clear the advantage RAM has over regular hard drives. Something as cheap as DDR2-533, something you could easily obtain an extra 1 GB of for about $30, has roughly a speed of 4266 MB/s. Compared to a regular 100MB/s hard drive, that's 42 times faster. If you have a slightly more expensive system that supports faster RAM (I have 6GB of DDR2-1066 RAM), the speed of the RAM can be up to 8533 MB/s. Imagine every file your operating system requests to read or write being served up at that speed!

"But doesn't RAM forget everything as soon as you reboot or shut down your computer?" you may ask. This is indeed the case, and normally that would be a deterrent as no one wants an Operating System that they can't save anything to. In Linux however, the separation of the OS from your home folder (where 90% of your data is stored) and the ability to mount partitions in place of folders at boot (with the help of fstab) solves this problem.

Simply put, your desktop, music, photos, videos and downloads are all stored in your /home folder. Even applications such as firefox, transmission, evolution, and many others (most applications actually) store their settings in your /home folder. What the script does is make sure that the /home folder is NOT located on your RAM, but on a partition located on your hard drive, allowing you to save all the files, folders, application settings, and multimedia you want while still getting the benefits of your OS running directly out of RAM. It does this in the background of course, so you never see the difference, and unless you go looking for it, you'd never be able to tell that your /home is not located where the rest of your system is.

Who it's for

If you've been upgrading your computer by buying faster and faster processors and never seem to get the speed you're hoping for, it may be because the processor isn't what's holding your computer back. Anyone who has ever owned or has experienced the speed of a computer running from an SSD can tell you that running your OS from a faster medium makes a huge difference in speed, and in most cases even a bigger difference than buying a faster processor. Unfortunately, SSDs are still rather expensive. Luckily, linux is capable of using RAM as the medium from which the OS runs, and since you already have RAM, and in many cases you already have enough RAM, you can run your computer faster than an SSD for free.

Technically, this method isn't really directly supported by the Ubuntu development team, so if you run into problems, you may have to do some digging online or post a comment and wait for a reply to find out how to solve it. This is something you have to be ready for. I suggest not trying this first in a production environment (if you have a work computer that NEEDS to work 100% of the time, it may not be the best idea to test this on it or you may end up with a problem that prevents you from meeting a deadline). Having said that however, I've been running my OS out of my RAM for months now, and once I got the hang of it, it's as stable as can be.

What it will run on

The old script was designed and tested on Ubuntu 10.04, and later updated for newer versions.

The new script was designed and tested on Ubuntu 14.10, and will be undated for newer versions of Ubuntu when they are released.

The git repo will tell you what versions of Ubuntu the script is available for - just look at the file names.

Manual

Most of the main script is self explanatory as it asks the user questions and says what it will and will not do. The only undocumented feature of the main script is the following command:

sudo ./RAM_booster.sh --uninstall

which, if run at any point, will undo everything the script has done. If the script is interrupted for any reason, it will actually ask you if you want to run the same uninstall method in order to revert the changes it made. This is why, if you run into problems or want to get rid of the RAM Session (which is what I refer to the OS that is created), the line above is your best friend. Although this is the best way to force an uninstall, if your install went fine and you want to uninstall, you can simply run the script again. It will detect that it has run before and offer to uninstall itself for you.

A few lines up, I referred to RAM_booster.sh as the "main" script because it actually creates 3 other scripts and places them into the RAM Session environment. The additional scripts are:

rupdate: Since the RAM Session is copied upon boot from a read only squashfs image, any software you download or updates you perform inside the RAM Session are undone as soon as you reboot. The only way to perform an update is to chroot into the folder to which the OS was copied to create the RAM Session (/var/squashfs), and update from inside that. After this, the squashfs image has to be recreated using /var/squashfs. As this can be a lengthy process, rupdate takes care of all that. All you need to do is run it, wait until it finishes, and reboot to take advantage of the newly installed updates.

redit: rupdate takes care of any updates you may need to do to the RAM Session from the repos, but what if you need to change some settings in /etc/ or download and install a .deb file, or even compile a program from source? redit allows you to chroot into /var/squashfs and edit the RAM Session directly. Any changes you make with redit will be permanent, assuming you let it recreate the squashfs image. Note: people who have used RAM Booster before will know this as the rchroot script.

rupgrade: while rupdate takes care of updates to the system, rupgrade takes care of any updates that I make to RAM Booster, or any accompanying scripts. You can run this script yourselves, but cron already does this for you once per day. If there are updates available, you'll get a popup letting you know to run rupgrade

Video

Yes, there's finally a video showing how to use RAM Booster!

https://www.youtube.com/watch?v=pmqx...ature=youtu.be

Updates in the new version

The new version now has:

Cleaner code
Support for many different Ubuntu partition configurations (except btrfs)
Support for upstart (by enabling proper upstart chroots)
Much better handling of chroot unmounting

It detects and stops any programs running in the chroot, and any filesystems mounted in the chroot before unmounting it

Support for an encrypted filesystem
Support for encrypted home directories
Support for efi
Support for lvm

Development

Original Script:

Although the idea of creating a script to do this seemed pretty simple before I started (I already had the commands the script should run - see Original HOWTO), it ended up being quite complex. The hardest part was probably getting grub2 to work the way I wanted. The script needed to find /home, /boot, identify them for grub using a UUID, and write a grub.d script. This took a bit of work.

For testing, I used Ubuntu 10.04 x64 Desktop (and later Ubuntu 10.10 x64 Desktop), and ran them in VMware. All I can say is thank god for VMWare and snapshots. I literally tested the script a hundred times during development, something I would have never been able to do without VMWare. At one point I had about 20 different snapshots at different points of testing with different hard drive configurations (different locations for partitions that were to hold /boot and /home) and my VM was about 120GB (no joke).

I also used apt-cacher-ng running in another debian VM to make sure that while working on the rupdate script, I wasn't downloading the same updates over and over and over again from the internet. Not only would that have taken a painfully long time, but would have gotten me in trouble with my ISP (I have a monthly limit for downloads that I already exceed with torrents), and would have put unnecessary strain on official update servers.

This took about a week to write, and I'm very curious how I did. If anyone looking at my bash code can improve the code's efficiency or make it work better, or even has a general comment about how bad I am at bash and why, let me know.

Updated Script:

Again, since I already had most of the code, and most of it was working, I figured I would be done in a week or two. One month of coding later, I was finally done. Why is coding never as quick as it seems it should be? Or is that just me?

It was mostly more of the same - lots of vmware testing and commiting to git.

Bugs

If you run into any bugs, big or small, post a reply to this thread and I'll do my best to fix them. Even if it's been months since the last reply, I don't check this thread every day, but do get an email as soon as someone posts, so I'll respond as soon as I can.

If your bug is serious and warrants reproducing, I may ask you to post the /var/log/ram_booster.log logfile from your Original OS so that I can duplicate your install.
The file shouldn't have any private info in it - just your partition table, lvm layout (if you're using that), /etc/fstab, the output of some commands that ran during install, and things like that. Look through the file before you post it to make sure you're ok sharing what's in it.

Final Words

Although I tested the script hundreds of times in VMware when I originally wrote it, and hundreds of times when I rewrote it, the updates for the new versions of Ubuntu get significantly less testing (mostly because most of the time, hardly anything changes). On the off chance that this introduces a bug that gets past me, let me know and I'll do my best to fix it.

Updates/Changes

For a breakdown of the changes made to the scripts over time, see the git log.

Download

In order to get the latest version of the script, first install git:

sudo apt-get install git

Once that's done, run:

git clone git://github.com/terminator14/RAM_booster.git

This will create a RAM_booster folder in your current working directory with the script in it. Keep in mind that there are different versions of the script in the folder that is created - each file is labelled for the version of Ubuntu it will work for. You only need to run the one that corresponds to your version of Ubuntu.

Leadership

I started this project in 2010, when I was still using Ubuntu, and was still going to school, having no job to pay for an SSD. Since then, I've moved on to other distributions, and have bought multiple SSD drives. This has resulted in a steady decrease in my dedication to this project.

I no longer have the will, and often lack the time to keep this project as up-to-date as it should be. While I've been trying to keep this project going by updating it to work on new releases of Ubuntu, and even rewrote it, there is so much more that could be done with it. For example:

• Support for other distros (at the very least - Linux Mint)
• More rigorous bug testing
• Use of LXC instead of chroot
• Added features

If anyone who knows how to write bash scripts (or any other language if you plan on a rewrite), is familiar with git, and is comfortable with virtualization (comes in very handy for testing) is interested in taking over the project, please contact me.

Until I find someone who wants to take the project over, I will continue to answer questions, try to do bug fixes, and try to keep updating it to work with new releases of Ubuntu.

[Sylslay]

Well, thx for script.

It works form me on Ubuntu 10.10.

Got only 1GB ram.

Ubuntu 10.10 was installed on USB key.
Fresh install without update, then connected to internet and su root,
run the script and :

script was perfect, working: (after I unmounted other partiton than sdb!!!, if not it laod file from fat32 and ubuntu 10.04 to squash )

sqash made by it is around 670MB

ubuntu was installed from liveCD on 8gb USB steak.

partiton sdb1: fat32 512 MB
partiton sdb2: ext4 6000MB mount at /
partiton sdb3: ext4 1024MB mount at /realusb
partiton sdb4: swap 512MB

/dev/sdb3 was used to copy home folder by that script.

thx its working , but IMHO I need at list 2GB RAM for bettr results, becosue swap is slower down that beast
give it try at december, when get 2x1GB ram from my parents PC.

thx
PS: My first gnome loaded to ram and it's faster then load normal way from USB

[terminator14]

Well, thx for script.

It works form me on Ubuntu 10.10.

Got only 1GB ram.

Ubuntu 10.10 was installed on USB key.
Fresh install without update, then connected to internet and su root,
run the script and :

script was perfect, working: (after I unmounted other partiton than sdb!!!, if not it laod file from fat32 and ubuntu 10.04 to squash )

sqash made by it is around 670MB

ubuntu was installed from liveCD on 8gb USB steak.

partiton sdb1: fat32 512 MB
partiton sdb2: ext4 6000MB mount at /
partiton sdb3: ext4 1024MB mount at /realusb
partiton sdb4: swap 512MB

/dev/sdb3 was used to copy home folder by that script.

thx its working , but IMHO I need at list 2GB RAM for bettr results, becosue swap is slower down that beast
give it try at december, when get 2x1GB ram from my parents PC.

thx
PS: My first gnome loaded to ram and it's faster then load normal way from USB

Awesome! Good to know it works. Thanks.

[hexion111]

Just was I was looking for. I will give it a try in a few days.

Thanks for posting this.

[terminator14]

Just was I was looking for. I will give it a try in a few days.

Thanks for posting this.

You get a chance to try it? How did it go?

[yesrno]

Hmmm so I tried your script out, but at some point it was asking me to choose between 3 options - where to store the home folder. So I entered a 2, which would be for choosing a disk manually, but when I entered 2 the scripts started doing all kinds of things and at one point I saw it copying my home folder to somewhere (?) and then I saw it was trying to copy all my VM disks and I canceled the script . I ran the un-installer and it says it's un-installed but when I run the script again, it says the only option is to un-uninstall !

[asmoore82]

Linux does this already.

The output of the `free` command tells you how much RAM is currently used by filesystem caching in the "cached" column.

Overbearing caching schemes are a trade-off: trading faster *first run* of any given App for increased, or *slower*, boot times. Subsequent runs of said App will be exactly the same in either case.

See the cache at work on stock Ubuntu 10.04:

ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
1404032+0 records in
1404032+0 records out
718864384 bytes (719 MB) copied, 10.9608 s, 65.6 MB/s
ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
1404032+0 records in
1404032+0 records out
718864384 bytes (719 MB) copied, 0.879604 s, 817 MB/s
ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
1404032+0 records in
1404032+0 records out
718864384 bytes (719 MB) copied, 0.881407 s, 816 MB/s
ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
1404032+0 records in
1404032+0 records out
718864384 bytes (719 MB) copied, 0.861888 s, 834 MB/s
ubuntu$

^repeated access to the same data is nearly instant

The vanilla Linux setup gives you the best of both worlds:
Extremely efficient on-demand caching *plus* at the kernel's discretion
it can still free this memory for use by your apps as needed.

[terminator14]

Linux does this already.

The output of the `free` command tells you how much RAM is currently used by filesystem caching in the "cached" column.

Overbearing caching schemes are a trade-off: trading faster *first run* of any given App for increased, or *slower*, boot times. Subsequent runs of said App will be exactly the same in either case.

See the cache at work on stock Ubuntu 10.04:

ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
1404032+0 records in
1404032+0 records out
718864384 bytes (719 MB) copied, 10.9608 s, 65.6 MB/s
ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
1404032+0 records in
1404032+0 records out
718864384 bytes (719 MB) copied, 0.879604 s, 817 MB/s
ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
1404032+0 records in
1404032+0 records out
718864384 bytes (719 MB) copied, 0.881407 s, 816 MB/s
ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
1404032+0 records in
1404032+0 records out
718864384 bytes (719 MB) copied, 0.861888 s, 834 MB/s
ubuntu$

^repeated access to the same data is nearly instant

The vanilla Linux setup gives you the best of both worlds:
Extremely efficient on-demand caching *plus* at the kernel's discretion it can still free this memory for use by your apps as needed.

I'm sure you're absolutely right and this is indeed correct (nice example too btw - I didn't know the OS would store an entire file like that). What the script does (like you said) is speed up the first time use of programs. So basically, if you just boot your Ubuntu, and open firefox (for example), regular Ubuntu will need to copy data from the HDD to RAM before it runs firefox which will take a few seconds (in my experience, anywhere from nearly instant up to 10 seconds - especially if it's a 5400 RPM drive like some laptops have). If the same scenario happens within the RAM Session after running the script, the data needed for firefox to load is already in RAM. That is the main difference. All subsequent launches of firefox in both cases will be from RAM and should be instant (in theory anyway). Perhaps I should have been more clear in my description of what the script does to speed up the OS when compared to a regular install.

Despite the fact that a regular OS will eventually be as fast as RAM if you use it long enough without a reboot, the fact remains that inside the RAM Session, the system just feels more responsive as applications load instantly every single time. Perhaps this is a temporary benefit as a regular Ubuntu installation would eventually store all the apps you have used in RAM anyway, but to me this felt like a huge difference and was worth it at the time.

You're also right about this system being a trade-off: faster first time run of apps vs slower boot time (as the system needs to copy everything to RAM at boot). I would just add that along with slower boot time, you also get some complexity added. By that I mean - you need to understand how this system works (at least a little) in order to know how to update your RAM Session properly. You also need to understand how to add files to the RAM Session that can survive a reboot, or edit config files in /etc. I think users need to realize this trade-off and make their own decision as to whether or not this trade-off is worth it to them. The best way to make the decision however if a reader of this thread is on the fence is to try the script and see for themselves if the benefits outweigh the trouble.

I used to think they did until I installed my OS onto my SSD. Now my OS is plenty fast, which means I don't use the script anymore as its benefits no longer outweigh its trouble for me. I will however try to fix any bugs that people report the script has.

[Slim Odds]

You could accomplish the same result by simply creating a list of applications that you want to be "first time fast" and just read them from disk in rc.local during startup.

That would put them in the file cache where they would be loaded "instantly" at the expense of reading them from the hard drive at boot.

[terminator14]

You could accomplish the same result by simply creating a list of applications that you want to be "first time fast" and just read them from disk in rc.local during startup.

That would put them in the file cache where they would be loaded "instantly" at the expense of reading them from the hard drive at boot.

Good point. I think this is the basic idea behind preload, but I never could get preload to function properly. I wonder if something as simple as that (reading files in rc.local at boot) would end up with the exact same results in terms of speed of opening apps and speed of the OS itself as this script provides. If so, that would make the issue this script addresses be solved by something 100x simpler, which is always a good thing.

The only potential problem I see with simply reading a file in rc.local (not running it but simply reading it) is that any libraries the file needs when it runs will not be read into RAM. This won't cause any problems - it simply means when the app is launched by the user, the main file will already be in RAM but the libraries it needs will still need to be copied from the HDD to RAM before they can be used. I'm not a programmer, and having limited knowledge of exactly how libraries are linked to apps, I may be wrong about this, but I'm pretty sure this is the case.