Page 1 of 29 12311 ... LastLast
Results 1 to 10 of 283

Thread: Making Ubuntu Fast using RAM (updated and simplified)

  1. #1
    Join Date
    May 2008
    Location
    Edmonton, Canada
    Beans
    Hidden!
    Distro
    Ubuntu

    Making Ubuntu Fast using RAM (updated and simplified)

    About the Project

    As some of you may know, a few months back I wrote a HOWTO that can be found here. With over 10,000 views, I can only assume that this was something quite a few people were interested in. With the number of comments being significantly less than 10,000 however, I assume the majority of those views are people who take one look at the HOWTO, see how complicated it looks, and run for the hills. This is why, as promised in the comments of that thread, I have been planning to rewrite it to make it simpler for people to use.

    Although rewriting it would have given it an update to work with the newly released Ubuntu 10.10, and perhaps made it simpler and more organized, there would still be a need for the user to run all those commands, and the problem of the HOWTO seeming complex, which would deter people from even reading it would remain. This is why I have decided to combine my knowledge and love of bash scripting with my knowledge of this project to create a script that was as bulletproof and helpful as possible.

    By bulletproof, I mean the script has exit codes being read like the bible. Every 2 lines or so you'll find an exit code being read of a command that was just run and the script acting accordingly. This is to make sure that even a beginner who is interested in speeding up their machine in this way would be as comfortable using this script as an expert would.

    The script also gives the user as many options as possible, while having good default settings that should work for anyone.

    What does it do?

    So you've probably read this far, but have never heard of my previous HOWTO so you're clueless as to what the script actually does. The basic concept is simple. Many people today have plenty of RAM. Enough so in fact, to load an entire Operating System into it, and run the OS entirely from RAM. Why would you want to do this? Because RAM is fast. Really really fast. To quote myself from my old HOWTO,

    Quote Originally Posted by terminator14 View Post
    Looking at the DDR2 page of wiki, it is clear the advantage RAM has over regular hard drives. Something as cheap as DDR2-533, something you could easily obtain an extra 1 GB of for about $30, has roughly a speed of 4266 MB/s. Compared to a regular 100MB/s hard drive, that's 42 times faster. If you have a slightly more expensive system that supports faster RAM (I have 6GB of DDR2-1066 RAM), the speed of the RAM can be up to 8533 MB/s. Imagine every file your operating system requests to read or write being served up at that speed!
    "But doesn't RAM forget everything as soon as you reboot or shut down your computer?" you may ask? This is indeed the case, and normally that would be a deterrent as no one wants an Operating System that they can't save anything to. In Linux however, the separation of the OS from your home folder (where 90% of your data is stored) and the ability to mount partitions in place of folders at boot (with the help of fstab) solves this problem.

    Simply put, your desktop, music, photos, videos and downloads are all stored in your /home folder. Even applications such as firefox, transmission, evolution, and many others (most applications actually) store their settings in your /home folder. What the script does is it makes sure the /home folder is NOT located on your RAM, but on a partition located on your hard drive, allowing you to save all the files, folders, application settings, and multimedia you want while still getting the benefits of your OS running directly out of RAM. It does this in the background of course, so you never see the difference, and unless you go looking for it, you'd never be able to tell that your /home is not located where the rest of your system is.

    Who it's for

    If you've been upgrading your computer by buying faster and faster processors and never seem to get the speed you're hoping for, it may be because the processor isn't what's holding your computer back. Anyone who has ever owned or has experienced the speed of a computer running from an SSD can tell you that running your OS from a faster medium makes a huge difference in speed, and in most cases even a bigger difference than buying a faster processor. Unfortunately, SSDs are still rather expensive. Luckily, linux is capable of using RAM as the medium from which the OS runs, and since you already have RAM, and in many cases you already have enough RAM, you can run your computer faster than an SSD for free.

    Technically, this method isn't really directly supported by the Ubuntu development team, so if you run into problems, you may have to do some digging online or post a comment and wait for a reply to find out how to solve it. This is something you have to be ready for. I suggest not trying this first in a production environment (if you have a work computer that NEEDS to work 100% of the time, it may not be the best idea to test this on it or you may end up with a problem that prevents you from meeting a deadline). Having said that however, I've been running my OS out of my RAM for months now, and once I got the hang of it, it's as stable as can be.

    What it will run on

    This script was designed and tested to run on Ubuntu 10.04. It has since been updated to be compatible with new Ubuntu releases. The git repo will tell you what versions of Ubuntu the script is available for - just look at the file names.

    While I no longer use this script myself day to day, when new versions of Ubuntu come out, if there is enough interest, I will continue to update it.

    NOTE: If you are using LVM, make sure that your /boot is on a separate partition that is NOT a logical volume.

    Manual

    Most of the main script is self explanatory as it asks the user questions and says what it will and will not do. The only undocumented feature of the main script is the following command:

    Code:
    sudo ./RAM_booster.sh --uninstall
    which, if run at any point, will undo everything the script has done. If the script is interrupted for any reason, it will actually ask you if you want to run the same uninstall method in order to revert the changes it made. This is why, if you run into problems or want to get rid of the RAM Session (which is what I refer to the OS that is created), the line above is your best friend. Although this is the best way to force an uninstall, if your install went fine and you want to uninstall, you can simply run the script again. It will detect that it has run before and offer to uninstall itself for you.

    A few lines up, I referred to RAM_booster.sh as the "main" script because it actually creates 2 other scripts and places them into the RAM Session environment. The additional scripts are:

    rupdate: Since the RAM Session is copied upon boot from a read only squashfs image, any software you download or updates you perform inside the RAM Session are undone as soon as you reboot. The only way to perform an update is to chroot into the folder to which the OS was copied to create the RAM Session (/var/squashfs), and update from inside that. After this, the squashfs image has to be recreated using /var/squashfs. As this can be a lengthy process, rupdate takes care of all that. All you need to do is run it, wait until it finishes, and reboot to take advantage of the newly installed updates. rupdate has a few arguments it can accept, most of which are explained if you run:

    Code:
    sudo rupdate --help
    if you have questions about it, either look at the comments I left inside the script (try: gksu gedit `which rupdate`), or post a comment in this thread and I'll try to answer it.

    rchroot: rupdate takes care of any updates you may need to do to the RAM Session from the repos, but what if you need to change some settings in /etc/ or download and install a .deb file, or even compile a program from source? rchroot allows you to chroot into /var/squashfs and edit the RAM Session directly. Any changes you make with rchroot will be permanent. As an added bonus, rchroot can also chroot into your Original OS in case you are too lazy to reboot to do something. One of the more useful things you may use it for is to manually generate a new squashfs image (although the rupdate script does this for you).

    If you make any changes using the rchroot script, you will need to regenerate the squashfs image. Since building this into the rchroot script would make it needlessly more complex, and the rupdate script already does this for you, all you have to do after you are done with rchroot is exit it, and run:

    Code:
    sudo rupdate -f
    Normally the rupdate script only recreates the squashfs image if some updates have been performed but the -f argument will force it to recreate the image in any case.

    Tricks

    The rupdate script mounts /boot while it runs and any updates that want to update /boot are free to do so. This works out great for the RAM Session, but not so well for the Original OS. This is because, in cases where there has been a kernel update, the /boot will now have a new kernel, and the grub menu will be updated to point to that kernel (for the Original OS and the RAM Session). The Original OS however will not have received the updates necessary to work well with the new kernel. This can, and probably will cause problems when you boot the unupdated Original OS using the new kernel.

    To fix this, the Original OS needs to be updated. This can be done in one of two ways.

    After the fact - if you have already run rupdate, it has already updated the kernel in /boot, and you are having problems booting back into the Original OS using the new kernel, simply choose the old kernel for the Original OS in the grub menu. Once the Original OS boots, update it, and reboot. It should now work fine with the new kernel.

    Preventing the problem in the first place - rupdate comes with a nice feature that allows for the Original OS to be updated at the same time as the RAM Session gets updated. The script even copies the .deb files downloaded for the update from the RAM Session into the Original OS so that the updates do not have to be downloaded twice. To use this feature, simply run:

    Code:
    sudo rupdate --both
    This will make sure the Original OS is always up to date. This method can also be used in the scenario above to update the Original OS after the fact. The inconvenience with this method is that it takes almost twice as long to update 2 Operating Systems as it does one (not quite twice as long since the updates don't have to be downloaded the second time). To save time, if you never boot into your Original OS anyways, you may not need to use this argument until you decide to boot back into it.

    Development

    Although the idea of creating a script to do this seemed pretty simple before I started (I already had the commands the script should run - see Original HOWTO), it ended up being quite complex. The hardest part was probably getting grub2 to work the way I wanted. The script needed to find /home, /boot, identify them for grub using a UUID, and write a grub.d script. This took a bit of work.

    For testing, I used Ubuntu 10.04 x64 Desktop (and later Ubuntu 10.10 x64 Desktop), and ran them in VMware. All I can say is thank god for VMWare and snapshots. I literally tested the script a hundred times during development, something I would have never been able to do without VMWare. At one point I had about 20 different snapshots at different points of testing with different hard drive configurations (different locations for partitions that were to hold /boot and /home) and my VM was about 120GB (no joke).

    I also used apt-cacher-ng running in another debian VM to make sure that while working on the rupdate script, I wasn't downloading the same updates over and over and over again from the internet. Not only would that have taken a painfully long time, but would have gotten me in trouble with my ISP (I have a monthly limit for downloads that I already exceed with torrents), and would have put unnecessary strain on official update servers.

    This took about a week to write, and I'm very curious how I did. If anyone looking at my bash code can improve the code's efficiency or make it work better, or even has a general comment about how bad I am at bash and why, let me know.

    Final Words

    Although I tested the script hundreds of times in VMware when I originally wrote it, the updates for the new versions of Ubuntu get significantly less testing (mostly because most of the time, hardly anything changes). On the off chance that this introduces a bug that gets past me, let me know and I'll do my best to fix it.

    Updates/Changes

    For a breakdown of the changes made to the scripts over time, see the git log.

    Download

    In order to get the latest version of the script, first install git:

    Code:
    sudo apt-get install git
    Once that's done, run:

    Code:
    git clone git://github.com/terminator14/RAM_booster.git
    This will create a RAM_booster folder in your current working directory with the script in it. Keep in mind that there are different versions of the script in the folder that is created - each file is labelled for the version of Ubuntu it will work for. You only need to run the one that corresponds to your version of Ubuntu.
    Last edited by terminator14; October 31st, 2013 at 02:15 AM.

  2. #2
    Join Date
    Jan 2009
    Location
    Europe
    Beans
    316
    Distro
    Ubuntu 10.04 Lucid Lynx

    Talking Re: Making Ubuntu Insanely Fast using RAM (updated and simplified)

    Well, thx for script.

    It works form me on Ubuntu 10.10.

    Got only 1GB ram.

    Ubuntu 10.10 was installed on USB key.
    Fresh install without update, then connected to internet and su root,
    run the script and :

    script was perfect, working: (after I unmounted other partiton than sdb!!!, if not it laod file from fat32 and ubuntu 10.04 to squash )

    sqash made by it is around 670MB

    ubuntu was installed from liveCD on 8gb USB steak.

    partiton sdb1: fat32 512 MB
    partiton sdb2: ext4 6000MB mount at /
    partiton sdb3: ext4 1024MB mount at /realusb
    partiton sdb4: swap 512MB

    /dev/sdb3 was used to copy home folder by that script.

    thx its working , but IMHO I need at list 2GB RAM for bettr results, becosue swap is slower down that beast
    give it try at december, when get 2x1GB ram from my parents PC.

    thx
    PS: My first gnome loaded to ram and it's faster then load normal way from USB
    Last edited by Sylslay; November 11th, 2010 at 12:19 AM.

  3. #3
    Join Date
    May 2008
    Location
    Edmonton, Canada
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Making Ubuntu Insanely Fast using RAM (updated and simplified)

    Quote Originally Posted by Sylslay View Post
    Well, thx for script.

    It works form me on Ubuntu 10.10.

    Got only 1GB ram.

    Ubuntu 10.10 was installed on USB key.
    Fresh install without update, then connected to internet and su root,
    run the script and :

    script was perfect, working: (after I unmounted other partiton than sdb!!!, if not it laod file from fat32 and ubuntu 10.04 to squash )

    sqash made by it is around 670MB

    ubuntu was installed from liveCD on 8gb USB steak.

    partiton sdb1: fat32 512 MB
    partiton sdb2: ext4 6000MB mount at /
    partiton sdb3: ext4 1024MB mount at /realusb
    partiton sdb4: swap 512MB

    /dev/sdb3 was used to copy home folder by that script.

    thx its working , but IMHO I need at list 2GB RAM for bettr results, becosue swap is slower down that beast
    give it try at december, when get 2x1GB ram from my parents PC.

    thx
    PS: My first gnome loaded to ram and it's faster then load normal way from USB
    Awesome! Good to know it works. Thanks.

  4. #4
    Join Date
    Nov 2010
    Beans
    1

    Re: Making Ubuntu Insanely Fast using RAM (updated and simplified)

    Just was I was looking for. I will give it a try in a few days.

    Thanks for posting this.

  5. #5
    Join Date
    May 2008
    Location
    Edmonton, Canada
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Making Ubuntu Insanely Fast using RAM (updated and simplified)

    Quote Originally Posted by hexion111 View Post
    Just was I was looking for. I will give it a try in a few days.

    Thanks for posting this.
    You get a chance to try it? How did it go?

  6. #6
    Join Date
    Dec 2009
    Beans
    46
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Making Ubuntu Insanely Fast using RAM (updated and simplified)

    Hmmm so I tried your script out, but at some point it was asking me to choose between 3 options - where to store the home folder. So I entered a 2, which would be for choosing a disk manually, but when I entered 2 the scripts started doing all kinds of things and at one point I saw it copying my home folder to somewhere (?) and then I saw it was trying to copy all my VM disks and I canceled the script . I ran the un-installer and it says it's un-installed but when I run the script again, it says the only option is to un-uninstall !

  7. #7
    Join Date
    Jul 2007
    Location
    Burlington, NC
    Beans
    1,994
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: Making Ubuntu Insanely Fast using RAM (updated and simplified)

    Linux does this already.

    The output of the `free` command tells you how much RAM is currently used by filesystem caching in the "cached" column.

    Overbearing caching schemes are a trade-off: trading faster *first run* of any given App for increased, or *slower*, boot times. Subsequent runs of said App will be exactly the same in either case.

    See the cache at work on stock Ubuntu 10.04:
    Code:
    ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
    1404032+0 records in
    1404032+0 records out
    718864384 bytes (719 MB) copied, 10.9608 s, 65.6 MB/s
    ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
    1404032+0 records in
    1404032+0 records out
    718864384 bytes (719 MB) copied, 0.879604 s, 817 MB/s
    ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
    1404032+0 records in
    1404032+0 records out
    718864384 bytes (719 MB) copied, 0.881407 s, 816 MB/s
    ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
    1404032+0 records in
    1404032+0 records out
    718864384 bytes (719 MB) copied, 0.861888 s, 834 MB/s
    ubuntu$
    ^repeated access to the same data is nearly instant

    The vanilla Linux setup gives you the best of both worlds:
    Extremely efficient on-demand caching *plus* at the kernel's discretion
    it can still free this memory for use by your apps as needed.
    Last edited by asmoore82; December 7th, 2010 at 08:06 PM.

  8. #8
    Join Date
    May 2008
    Location
    Edmonton, Canada
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Making Ubuntu Insanely Fast using RAM (updated and simplified)

    Quote Originally Posted by asmoore82 View Post
    Linux does this already.

    The output of the `free` command tells you how much RAM is currently used by filesystem caching in the "cached" column.

    Overbearing caching schemes are a trade-off: trading faster *first run* of any given App for increased, or *slower*, boot times. Subsequent runs of said App will be exactly the same in either case.

    See the cache at work on stock Ubuntu 10.04:
    Code:
    ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
    1404032+0 records in
    1404032+0 records out
    718864384 bytes (719 MB) copied, 10.9608 s, 65.6 MB/s
    ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
    1404032+0 records in
    1404032+0 records out
    718864384 bytes (719 MB) copied, 0.879604 s, 817 MB/s
    ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
    1404032+0 records in
    1404032+0 records out
    718864384 bytes (719 MB) copied, 0.881407 s, 816 MB/s
    ubuntu$ dd if=ubuntu-10.04.1-desktop-i386.iso of=/dev/null
    1404032+0 records in
    1404032+0 records out
    718864384 bytes (719 MB) copied, 0.861888 s, 834 MB/s
    ubuntu$
    ^repeated access to the same data is nearly instant

    The vanilla Linux setup gives you the best of both worlds:
    Extremely efficient on-demand caching *plus* at the kernel's discretion it can still free this memory for use by your apps as needed.
    I'm sure you're absolutely right and this is indeed correct (nice example too btw - I didn't know the OS would store an entire file like that). What the script does (like you said) is speed up the first time use of programs. So basically, if you just boot your Ubuntu, and open firefox (for example), regular Ubuntu will need to copy data from the HDD to RAM before it runs firefox which will take a few seconds (in my experience, anywhere from nearly instant up to 10 seconds - especially if it's a 5400 RPM drive like some laptops have). If the same scenario happens within the RAM Session after running the script, the data needed for firefox to load is already in RAM. That is the main difference. All subsequent launches of firefox in both cases will be from RAM and should be instant (in theory anyway). Perhaps I should have been more clear in my description of what the script does to speed up the OS when compared to a regular install.

    Despite the fact that a regular OS will eventually be as fast as RAM if you use it long enough without a reboot, the fact remains that inside the RAM Session, the system just feels more responsive as applications load instantly every single time. Perhaps this is a temporary benefit as a regular Ubuntu installation would eventually store all the apps you have used in RAM anyway, but to me this felt like a huge difference and was worth it at the time.

    You're also right about this system being a trade-off: faster first time run of apps vs slower boot time (as the system needs to copy everything to RAM at boot). I would just add that along with slower boot time, you also get some complexity added. By that I mean - you need to understand how this system works (at least a little) in order to know how to update your RAM Session properly. You also need to understand how to add files to the RAM Session that can survive a reboot, or edit config files in /etc. I think users need to realize this trade-off and make their own decision as to whether or not this trade-off is worth it to them. The best way to make the decision however if a reader of this thread is on the fence is to try the script and see for themselves if the benefits outweigh the trouble.

    I used to think they did until I installed my OS onto my SSD. Now my OS is plenty fast, which means I don't use the script anymore as its benefits no longer outweigh its trouble for me. I will however try to fix any bugs that people report the script has.
    Last edited by terminator14; December 7th, 2010 at 09:32 PM.

  9. #9
    Join Date
    Apr 2006
    Location
    Ubuntuland
    Beans
    2,118
    Distro
    Ubuntu 13.10 Saucy Salamander

    Lightbulb Re: Making Ubuntu Insanely Fast using RAM (updated and simplified)

    You could accomplish the same result by simply creating a list of applications that you want to be "first time fast" and just read them from disk in rc.local during startup.

    That would put them in the file cache where they would be loaded "instantly" at the expense of reading them from the hard drive at boot.
    24 beers in a case, 24 hours in a day. Coincidence? I think not!

    Saucy Salamander 64 bit, AMD Phenom II 955 Quad Core 3.2GHz, GeForce 8300
    8G PC2-6400 RAM, 128 GB SSD, Twin 1TB SATA 7200 RPM RAID0

  10. #10
    Join Date
    May 2008
    Location
    Edmonton, Canada
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Making Ubuntu Insanely Fast using RAM (updated and simplified)

    Quote Originally Posted by Slim Odds View Post
    You could accomplish the same result by simply creating a list of applications that you want to be "first time fast" and just read them from disk in rc.local during startup.

    That would put them in the file cache where they would be loaded "instantly" at the expense of reading them from the hard drive at boot.
    Good point. I think this is the basic idea behind preload, but I never could get preload to function properly. I wonder if something as simple as that (reading files in rc.local at boot) would end up with the exact same results in terms of speed of opening apps and speed of the OS itself as this script provides. If so, that would make the issue this script addresses be solved by something 100x simpler, which is always a good thing.

    The only potential problem I see with simply reading a file in rc.local (not running it but simply reading it) is that any libraries the file needs when it runs will not be read into RAM. This won't cause any problems - it simply means when the app is launched by the user, the main file will already be in RAM but the libraries it needs will still need to be copied from the HDD to RAM before they can be used. I'm not a programmer, and having limited knowledge of exactly how libraries are linked to apps, I may be wrong about this, but I'm pretty sure this is the case.

Page 1 of 29 12311 ... LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •