HOWTO: Decrease disk activity

[sdennie]

Overview
This guide is aimed at people who wish to reduce the frequency at which their disks are written to. Reasons to do that may include:

- The noise or sight of the disk constantly writing every few seconds is annoying.
- To reduce heat.
- To reduce Load_Cycle_Count problems as described here.

By default, Ubuntu (actually, Linux in general), has two important features that cause the disk to write every few seconds. The first is the default journal commit time of the ext3 filesystem and the second is the rate at which the kernel wakes up to write dirty pages to disk.

By default, both of these settings are set to 5 seconds which is one of the primary reasons that Ubuntu systems seem to never have idle disks. This guide will show you how to raise those settings to a higher value in order to reduce disk activity.

Disclaimer
This guide shows you how to change default Ubuntu settings that are specifically in place to reduce the amount of work lost in the case of unexpected system shutdown. If you don't feel your system is 100% stable, do not use this guide. If you live in an area with frequent power outages, do not use this guide.

Implementation
We are going to change the default 5 second settings to 60 seconds. Anywhere were we are using the 60 seconds number, I will highlight in blue so that if you wish to adjust this value, you know exactly what to change. I'm using 60 seconds because it means that in the event of an unexpected power outage, you will lose at most around 60 seconds of work. If you are using a laptop or have an UPS for your machine, you can set this number higher if you wish.

To do this, we will backup 2 files, change the settings and then reboot to test. All these commands should be run from a new terminal where the current directory is your home directory.

The first file we will change is /etc/fstab so that it only commits the journal every 60 seconds. First, backup /etc/fstab (dda stands for Decrease Disk Activity):

sudo cp /etc/fstab /etc/fstab.dda.bak

Then, tinivole was nice enough to provide an awk command to automatically change all the ext3 partitions to have a commit time of 60 seconds. We will run the command and output to a temporary file so that we can verify that the output is correct:

awk '{if ($3 == "ext3") print $1" "$2"\t"$3"\t"$4",commit=60 "$5"\t"$6; else print}' /etc/fstab > fstab.tmp

Now, we will verify that the changes are correct:

diff /etc/fstab fstab.tmp

You should see output that looks similar to this:

6c6
< UUID=d12eeb71-f5a8-4062-95da-d0513c37ced2 /               ext3    defaults,errors=remount-ro 0       1
---
> UUID=d12eeb71-f5a8-4062-95da-d0513c37ced2 /	ext3	defaults,errors=remount-ro,commit=60 0	1

Notice the ,commit=60. You may have more output than that but, any line in the output that begins with ">" should have it and it's very important that it looks like this. If you have any doubt whatsoever, post the output of that command in this thread and wait until myself or someone else says that the output is ok before proceeding.

If the output looks ok, then we can replace /etc/fstab with our temporary file and then remove it:

sudo cp fstab.tmp /etc/fstab
rm fstab.tmp

That's it for ext3 changes and we will move on to kernel tunable changes. First we will make a backup of /etc/sysctl.conf and create a file for editing:

sudo cp /etc/sysctl.conf /etc/sysctl.conf.dda.bak
cp /etc/sysctl.conf sysctl.tmp

Now open sysctl.tmp with your editor of choice and, at the bottom of the file, add:

vm.dirty_ratio = 40
vm.dirty_background_ratio = 1
vm.dirty_writeback_centisecs = 6000

In this case, we are specifying centiseconds so, rather than 60 seconds, we use 6000 centiseconds here. The top two options simply wait slightly longer to request disk writes and then write as much as possible.

Once those changes have been made, we can copy our temporary file back and remove it:

sudo cp sysctl.tmp /etc/sysctl.conf
rm sysctl.tmp

Update:
For Hardy, the final step is to disable the part of pm-utils that will reset our values. To do this we simply make the script that resets these values not executable:

sudo chmod -x /usr/lib/pm-utils/power.d/laptop-tools

This final step is not needed for versions of Ubuntu before Hardy.

That's it for system changes.

Testing
The easiest way to test is to simply reboot your system and, once you have logged in and the system is fully up, watch your hard drive lights. You should notice them flickering far less often. As a more thorough test, you can run the following commands:

cat /proc/mounts | grep commit

You should see your commit=60 changes with that command. To test the other file we changed, issue the following three commands:

cat /proc/sys/vm/dirty_ratio
cat /proc/sys/vm/dirty_background_ratio
cat /proc/sys/vm/dirty_writeback_centisecs

The output should be, "40", "1" and "6000" respectively (it may actually be 5999 and not 6000. This is normal).

Questions/Comments/Suggestions welcome.

[logos34]

Interesting. Will this cure the problem so many people complain about (myself included as recently as Feisty)--viz. constant disk accessing/flickering hard drive light even when idle?

Also, what about using this:

... / ext3 noatime,errors=remount-ro 0 1

?

[sdennie]

Interesting. Will this cure the problem so many people complain about (myself included as recently as Feisty)--viz. constant disk accessing/flickering hard drive light even when idle?

Also, what about using this:


?

This will drastically reduce the frequency in which the disk is written so, yes. If you are using Hardy, noatime shouldn't be needed. I have a clean virtual machine that is upgraded from Gutsy to Hardy and:

cat /etc/fstab | grep ext3

Says:

UUID=d12eeb71-f5a8-4062-95da-d0513c37ced2 /               ext3    defaults,errors=remount-ro 0       1

(Notice, no noatime, no relatime, etc)

However, when I look at how the disk is actually mounted with:

cat /proc/mounts | grep ext3

I get (among other things):

/dev/disk/by-uuid/d12eeb71-f5a8-4062-95da-d0513c37ced2 / ext3 rw,relatime,errors=remount-ro,data=ordered 0 0

Notice that it added relatime without any intervention on my part. So, explicitly stating noatime or relatime in /etc/fstab probably isn't needed in Hardy.

[logos34]

This will drastically reduce the frequency in which the disk is written so, yes. If you are using Hardy, noatime shouldn't be needed. I have a clean virtual machine that is upgraded from Gutsy to Hardy and...
...So, explicitly stating noatime or relatime in /etc/fstab probably isn't needed in Hardy.

hmm, interesting. maybe I'll try it like yours, without the noatime and relatime (my /home partition) options.

[sdennie]

Yes, you shouldn't need noatime or relatime explicitly in /etc/fstab for ext3 partitions (probably others as well but, I'm not positive) in Hardy.

[shae]

I followed your guide, but some of your settings in sysctl.conf seem to be ignored. (Hardware is in my signature.)

shae@shae-laptop:~$ cat /proc/mounts | grep commit
/dev/disk/by-uuid/997a212b-d078-4b33-9a42-3e9cd62ebf23 / ext3 rw,relatime,errors=remount-ro,commit=60,data=ordered 0 0
/dev/disk/by-uuid/997a212b-d078-4b33-9a42-3e9cd62ebf23 /dev/.static/dev ext3 rw,relatime,errors=remount-ro,commit=60,data=ordered 0 0
shae@shae-laptop:~$ cat /proc/sys/vm/dirty_ratio
40
shae@shae-laptop:~$ cat /proc/sys/vm/dirty_background_ratio
10
shae@shae-laptop:~$ cat /proc/sys/vm/dirty_writeback_centisecs
500

[Nullack]

What I dont understand is why mounted drives with no activity from any processes will not sleep. My desktop has 1 physical drive that runs Linux and three other data drives running ntfs. Even when these three ntfs drives have no items running, they still wont sleep. There is no "dirty" journal data on these drives.

[sdennie]

I followed your guide, but some of your settings in sysctl.conf seem to be ignored. (Hardware is in my signature.)

shae@shae-laptop:~$ cat /proc/mounts | grep commit
/dev/disk/by-uuid/997a212b-d078-4b33-9a42-3e9cd62ebf23 / ext3 rw,relatime,errors=remount-ro,commit=60,data=ordered 0 0
/dev/disk/by-uuid/997a212b-d078-4b33-9a42-3e9cd62ebf23 /dev/.static/dev ext3 rw,relatime,errors=remount-ro,commit=60,data=ordered 0 0
shae@shae-laptop:~$ cat /proc/sys/vm/dirty_ratio
40
shae@shae-laptop:~$ cat /proc/sys/vm/dirty_background_ratio
10
shae@shae-laptop:~$ cat /proc/sys/vm/dirty_writeback_centisecs
500

What happens if you run:

sudo sysctl -p

That should reload /etc/sysctl.conf as well as print the values that it's reloading. If you don't see your changes at the bottom, you may have made a mistake somewhere in the process.

[sdennie]

What I dont understand is why mounted drives with no activity from any processes will not sleep. My desktop has 1 physical drive that runs Linux and three other data drives running ntfs. Even when these three ntfs drives have no items running, they still wont sleep. There is no "dirty" journal data on these drives.

By "sleep" do you mean go into suspend mode? If so, have you used hdparm to set them to do so? For example, to set /dev/sdb to suspend after 60 seconds of inactivity:

sudo hdparm -S12 /dev/sdb

For some reason the -S parameter is defined in increments of 5 seconds so, -S12 is 60 seconds.

[shae]

What happens if you run:

sudo sysctl -p

That should reload /etc/sysctl.conf as well as print the values that it's reloading. If you don't see your changes at the bottom, you may have made a mistake somewhere in the process.

After running that, the settings are properly set, but they do not survive a reboot.