HOWTO: Set maximum CPU consumption in percentage by any process

[abcuser]

Purpose of cpulimit daemon:
Daemon runs in background and checks if some process is consuming more then 20% of CPU and if it does then daemon lowers CPU consumption of particular process to maximum of 20%. The goal is to have no single process that consumes more then 20% of CPU power.

Tested environment:
This guide was tested on Ubuntu 8.10 and is sum of topic discussed in post How to control process to not consume more then 20% of CPU?

1. INSTALL CPULIMIT PACKAGE

sudo apt-get install cpulimit

Note: daemon also requires gawk package to be present. Install gawk package:

sudo apt-get install gawk

2. WRITE CPULIMIT DAEMON FILE
UPDATE: Script was updated May 4th 2009. New features have been added and some bugs fixed.

#!/bin/bash
# ==============================================================
# CPU limit daemon - set PID's max. percentage CPU consumptions
# ==============================================================

# Variables
CPU_LIMIT=20       	# Maximum percentage CPU consumption by each PID
DAEMON_INTERVAL=3  	# Daemon check interval in seconds
BLACK_PROCESSES_LIST=   # Limit only processes defined in this variable. If variable is empty (default) all violating processes are limited.
WHITE_PROCESSES_LIST=   # Limit all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited.

# Check if one of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST is defined.
if [[ -n "$BLACK_PROCESSES_LIST" &&  -n "$WHITE_PROCESSES_LIST" ]] ; then    # If both variables are defined then error is produced.
   echo "At least one or both of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST must be empty."
   exit 1
elif [[ -n "$BLACK_PROCESSES_LIST" ]] ; then                                 # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
   NEW_PIDS_COMMAND="top -b -n1 -c | grep '$BLACK_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
elif [[ -n "$WHITE_PROCESSES_LIST" ]] ; then                                 # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
   NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6' | grep -v '$WHITE_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
else
   NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6 && \$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
fi

# Search and limit violating PIDs
while sleep $DAEMON_INTERVAL
do
   NEW_PIDS=$(eval "$NEW_PIDS_COMMAND")                                                                    # Violating PIDs
   LIMITED_PIDS=$(ps -eo args | gawk '$1=="cpulimit" {print $3}')                                          # Already limited PIDs
   QUEUE_PIDS=$(comm -23 <(echo "$NEW_PIDS" | sort -u) <(echo "$LIMITED_PIDS" | sort -u) | grep -v '^$')   # PIDs in queue

   for i in $QUEUE_PIDS
   do
       cpulimit -p $i -l $CPU_LIMIT -z &   # Limit new violating processes
   done
done

Note: one or both of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST has to be empty - it is not logical that both variables are defined.

You can specify multiple processes in one of this two variables using two delimiter characters "\|" (without double quotes). Sample: if you would like to cpulimit all processes expecpt mysql, firefox and gedit proceses set variable: WHITE_PROCESSES_LIST="mysql\|firefox\|gedit".

3. PROCEDURE TO SAVE DAEMON AND BOOT-UP IT AT BOOT TIME

1. copy above daemon script to /root/cpulimit_daemon.sh file and set file permissions for root user:

sudo chmod 700 /root/cpulimit_daemon.sh

2. with text editor save the following commands to file /etc/init.d/cpulimit
UPDATE: In bellow script "restart" option was simplified on May 4th 2009.

#!/bin/sh
#
# Script to start CPU limit daemon
#
set -e
. /lib/lsb/init-functions
case "$1" in
start)
nohup /root/cpulimit_daemon.sh &
;;
stop)
ps -eo pid,args | gawk '$3=="/root/cpulimit_daemon.sh"  {print $1}' | xargs kill -9   # kill cpulimit daemon
ps -eo pid,args | gawk '$2=="cpulimit" {print $1}' | xargs kill -9       # release cpulimited process to normal priority
;;
restart)
$0 stop
sleep 5
$0 start
;;
status)
ps -eo pid,args | gawk '$3=="/root/cpulimit_daemon.sh"  {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon is running"; else print " * cpulimit daemon is not running" }'
;;
esac
exit 0

3. change file's owner to root:

cd /etc/init.d/
sudo chown root:root cpulimit

4. change permissions:

sudo chmod 755 cpulimit

5. add script to boot-up procedure directories:

sudo update-rc.d cpulimit defaults

6. reboot

sudo reboot

4. MANUALLY CHECK, STOP, START AND RESTART DAEMON
After reboot check:

sudo service cpulimit status

it should return: "cpulimit daemon is running" or "cpulimit daemon is not running".

Note: For users using prior to Ubuntu 8.10 instead of service command use "sudo /etc/init.d/cpulimit status" syntax or install sysvconfig package using command: sudo apt-get install sysvconfig

To stop cpulimit service:

sudo service cpulimit stop

To start cpulimit service:

sudo service cpulimit start

If you change some settings in /root/cpulimit_daemon.sh for example CPU limit to something else then 20% or daemon interval to something else then 3 seconds, then after changing settings you can restart service:

sudo service cpulimit restart

5. CHECK CPU CONSUMPTION WITH OR WITHOUT CPULIMIT DAEMON
Without daemon:
1. stop cpulimit daemon (sudo service cpulimit stop)
2. execute CPU intensive tasks in background
3. execute command: top and check for %CPU column
Result of %CPU is probably more then 20% for each process.

With daemon turned on:
1. start cpulimit daemon (sudo service cpulimit start)
2. execute the same CPU intensive tasks in background
3. execute command: top and check for %CPU column
Result of %CPU should be maximum 20% for each process.
Note: Don't forget at beginning %CPU can be more then 20%, because daemon has to catch violating process in interval of 3 seconds (set in script by default)

6. IF USING SMP COMPUTER
I have tested this code on Intel dual-core CPU computer - that behaves like SMP computer. Don't forget that top command and also cpulimit by default behaves in Irix mode, where 20% means 20% of one CPU. If there are two CPUs (or dual-core) then total %CPU can be 200%. In top command Irix mode can be turned off with command I (pressing <Shift>+i when top command is running) and Solaris mode is turned on, where total amount of CPU is divided by number of CPUs, so %CPU can be no more then 100% on any number of CPU computer. Please read more info about top command in top man page (search for I command). Please also read more about how cpulimit is operating on SMP computer in cpulimit official page.

But how does cpulimit daemon operates on SMP computer? Always in Irix mode. So if you would like to spend 20% of CPU power on 2-CPU computer then 40% should be used for CPU_LIMIT variable in cpulimit daemon script.

7. UNINSTALL CPULIMIT DAEMON AND CPULIMIT PROGRAM
If you would like to get rid of cpulimit daemon you can clean up your system by removing cpulimit daemon and uninstalling cpulimit program.

Stop cpulimit daemon:

sudo service cpulimit stop       # stop cpulimit daemon and all cpulimit process

Remove daemon from boot-up procedure:

sudo update-rc.d -f cpulimit remove  # Remove symbolic links
sudo rm /etc/init.d/cpulimit         # Delete cpulimit boot-up script
sudo rm /root/cpulimit_daemon.sh     # Delete cpulimit daemon script

Uninstall cpulimit program:

sudo apt-get cpulimit remove

8. NOTE ABOUT AUTHORS:
I have just written daemon for cpulimit (bash scripts above). I am not the author of cpulimit project. If you need more info about cpulimit program, please read official cpulimit web page: http://cpulimit.sourceforge.net/.

Regards,
Abcuser

[bapoumba]

Edited thread title

[bekirserifoglu]

a very handy daemon to control the cpu usage in userspace. your instructions are comprehensive but you forgot to include one thing. you have to make the daemon executable. otherwise it will say something like "nohup: cannot run command `/root/cpulimit_daemon.sh': Permission denied".

nice job!

[abcuser]

@bekirserifoglu, thanks I have updated the how-to.

[3rdalbum]

Hang on a sec... can I just ask for clarification on something? Let's say you're playing a game or transcoding video, where the process uses nearly 100% CPU and you want it that way. Will it then run at 1.5th of the speed with your daemon turned on?

If so, who wants that? I can understand limiting a specified process, but I can't understand limiting every process.

[abcuser]

Hi,
@3rdalbum, thanks for sharing your comment.

Let me explain. On PCs this daemon behaviour is probably not desired. Limiting specific process is more desired, like you described. But I have written this daemon for server environment and not every server environment is suitable for this daemon! I have virtual computers server environment.

Lets see a sample. In my case I have an Intel server where I have multiple virtual machines (also Ubuntu and some Windows too) using VirtualBox virtualization tool. On Ubuntu virtual host computer I have several virtual machines. I can omit how much memory can be consumed by each application in VirtualBox GUI, but can't enforce CPU usage. So I have written a daemon to control maximum CPU consumption by each virtual machine. I know I could run every virtual machine with only cpulimit command, but I am not only administrator on this Linux boxes and new virtual machines are added on the fly and others are moved to other Intel computers. So if administrator forgets to set cpulimit command to one new specific virtual machine, this new machine would consume all CPU power and take down other very important virtual machines.

Beside using cpulimit daemon, I can still set that some virtual machine can only use for example 10% of CPU power if desirable by executing cpulimit command by hand. But I can't allow one single process to make all others virtual systems down.
Regards

[akoblentz]

Thank you, your time and effort is much appreciated.

I know how to run a single process through cpulimit,
is there a way to limit a boot service with cpulimit?
I see that there is a way to limit a service, but I want it
limited from startup without me having to type anything...

Thanks

[abcuser]

Hi,
don't know if this is possible, it is strange because boot process starts deamon that handles CPU.

Why would you like to omit CPU to boot process? I always like to boot up as soon as possible.
Regards

[akoblentz]

Originally Posted by abcuser Hi, don't know if this is possible, it is strange because boot process starts deamon that handles CPU. Why would you like to omit CPU to boot process? I always like to boot up as soon as possible. Regards

An example is: Folding@Home, I have it set up to use a service, so at boot it starts working.
I don't want to dedicate a full core to it though. I can manually cpulimit the PID, however
I'd like a way to do that automatically. So, I didn't really mean a "boot process"
so much as a process that starts on boot.

Thanks.

[abcuser]

Hi,
OK, now I understand. The idea would probably go in this way. At boot time you start some script to start you program and after that some kind of cpulimit daemon has to start and try to catch PID of your desired program and lower the CPU of that program.

I have written this script some time ago, so it should be something like this (instead of cpulimit daemon write this script)
#!/bin/bash
cpulimit -p $( ps -eo pid,args | grep "process name you would like to CPU limit" | grep -v grep | gawk '{print $1}' ) -l 5 -z &

Note: instead of string "process name you would like to CPU limit" write your process name getting from "ps -eo pid, args" command. Above script assumes you would like to lower CPU to 5% (string: "-l 5" at the end of command), if you would like any other limit change this settings.

After that you have to make sure that your program has to start first and then this new cpulimit script goes second. Assuming you scrip has 20 priority which is by default, run the following script:
update-rc.d cpulimit-daemon defaults 21 19

Note: cpulimit-daemon is your script file name (select appropriate name) to start cpulimit daemon.

Hope this helps,
Regards