HOWTO: Set maximum CPU consumption in percentage by any process
EDIT: Tutorial updated on September 15, 2011. New features added to scripts and some bugs fixed.
EDIT: 'Attention' info added for properly specifying the variables on January 14, 2014.
Purpose of cpulimit daemon:
Daemon runs in background and checks if some process is consuming more then 20% of CPU and if it does then daemon lowers CPU consumption of particular process to maximum of 20%. The goal is to have no single process that consumes more then 20% of CPU power.
Note: If you would like to omit only one process then you don't need this "cpulimit daemon". In this case you only need cpulimit program to be executed from terminal.
Tested environment:
Cpulimit daemon was tested on Ubuntu 8.04 LTS and Ubuntu 10.04 LTS. However it should be running fine on other Ubuntu versions and also on other Linux distributions, because it does not uses any Ubuntu specific code.
1. INSTALL PACKAGES
1. Install cpulimit package.
Code:
sudo apt-get install cpulimit
2. Install gawk package.
Code:
sudo apt-get install gawk
2. CREATE CPULIMIT DAEMON FILE
Open text editor with root privileges and save bellow daemon script text to new file /usr/bin/cpulimit_daemon.sh
Code:
#!/bin/bash
# ==============================================================
# CPU limit daemon - set PID's max. percentage CPU consumptions
# ==============================================================
# Variables
CPU_LIMIT=20 # Maximum percentage CPU consumption by each PID
DAEMON_INTERVAL=3 # Daemon check interval in seconds
BLACK_PROCESSES_LIST= # Limit only processes defined in this variable. If variable is empty (default) all violating processes are limited.
WHITE_PROCESSES_LIST= # Limit all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited.
# Check if one of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST is defined.
if [[ -n "$BLACK_PROCESSES_LIST" && -n "$WHITE_PROCESSES_LIST" ]] ; then # If both variables are defined then error is produced.
echo "At least one or both of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST must be empty."
exit 1
elif [[ -n "$BLACK_PROCESSES_LIST" ]] ; then # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
NEW_PIDS_COMMAND="top -b -n1 -c | grep -E '$BLACK_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
elif [[ -n "$WHITE_PROCESSES_LIST" ]] ; then # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6' | grep -E -v '$WHITE_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
else
NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6 && \$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
fi
# Search and limit violating PIDs
while sleep $DAEMON_INTERVAL
do
NEW_PIDS=$(eval "$NEW_PIDS_COMMAND") # Violating PIDs
LIMITED_PIDS=$(ps -eo args | gawk '$1=="cpulimit" {print $3}') # Already limited PIDs
QUEUE_PIDS=$(comm -23 <(echo "$NEW_PIDS" | sort -u) <(echo "$LIMITED_PIDS" | sort -u) | grep -v '^$') # PIDs in queue
for i in $QUEUE_PIDS
do
cpulimit -p $i -l $CPU_LIMIT -z & # Limit new violating processes
done
done
3. CHANGE VARIABLES TO YOUR ENVIRONMENT NEEDS
CPU_LIMIT
Change this variable in above script if you would like to omit CPU consumption for every process to any other percentage then 20%. Please read "If using SMP computer" chapter bellow if you have SMP computer (more then 1 CPU or CPU with more then 1 core).
DAEMON_INTERVAL
Change this variable in above script if you would like to have more/less regular checking. Interval is in seconds and default is set to 3 seconds.
BLACK_PROCESS_LIST and WHITE_PROCESSES_LIST
Variable BLACK_PROCESSES_LIST limits only specified processes. If variable is empty (default) all violating processes are limited.
Variable WHITE_PROCESSES_LIST limits all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited.
One or both of the variables BLACK_PROCESSES_LIST and WHITE_PROCESSES_LIST has to be empty - it is not logical that both variables are defined.
You can specify multiple processes in one of this two variables using delimiter characters "|" (without double quotes). Sample: if you would like to cpulimit all processes except mysql, firefox and gedit processes set variable: WHITE_PROCESSES_LIST="mysql|firefox|gedit"
Attention: Make sure you don't put any spaces in variable, for example WHITE_PROCESS_LIST= "mysql|firefox|gedit" (notice space character after = character) is not going to work correctly. You need to specify the variable without spaces: WHITE_PROCESS_LIST="mysql|firefox|gedit"
4. PROCEDURE TO AUTOMATICALLY START DAEMON AT BOOT TIME
1. Set file permissions for root user:
Code:
sudo chmod 755 /usr/bin/cpulimit_daemon.sh
2. Open text editor with root privileges and save bellow script to new file /etc/init.d/cpulimit
Code:
#!/bin/sh
#
# Script to start CPU limit daemon
#
set -e
case "$1" in
start)
if [ $(ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print $1}' | wc -l) -eq 0 ]; then
nohup /usr/bin/cpulimit_daemon.sh >/dev/null 2>&1 &
ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon started successfully"; else print " * cpulimit daemon can not be started" }'
else
echo " * cpulimit daemon can't be started, because it is already running"
fi
;;
stop)
CPULIMIT_DAEMON=$(ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print $1}' | wc -l)
CPULIMIT_INSTANCE=$(ps -eo pid,args | gawk '$2=="cpulimit" {print $1}' | wc -l)
CPULIMIT_ALL=$((CPULIMIT_DAEMON + CPULIMIT_INSTANCE))
if [ $CPULIMIT_ALL -gt 0 ]; then
if [ $CPULIMIT_DAEMON -gt 0 ]; then
ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print $1}' | xargs kill -9 # kill cpulimit daemon
fi
if [ $CPULIMIT_INSTANCE -gt 0 ]; then
ps -eo pid,args | gawk '$2=="cpulimit" {print $1}' | xargs kill -9 # release cpulimited process to normal priority
fi
ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon can not be stopped"; else print " * cpulimit daemon stopped successfully" }'
else
echo " * cpulimit daemon can't be stopped, because it is not running"
fi
;;
restart)
$0 stop
sleep 3
$0 start
;;
status)
ps -eo pid,args | gawk '$3=="/usr/bin/cpulimit_daemon.sh" {print}' | wc -l | gawk '{ if ($1 == 1) print " * cpulimit daemon is running"; else print " * cpulimit daemon is not running" }'
;;
esac
exit 0
3. Change file's owner to root:
Code:
sudo chown root:root /etc/init.d/cpulimit
4. Change permissions:
Code:
sudo chmod 755 /etc/init.d/cpulimit
5. Add script to boot-up procedure directories:
Code:
sudo update-rc.d cpulimit defaults
6. Reboot to check if script starts cpulimit daemon at boot time:
5. MANUALLY CHECK, STOP, START AND RESTART DAEMON
Note: Daemon and service in this tutorial has equal meaning.
Note: For users using prior to Ubuntu 8.10 (like Ubuntu 8.04 LTS) instead of service command use "sudo /etc/init.d/cpulimit status/start/stop/restart" syntax or install sysvconfig package using command: sudo apt-get install sysvconfig
Check if cpulimit service is running
Check command returns: "cpulimit daemon is running" if service is running, or "cpulimit daemon is not running" if service is not running.
Code:
sudo service cpulimit status
Start cpulimit service
You can manually start cpulimit daemon which will start to omit CPU consumption.
Code:
sudo service cpulimit start
Stop cpulimit service
Stop command stops cpulimit daemon (so no new process will be limited) and also sets to all existing limited processes to have full access to CPU, just like it was before cpulimit was not running.
Code:
sudo service cpulimit stop
Restart cpulimit service
If you change some variables settings in /usr/bin/cpulimit_daemon.sh like CPU_LIMIT, DAEMON_INTERVAL, BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST, then after changing settings you must restart service.
Code:
sudo service cpulimit restart
6. CHECK CPU CONSUMPTION WITH OR WITHOUT CPULIMIT DAEMON
Without daemon
1. stop cpulimit daemon (sudo service cpulimit stop)
2. execute CPU intensive tasks in background
3. execute command: top and check for %CPU column
Result of %CPU is probably more then 20% for each process.
With daemon turned on
1. start cpulimit daemon (sudo service cpulimit start)
2. execute the same CPU intensive tasks in background
3. execute command: top and check for %CPU column
Result of %CPU should be maximum 20% for each process.
Note: Don't forget at beginning %CPU can be more then 20%, because daemon has to catch violating process in interval of 3 seconds (set in script by default)
7. IF USING SMP COMPUTER
I have tested this code on Intel dual-core CPU computer - that behaves like SMP computer. Don't forget that top command and also cpulimit by default behaves in Irix mode, where 20% means 20% of one CPU. If there are two CPUs (or dual-core) then total %CPU can be 200%. In top command Irix mode can be turned off with command I (pressing <Shift>+i when top command is running) and Solaris mode is turned on, where total amount of CPU is divided by number of CPUs, so %CPU can be no more then 100% on any number of CPU computer. Please read more info about top command in top man page (search for I command). Please also read more about how cpulimit is operating on SMP computer in cpulimit official page.
But how does cpulimit daemon operates on SMP computer? Always in Irix mode. So if you would like to spend 20% of CPU power on 2-CPU computer then 40% should be used for CPU_LIMIT variable in cpulimit daemon script.
8. UNINSTALL CPULIMIT DAEMON AND CPULIMIT PROGRAM
If you would like to get rid of cpulimit daemon you can clean up your system by removing cpulimit daemon and uninstalling cpulimit program.
1. Stop cpulimit daemon
Code:
sudo service cpulimit stop # Stop cpulimit daemon and all cpulimited processes
2. Remove daemon from boot-up procedure
Code:
sudo update-rc.d -f cpulimit remove # Remove symbolic links
3. Delete boot-up procedure
Code:
sudo rm /etc/init.d/cpulimit # Delete cpulimit boot-up script
4. Delete cpulimit daemon
Code:
sudo rm /usr/bin/cpulimit_daemon.sh # Delete cpulimit daemon script
5. Uninstall cpulimit program
Code:
sudo apt-get remove cpulimit
6. Uninstall gawk program
If you don't need this program for any other script, you can remote it.
Code:
sudo apt-get remove gawk
9. NOTE ABOUT AUTHORS
I have just written daemon for cpulimit (bash scripts above). I am not the author of cpulimit project. If you need more info about cpulimit program, please read official cpulimit web page: http://cpulimit.sourceforge.net/.
Regards,
Abcuser
Re: HOWTO: Set maximum CPU consumption in percentage by any process
Re: HOWTO: Set maximum CPU consumption in percentage by any process
a very handy daemon to control the cpu usage in userspace. your instructions are comprehensive but you forgot to include one thing. you have to make the daemon executable. otherwise it will say something like "nohup: cannot run command `/root/cpulimit_daemon.sh': Permission denied".
nice job!
Re: HOWTO: Set maximum CPU consumption in percentage by any process
@bekirserifoglu, thanks I have updated the how-to.
Re: HOWTO: Set maximum CPU consumption in percentage by any process
Hang on a sec... can I just ask for clarification on something? Let's say you're playing a game or transcoding video, where the process uses nearly 100% CPU and you want it that way. Will it then run at 1.5th of the speed with your daemon turned on?
If so, who wants that? I can understand limiting a specified process, but I can't understand limiting every process.
Re: HOWTO: Set maximum CPU consumption in percentage by any process
Hi,
@3rdalbum, thanks for sharing your comment.
Let me explain. On PCs this daemon behaviour is probably not desired. Limiting specific process is more desired, like you described. But I have written this daemon for server environment and not every server environment is suitable for this daemon! I have virtual computers server environment.
Lets see a sample. In my case I have an Intel server where I have multiple virtual machines (also Ubuntu and some Windows too) using VirtualBox virtualization tool. On Ubuntu virtual host computer I have several virtual machines. I can omit how much memory can be consumed by each application in VirtualBox GUI, but can't enforce CPU usage. So I have written a daemon to control maximum CPU consumption by each virtual machine. I know I could run every virtual machine with only cpulimit command, but I am not only administrator on this Linux boxes and new virtual machines are added on the fly and others are moved to other Intel computers. So if administrator forgets to set cpulimit command to one new specific virtual machine, this new machine would consume all CPU power and take down other very important virtual machines.
Beside using cpulimit daemon, I can still set that some virtual machine can only use for example 10% of CPU power if desirable by executing cpulimit command by hand. But I can't allow one single process to make all others virtual systems down.
Regards
Re: HOWTO: Set maximum CPU consumption in percentage by any process
Thank you, your time and effort is much appreciated.
I know how to run a single process through cpulimit,
is there a way to limit a boot service with cpulimit?
I see that there is a way to limit a service, but I want it
limited from startup without me having to type anything...
Thanks
Re: HOWTO: Set maximum CPU consumption in percentage by any process
Hi,
don't know if this is possible, it is strange because boot process starts deamon that handles CPU.
Why would you like to omit CPU to boot process? I always like to boot up as soon as possible.
Regards
Re: HOWTO: Set maximum CPU consumption in percentage by any process
Quote:
Originally Posted by
abcuser
Hi,
don't know if this is possible, it is strange because boot process starts deamon that handles CPU.
Why would you like to omit CPU to boot process? I always like to boot up as soon as possible.
Regards
An example is: Folding@Home, I have it set up to use a service, so at boot it starts working.
I don't want to dedicate a full core to it though. I can manually cpulimit the PID, however
I'd like a way to do that automatically. So, I didn't really mean a "boot process"
so much as a process that starts on boot.
Thanks.
Re: HOWTO: Set maximum CPU consumption in percentage by any process
Hi,
OK, now I understand. The idea would probably go in this way. At boot time you start some script to start you program and after that some kind of cpulimit daemon has to start and try to catch PID of your desired program and lower the CPU of that program.
I have written this script some time ago, so it should be something like this (instead of cpulimit daemon write this script)
#!/bin/bash
cpulimit -p $( ps -eo pid,args | grep "process name you would like to CPU limit" | grep -v grep | gawk '{print $1}' ) -l 5 -z &
Note: instead of string "process name you would like to CPU limit" write your process name getting from "ps -eo pid, args" command. Above script assumes you would like to lower CPU to 5% (string: "-l 5" at the end of command), if you would like any other limit change this settings.
After that you have to make sure that your program has to start first and then this new cpulimit script goes second. Assuming you scrip has 20 priority which is by default, run the following script:
update-rc.d cpulimit-daemon defaults 21 19
Note: cpulimit-daemon is your script file name (select appropriate name) to start cpulimit daemon.
Hope this helps,
Regards