Page 7 of 13 FirstFirst ... 56789 ... LastLast
Results 61 to 70 of 122

Thread: HOWTO: Set maximum CPU consumption in percentage by any process

  1. #61
    Join Date
    Apr 2008
    Beans
    41

    Re: HOWTO: Set maximum CPU consumption in percentage by any process

    Thanks for this. It was most helpful for my home media server ...

    I had to make some changes for my needs though as your method for polling for limited PIDs was always returning null... I ended up rewriting abit but the key change was the line:


    Code:
     LIMITED_PIDS=$(ps -eo args | gawk '$1=="cpulimit" {print $3}')
    ## changed to (more explaination below) ##
     LIMITED_PIDS=$(ps -eo args | grep cpulimited-bin |grep -v grep |awk '{print $3}')

    I needed a version with start/stop built in and also wanted logging.. So my finial code which is still being tested is:

    cpulimited.sh
    Code:
    #!/bin/bash
    # ==============================================================
    # CPU limit daemon - set PID's max. percentage CPU consumptions
    # ==============================================================
    # Variables
    CPU_LIMIT=25                   # Maximum percentage CPU consumption by each PID
    DAEMON_INTERVAL=3              # Daemon check interval in seconds
    LOG=/var/log/cuplimited.log
    CPULIMITED_TMPDIR=/tmp/cpulimited
    CONFIG=/etc/cpulimited/cpulimited.conf
    
    BLACKLIST=0
    WHITELIST=0
    PROCESSES_LIST=
    
    do_help() {
    cat <<EOF
    Usage:  $0 <options> start|stop
                         -f   ::  Run in foreground
           -c <config file>   ::  Use specified config file
    
    EOF
    exit 0
    }
    do_cpuwatcher() {
    
    # Search and limit violating PIDs
    while [ -z "$EXIT" ]; do
       sleep $DAEMON_INTERVAL
       # Check for exit
       DCMD=$(cat "$CONTROL_FILE" |tail -n 1)
       if [ "$DCMD" = "exit" ]; then
         echo "Shutdown requested..."
         PIDS="$(ps ax |grep cpulimited-bin |awk '{print $1}')"
         echo "Removing CPU limits ..."
         kill -9 $PIDS >/dev/null 2>&1
         sleep 1
         cd /tmp
         echo "Cleaning up ..."
         rm -rfd $CPULIMITED_TMPDIR
         echo "Goodbye"
         EXIT=1
         exit 0
       fi
       NEW_PIDS=$(eval "$NEW_PIDS_COMMAND")
       LIMITED_PIDS=$(ps -eo args | grep cpulimited-bin |grep -v grep |awk '{print $3}')
       QUEUE_PIDS=$(comm -23 <(echo "$NEW_PIDS" | sort -u) <(echo "$LIMITED_PIDS" | sort -u) | grep -v '^$')
       if [ -n "$QUEUE_PIDS" ]; then
         echo "These PIDs are already limited: $LIMITED_PIDS"
         echo "These PIDs are now entering the queue: $QUEUE_PIDS"
       fi
       for i in $QUEUE_PIDS; do
            echo "Setting limits for PID: $i"
            if [ -z "$FOREGROUND" ]; then
               ./cpulimited-bin -p $i -l $CPU_LIMIT -z >>$LOG 2>&1 &
            else
               ./cpulimited-bin -p $i -l $CPU_LIMIT -z &
            fi
       done
    done
    exit 0
    }
    
    
    # Init
    ##  Process commandline options
    while [ "$1" ]; do
       case "$1" in
         -f)
             FOREGROUND=1
             shift 1
          ;;
         -c)
             CONFIG=$2
             shift 2
          ;;
      start)
             CMD=start
             shift 1
             break
           ;;
       stop)
             CMD=stop
             shift 1
             break
          ;;
          *)
           do_help
           break
       esac
    done
    if [ -z "$CMD" ]; then
      do_help
      exit 0
    fi
    
    
    [ -f $CONFIG ] && . $CONFIG
    if [ $BLACKLIST = 1 -a $WHITELIST = 1 ]; then
       echo "Cannot use both whitelist AND blacklist.  Please check your config"
       exit 1
    else
       if [ "$BLACKLIST" = "yes" ]; then
         MODE=blacklist
         NEW_PIDS_COMMAND="top -b -n1 -c | grep -E '$BLACK_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
       fi
       if [ "$WHITELIST" = "yes" ]; then
         MODE=whitelist
         NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6' | grep -E -v '$WHITE_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
       fi
       if [ -z "$MODE" ]; then
         MODE=supervisor
         NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6 && \$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
       fi
    fi
    
    ##########  END Init ############
    
    
    case "$CMD" in
       start)
           echo "Starting up in $MODE mode ..."
           if [ -d $CPULIMITED_TMPDIR ]; then
             echo "Already running or stale tmp directory detected. Aborting ..."
             exit 23
           else
             mkdir -p $CPULIMITED_TMPDIR
             CONTROL_FILE="${CPULIMITED_TMPDIR}/control"
             chown -R root:root $CPULIMITED_TMPDIR
             chmod 700 $CPULIMITED_TMPDIR
             cd $CPULIMITED_TMPDIR
             CPULIMIT=$(which cpulimit)
             ln -s $CPULIMIT cpulimited-bin
             touch $CONTROL_FILE
             if [ -n "$FOREGROUND" ]; then
               # Do not fork
               do_cpuwatcher
             else
               # Fork into the background
               do_cpuwatcher >>$LOG &
             fi
           fi
           ;;
        stop)
            echo "Sending stop signal ..."
            CONTROL_FILE="${CPULIMITED_TMPDIR}/control"
            echo "exit" >$CONTROL_FILE
          ;;
    esac
    I set a temp directory up for the script and symlinked /usr/bin/cpulimit to ./cpulimited-bin to ease in grep'ing for cpulimit processes spawned by this script. I also added the ability to set changes from a config file. The other change is that BLACKLIST/WHITELIST is a mode toggle (0/1) and PROCESS_LIST would contain the strings to act on... Most all other changes were simply stylistic.

    Thanks again!
    Last edited by onemyndseye; December 17th, 2010 at 03:47 PM.

  2. #62
    Join Date
    May 2011
    Beans
    9

    Re: HOWTO: Set maximum CPU consumption in percentage by any process

    First, I want to thank you for this information. Second, I'd like to confess that for the life of me, I cannot figure out how to make cpulimit work properly with a multi-core processor.

    I see numerous posts both on these forums and elsewhere claiming that the limit can be a multiple of 100% based on the number of processors the machine has. I have four processors (phenom II 840 @3.2 ghz), and yet, cpulimit will not allow me to set any limit higher than 100%.

    I see this error when I try:

    "Error: limit must be in the range 0-100"

    Additionally, the help message states:

    "percentage of cpu allowed from 0 to 100 (mandatory)"

    I thought maybe I would be clever and run the top command followed by <shift> i to switch the irix mode. This did not work either. If I input 90& in cpulimit, top then displays that I am using 22% of processing time.

    In other words, I cannot figure out how to make it believe that I have a multi-core processor.

  3. #63
    Join Date
    Mar 2007
    Beans
    627

    Re: HOWTO: Set maximum CPU consumption in percentage by any process

    Hi,
    can you please check the number of CPUs (cores) on you computer with command:
    Code:
    cat /proc/cpuinfo | grep processor | wc -l
    Thanks

  4. #64
    Join Date
    May 2011
    Beans
    9

    Re: HOWTO: Set maximum CPU consumption in percentage by any process

    Quote Originally Posted by abcuser View Post
    Hi,
    can you please check the number of CPUs (cores) on you computer with command:
    Code:
    cat /proc/cpuinfo | grep processor | wc -l
    Thanks
    Yes, the command gives me "2" on my laptop. I am away from my desktop as I am currently traveling, but I can assure you that the desktop is 4.

    The laptop is a core 2 duo t7500 @ 2.2 gz and the desktop is a phenom 2 840 X4 @ 3.2 gz.

    This issue occurs on both systems. If it makes a difference, I installed cpulimit from the ubuntu repos in both maverick (on the laptop) and natty on the desktop. Thank you.

  5. #65
    Join Date
    Mar 2007
    Beans
    627

    Re: HOWTO: Set maximum CPU consumption in percentage by any process

    Quote Originally Posted by Cammer Pants View Post
    This issue occurs on both systems. If it makes a difference, I installed cpulimit from the ubuntu repos in both maverick (on the laptop) and natty on the desktop.
    I see the main problem is not with my script but with cpulimit in the first place. I suggest to open a bug report with command in Terminal:
    Code:
    ubuntu-bug cpulimit
    You have to register in Launcphad. See more info: https://help.ubuntu.com/community/ReportingBugs

  6. #66
    Join Date
    Sep 2011
    Beans
    2

    Re: HOWTO: Set maximum CPU consumption in percentage by any process

    Thanks for the scripts abcuser.

    I was having processor issues on Amazon EC2 (Ubuntu 10.04.3 LTS 64bit server) and this has really helped me out.

    However, I had to change

    Code:
    sudo chmod 700 /usr/bin/cpulimit_daemon.sh
    to

    Code:
    sudo chmod 755 /usr/bin/cpulimit_daemon.sh
    for the daemon wouldn’t start

    Code:
    * cpulimit daemon can not be started
    Thanks again
    MaxBux

  7. #67
    Join Date
    Mar 2007
    Beans
    627

    Re: HOWTO: Set maximum CPU consumption in percentage by any process

    MaxBux, maybe you opened file with different user then specified in documentation and so you get other owner then you should.

    Anyway I have updated instruction in my first post with your recommendation (755 instead 0f 700), just to avoid someone else having the same problem.

    Thanks for reporting this issue.

  8. #68
    Join Date
    Oct 2011
    Beans
    13

    Red face Re: HOWTO: Set maximum CPU consumption in percentage by any process

    Thanks for all the effort of putting up this script and helping us out with it, it's exactly what I need.

    I have been running it for about a week and found my system overrun by about 400+ cpulimit processes running (count as of today).

    The phenomenon appears to be cpulimit limiting itself (I have it set to limit anything over 3%).

    Here is one example of the 400, every PID I took from the -p argument was a cpulimit process its self (I check 4 or 5 examples)

    Code:
    root      6114 20975  0 Oct29 ?        00:02:30 cpulimit -p 1061 -l 3 -z
    root     13470 20975  0 Oct30 ?        00:02:18 cpulimit -p 6114 -l 3 -z
    Here are my varilables in cpulimit_daemon.sh, I installed the script as suggested in this thread:


    Code:
    # Variables
    CPU_LIMIT=3             # Maximum percentage CPU consumption by each PID
    DAEMON_INTERVAL=2       # Daemon check interval in seconds
    BLACK_PROCESSES_LIST=   # Limit only processes defined in this variable. If variable is empty (default) all violating processes are limited.
    WHITE_PROCESSES_LIST="openvpn|squid|log_helper|url_rewrite_helper|ssh|nrpe"   # Limit all processes except processes defined in this variable. If variable is empty (default) all violat$
    I'm going to try fixing this by whitelisting cpulimit.

    But even if that's the solution it's worth a report here.

  9. #69
    Join Date
    Aug 2007
    Location
    Belgium
    Beans
    176
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: HOWTO: Set maximum CPU consumption in percentage by any process

    Whitelisting cpulimit should be good, but IMHO setting everything to be limited at 3% is kind of low, don't you start to notice everything going slower?
    I tend to think the other way round, and i got a few processes in my blacklist, the rest can go on... No noticeable unwanted slowdowns... For example I got mencoder limited at 40%: everything else stays (most of the time) within reasonable boundaries, so no need to slow them down...
    El Belgicano
    -----------------
    Laptop: 5 years old Asus M6N (ATI9600/9700 graphics, 512Mb RAM, Intel Mobile 1.66GHz, 60Gb HDD) running 10.04-Lucid Lynx pretty nicely.

  10. #70
    Join Date
    Oct 2011
    Beans
    13

    Smile Re: HOWTO: Set maximum CPU consumption in percentage by any process

    I figured someone would comment on 3%. It's on an AWS micro box. Sustained CPU over about 5% will cause AWS to trigger a near shutdown of the box (98% stolen cpu). Unless the process bursts up and down (such as is the case with the limited whitelisted processes). Anything else *must* be limited to avoid triggering the AWS limiter. Typically cron jobs trigger the AWS limiter when I'm not looking.

    Basically the whitelisted processes are the only ones I care about and and have tested, the rest should mozy on through life at an easy pace.

Page 7 of 13 FirstFirst ... 56789 ... LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •