Results 1 to 10 of 122

Thread: HOWTO: Set maximum CPU consumption in percentage by any process

Threaded View

  1. #20
    Join Date
    Mar 2007
    Beans
    671

    Re: HOWTO: Set maximum CPU consumption in percentage by any process

    Hi,
    first for all I don't know why MySQL is crashing. According to your post it looks like regardless of variable you put in, ALL of the processes gets into limiting state (echo 40) and so this affects MySQL processes in some way - maybe MySQL recognizes that something has changed with its process and it crashes.

    When you change the code and run it one more time are you sure you have STOPPED previously executed daemon? You see in the code is endless while loop, so you have to kill existing running application (= daemon) before running another instance of it. Before running daemon again search if there is an already running process by:
    Code:
    ps -eo pid,args | grep "cpulimit_daemon.sh"
    If you get some output then there is already a daemon running and you have to kill it first before running another instance of daemon.

    The next thing you can do is to add "echos" into after all of the lines that some variable is specified and echo the variable content. I have prepared debugging code see bellow. Try to run this code (but make sure you don't have a daemon already running).

    Code:
    #!/bin/bash
    # ==============================================================
    # CPU limit daemon - set PID's max. percentage CPU consumptions
    # ==============================================================
    
    # Variables
    CPU_LIMIT=20       	# Maximum percentage CPU consumption by each PID
    DAEMON_INTERVAL=3  	# Daemon check interval in seconds
    BLACK_PROCESSES_LIST=   # Limit only processes defined in this variable. If variable is empty (default) all violating processes are limited.
    WHITE_PROCESSES_LIST=   # Limit all processes except processes defined in this variable. If variable is empty (default) all violating processes are limited.
    
    # Check if one of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST is defined.
    if [[ -n "$BLACK_PROCESSES_LIST" &&  -n "$WHITE_PROCESSES_LIST" ]] ; then    # If both variables are defined then error is produced.
       echo "At least one or both of the variables BLACK_PROCESSES_LIST or WHITE_PROCESSES_LIST must be empty."
       echo "1. BLACK_PROCESSES_LIST: " $BLACK_PROCESSES_LIST
       echo "2. WHITE_PROCESSES_LIST: " $WHITE_PROCESSES_LIST
       exit 1
    elif [[ -n "$BLACK_PROCESSES_LIST" ]] ; then                                 # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
       NEW_PIDS_COMMAND="top -b -n1 -c | grep -E '$BLACK_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
       echo "3. CPU_LIMI: " $CPU_LIMIT
       echo "4. BLACK_PROCESSES_LIST: " $BLACK_PROCESSES_LIST
       echo "5. NEW_PIDS_COMMAND: " $NEW_PIDS_COMMAND
    elif [[ -n "$WHITE_PROCESSES_LIST" ]] ; then                                 # If this variable is non-empty then set NEW_PIDS_COMMAND variable to bellow command
       NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6' | grep -E -v '$WHITE_PROCESSES_LIST' | gawk '\$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
       echo "6. CPU_LIMI: " $CPU_LIMIT
       echo "7. WHITE_PROCESSES_LIST: " $WHITE_PROCESSES_LIST
       echo "8. NEW_PIDS_COMMAND: " $NEW_PIDS_COMMAND
    else
       NEW_PIDS_COMMAND="top -b -n1 -c | gawk 'NR>6 && \$9>CPU_LIMIT {print \$1}' CPU_LIMIT=$CPU_LIMIT"
       echo "9. CPU_LIMI: " $CPU_LIMIT
       echo "10. NEW_PIDS_COMMAND: " $NEW_PIDS_COMMAND
    fi
    
    # Search and limit violating PIDs
    while sleep $DAEMON_INTERVAL
    do
       NEW_PIDS=$(eval "$NEW_PIDS_COMMAND")                                                                    # Violating PIDs
       LIMITED_PIDS=$(ps -eo args | gawk '$1=="cpulimit" {print $3}')                                          # Already limited PIDs
       QUEUE_PIDS=$(comm -23 <(echo "$NEW_PIDS" | sort -u) <(echo "$LIMITED_PIDS" | sort -u) | grep -v '^$')   # PIDs in queue
    
       echo "11. DAEMON_INTERVAL: " $DAEMON_INTERVAL
       echo "12. CPU_LIMI: " $CPU_LIMIT
       echo "13. NEW_PIDS: " $NEW_PIDS
       echo "14. LIMITED_PIDS: " $LIMITED_PIDS
       echo "15. QUEUE_PIDS: " $QUEUE_PIDS
    
       for i in $QUEUE_PIDS
       do
           cpulimit -p $i -l $CPU_LIMIT -z &   # Limit new violating processes
       done
    done
    Please post the outputs of echos to the forum.

    There is also possible that Debian has something different then Ubuntu. This code was tested on Ubuntu 10.04 (= April 2010) few years back. It is possible that Debian uses some never version of one of the programs and e.g. outputs something that script does not expects. For example there are "gawk" commands with searches the data in one of the columns, if some output has changed then script is not going to be working fine.

    One more thing to discuss is are you sure you need a cpulimit_daemon? How do you run tar and zip commands? Are they executed manually from shell by you or some other way? You see from my tutorial there is a note at the top of the tutorial: "Note: If you would like to omit only one process then you don't need this "cpulimit daemon". In this case you only need cpulimit program to be executed from terminal."

    So if you execute the code from shell manually then you can:
    1. Execute the 'tar' or 'zip' command.
    2. Check the process_pid: ps -eo pid,args | grep tar
    3. Limit process manually: cpulimit -p process_pid -l cpu_limit -z &
    Note: process_pid is from command 2, cpu_limit is for example 20 for 20%.

    Regards
    Last edited by abcuser; January 14th, 2014 at 08:05 AM.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •