Results 1 to 3 of 3

Thread: Does Apache mod_status give any insight to tuning Apache config?

  1. #1
    Join Date
    Nov 2014
    Beans
    12

    Does Apache mod_status give any insight to tuning Apache config?

    My server keeps crashing (running out of memory, oom-killer is killing mysql) at 40+ "requests currently being processed", as output by Apache's mod_status.

    Apache's mod_status also tells me that:

    • My server gets 3 requests/sec average, the range is 2-10 requests/sec, and one spike was 28 requests/sec.
    • My server has 7 average requests currently being processed, the range is 1-86.
    • My server has 10 average idle workers, the range is 0-99.


    I'm assuming that I need to change my Apache prefork config.
    Does this mod_status output give any insight as to what my Apache prefork settings should be - in particular, MinSpareServers, MaxSpareServers, MaxClients, and MaxRequestsPerChild? Or is this data unrelated?

    Here is my current Apache config:

    Code:
    # This is the main Apache server configuration file.  It contains the
    # configuration directives that give the server its instructions.
    # See http://httpd.apache.org/docs/2.2/ for detailed information about
    # the directives and /usr/share/doc/apache2-common/README.Debian.gz about
    # Debian specific hints.
    #
    #
    # Summary of how the Apache 2 configuration works in Debian:
    # The Apache 2 web server configuration in Debian is quite different to
    # upstream's suggested way to configure the web server. This is because Debian's
    # default Apache2 installation attempts to make adding and removing modules,
    # virtual hosts, and extra configuration directives as flexible as possible, in
    # order to make automating the changes and administering the server as easy as
    # possible.
    
    
    # It is split into several files forming the configuration hierarchy outlined
    # below, all located in the /etc/apache2/ directory:
    #
    #    /etc/apache2/
    #    |-- apache2.conf
    #    |    `--  ports.conf
    #    |-- mods-enabled
    #    |    |-- *.load
    #    |    `-- *.conf
    #    |-- conf.d
    #    |    `-- *
    #     `-- sites-enabled
    #         `-- *
    #
    #
    # * apache2.conf is the main configuration file (this file). It puts the pieces
    #   together by including all remaining configuration files when starting up the
    #   web server.
    #
    #   In order to avoid conflicts with backup files, the Include directive is
    #   adapted to ignore files that:
    #   - do not begin with a letter or number
    #   - contain a character that is neither letter nor number nor _-:.
    #   - contain .dpkg
    #
    #   Yet we strongly suggest that all configuration files either end with a
    #   .conf or .load suffix in the file name. The next Debian release will
    #   ignore files not ending with .conf (or .load for mods-enabled).
    #
    # * ports.conf is always included from the main configuration file. It is
    #   supposed to determine listening ports for incoming connections, and which
    #   of these ports are used for name based virtual hosts.
    #
    # * Configuration files in the mods-enabled/ and sites-enabled/ directories
    #   contain particular configuration snippets which manage modules or virtual
    #   host configurations, respectively.
    #
    #   They are activated by symlinking available configuration files from their
    #   respective *-available/ counterparts. These should be managed by using our
    #   helpers a2enmod/a2dismod, a2ensite/a2dissite. See
    #   their respective man pages for detailed information.
    #
    # * Configuration files in the conf.d directory are either provided by other
    #   packages or may be added by the local administrator. Local additions
    #   should start with local- or end with .local.conf to avoid name clashes. All
    #   files in conf.d are considered (excluding the exceptions noted above) by
    #   the Apache 2 web server.
    #
    # * The binary is called apache2. Due to the use of environment variables, in
    #   the default configuration, apache2 needs to be started/stopped with
    #   /etc/init.d/apache2 or apache2ctl. Calling /usr/bin/apache2 directly will not
    #   work with the default configuration.
    
    
    
    
    # Global configuration
    #
    
    
    ServerName seitan
    #
    # ServerRoot: The top of the directory tree under which the server's
    # configuration, error, and log files are kept.
    #
    # NOTE!  If you intend to place this on an NFS (or otherwise network)
    # mounted filesystem then please read the LockFile documentation (available
    # at <URL:http://httpd.apache.org/docs/2.2/mod/mpm_common.html#lockfile>);
    # you will save yourself a lot of trouble.
    #
    # Do NOT add a slash at the end of the directory path.
    #
    ServerRoot "/etc/apache2"
    
    
    #
    # The accept serialization lock file MUST BE STORED ON A LOCAL DISK.
    #
    LockFile ${APACHE_LOCK_DIR}/accept.lock
    
    
    #
    # PidFile: The file in which the server should record its process
    # identification number when it starts.
    # This needs to be set in /etc/apache2/envvars
    #
    PidFile ${APACHE_PID_FILE}
    
    
    #
    # Timeout: The number of seconds before receives and sends time out.
    #
    Timeout 300
    
    
    #
    # KeepAlive: Whether or not to allow persistent connections (more than
    # one request per connection). Set to "Off" to deactivate.
    #
    KeepAlive On
    
    
    #
    # MaxKeepAliveRequests: The maximum number of requests to allow
    # during a persistent connection. Set to 0 to allow an unlimited amount.
    # We recommend you leave this number high, for maximum performance.
    #
    MaxKeepAliveRequests 100
    
    
    #
    # KeepAliveTimeout: Number of seconds to wait for the next request from the
    # same client on the same connection.
    #
    KeepAliveTimeout 5
    
    
    ##
    ## Server-Pool Size Regulation (MPM specific)
    ## 
    
    
    # prefork MPM
    # StartServers: number of server processes to start
    # MinSpareServers: minimum number of server processes which are kept spare
    # MaxSpareServers: maximum number of server processes which are kept spare
    # MaxClients: maximum number of server processes allowed to start
    # MaxRequestsPerChild: maximum number of requests a server process serves
    <IfModule mpm_prefork_module>
        StartServers          5
        MinSpareServers       5
        MaxSpareServers      10
        MaxClients          150
        MaxRequestsPerChild   1000
    </IfModule>
    
    
    # worker MPM
    # StartServers: initial number of server processes to start
    # MinSpareThreads: minimum number of worker threads which are kept spare
    # MaxSpareThreads: maximum number of worker threads which are kept spare
    # ThreadLimit: ThreadsPerChild can be changed to this maximum value during a
    #              graceful restart. ThreadLimit can only be changed by stopping
    #              and starting Apache.
    # ThreadsPerChild: constant number of worker threads in each server process
    # MaxClients: maximum number of simultaneous client connections
    # MaxRequestsPerChild: maximum number of requests a server process serves
    <IfModule mpm_worker_module>
        StartServers          2
        MinSpareThreads      25
        MaxSpareThreads      75 
        ThreadLimit          64
        ThreadsPerChild      25
        MaxClients          150
        MaxRequestsPerChild   0
    </IfModule>
    
    
    # event MPM
    # StartServers: initial number of server processes to start
    # MinSpareThreads: minimum number of worker threads which are kept spare
    # MaxSpareThreads: maximum number of worker threads which are kept spare
    # ThreadsPerChild: constant number of worker threads in each server process
    # MaxClients: maximum number of simultaneous client connections
    # MaxRequestsPerChild: maximum number of requests a server process serves
    <IfModule mpm_event_module>
        StartServers          2
        MinSpareThreads      25
        MaxSpareThreads      75 
        ThreadLimit          64
        ThreadsPerChild      25
        MaxClients          150
        MaxRequestsPerChild   0
    </IfModule>
    
    
    # These need to be set in /etc/apache2/envvars
    User ${APACHE_RUN_USER}
    Group ${APACHE_RUN_GROUP}
    
    
    #
    # AccessFileName: The name of the file to look for in each directory
    # for additional configuration directives.  See also the AllowOverride
    # directive.
    #
    
    
    AccessFileName .htaccess
    
    
    #
    # The following lines prevent .htaccess and .htpasswd files from being 
    # viewed by Web clients. 
    #
    <Files ~ "^\.ht">
        Order allow,deny
        Deny from all
        Satisfy all
    </Files>
    
    
    #
    # DefaultType is the default MIME type the server will use for a document
    # if it cannot otherwise determine one, such as from filename extensions.
    # If your server contains mostly text or HTML documents, "text/plain" is
    # a good value.  If most of your content is binary, such as applications
    # or images, you may want to use "application/octet-stream" instead to
    # keep browsers from trying to display binary files as though they are
    # text.
    #
    # It is also possible to omit any default MIME type and let the
    # client's browser guess an appropriate action instead. Typically the
    # browser will decide based on the file's extension then. In cases
    # where no good assumption can be made, letting the default MIME type
    # unset is suggested  instead of forcing the browser to accept
    # incorrect  metadata.
    #
    DefaultType None
    
    
    
    
    #
    # HostnameLookups: Log the names of clients or just their IP addresses
    # e.g., www.apache.org (on) or 204.62.129.132 (off).
    # The default is off because it'd be overall better for the net if people
    # had to knowingly turn this feature on, since enabling it means that
    # each client request will result in AT LEAST one lookup request to the
    # nameserver.
    #
    HostnameLookups Off
    
    
    # ErrorLog: The location of the error log file.
    # If you do not specify an ErrorLog directive within a <VirtualHost>
    # container, error messages relating to that virtual host will be
    # logged here.  If you *do* define an error logfile for a <VirtualHost>
    # container, that host's errors will be logged there and not here.
    #
    ErrorLog ${APACHE_LOG_DIR}/error.log
    
    
    #
    # LogLevel: Control the number of messages logged to the error_log.
    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    #
    LogLevel warn
    
    
    # Include module configuration:
    Include mods-enabled/*.load
    Include mods-enabled/*.conf
    
    
    # Include list of ports to listen on and which to use for name based vhosts
    Include ports.conf
    
    
    #
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    # If you are behind a reverse proxy, you might want to change %h into %{X-Forwarded-For}i
    #
    LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
    LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %O" common
    LogFormat "%{Referer}i -> %U" referer
    LogFormat "%{User-agent}i" agent
    
    
    # Include of directories ignores editors' and dpkg's backup files,
    # see the comments above for details.
    
    
    # Include generic snippets of statements
    Include conf.d/
    
    
    # Include the virtual host configurations:
    Include sites-enabled/
    
    
    # Deny all Git repos
    <Directorymatch "^/.*/\.git/">
    Order deny,allow
    Deny from all
    </Directorymatch>


  2. #2
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Does Apache mod_status give any insight to tuning Apache config?

    Why do you think the issue is apache related? What else is running and sucking RAM on the system? Do you have any monitoring setup to gather facts - munin is easy to setup.

  3. #3
    Join Date
    Nov 2014
    Beans
    12

    Re: Does Apache mod_status give any insight to tuning Apache config?

    I have been monitoring my system for quite some time, trying to figure out the cause of these crashes. One of those methods involves dumping the output of mod_status to a file every minute. I've found that the only times the system runs out of memory (and crashes) is when mod_status has lots of W (sending reply) and K (keepalive). This is the mod_status output from the past 3 crashes:

    CRASH: 4/15
    Total Traffic: 4.4gb
    CPU Load: .85%
    Requests/sec: 6.63
    KB/sec: 267
    KB/request: 40
    Requests currently being processed: 40 (mix of W and K)
    Idle workers: 0


    CRASH: 3/15
    Total Traffic: 42gb
    CPU Load: .06% cpu
    Requests/sec: 3.73
    KB/sec: 72
    KB/request: 19
    Requests currently being processed: 45 (almost all W)
    Idle workers: 4


    CRASH: 2/26
    Total Traffic: 35gb
    CPU Load: .1%
    Requests/sec: 2.38
    KB/sec: 68
    KB/request: 28
    Requests currently being processed: 35 (all W)
    Idle workers: 5

    I also have mysql running on the system, using 3.3GB (of 8GB).

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •