Page 1 of 2 12 LastLast
Results 1 to 10 of 17

Thread: CPU load average spikes every hour with CPU 55% and memory 82% only

  1. #1
    Join Date
    Jul 2020
    Beans
    8

    CPU load average spikes every hour with CPU 55% and memory 82% only

    Our Digital Ocean droplet's CPU load average spikes every 1-1.5 hour. Here is the the performance graph for the past 24 hours.

    Here are some MySQL error logs for 1.5 hours (one cycle) as well as syslog for 1.5 hours (one cycle).

    Some folks commented that "our mysql instance is being killed regularly by Linux’s OOM (out of memory) killer. You should review the load consumed by the resources running on the server. Based on experience it’s usually down to a greedy query that causes a resource contention and makes Linux sacrifice the most expensive process."

    However, if you look at the syslog, it is because there were a bunch of apache child processes running which crashed mysql. These apache child processes killed the process! They run every 1-1.5 hours! We don't have any cron jobs.

    Besides, if you look at the performance graph, even when the cpu load average (1 min) peaks at 20.71, the cpu hit 46.47% only and the memory hit 53%. The highest cpu and memory have never exceeded 57.5% and 88% respectively.

    I found many related threads on Google but they don't seem make any sense in my case including using swap, limiting apache child processes, etc. For instance, limiting apache child processes is just fire fighting. We got to find out the crux of the problem which is why there are so many apache child processes running in the first place which only cropped up about 10 days ago.

    Also, here is the htop log when the sites were momentarily down. Both cores shot up to 100% momentarily but the CPU stayed below 50%.

    Any idea will be very much appreciated!!

    Thanks a lot!

    BR, Tom
    Last edited by QIII; 4 Weeks Ago at 03:00 PM. Reason: Default font and color

  2. #2
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    20,468
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: CPU load average spikes every hour with CPU 55% and memory 82% only

    i am clueless but looks like something happens every 60 minutes. Does mysql recreate any views every 60 minutes?
    Have you considered taking a backup over to MariaDB and seeing if the problem follows??
    Last edited by TheFu; 4 Weeks Ago at 03:03 PM.

  3. #3
    Join Date
    Jul 2008
    Location
    The Left Coast of the USA
    Beans
    Hidden!
    Distro
    Kubuntu

    Re: CPU load average spikes every hour with CPU 55% and memory 82% only

    Hello!

    Please use the default font and color unless there is a pressing need to draw attention to a specific part of our post for clarity.
    Please read The Forum Rules and The Forum Posting Guidelines

    A thing discovered and kept to oneself must be discovered time and again by others. A thing discovered and shared with others need be discovered only the once.
    This universe is crazy. I'm going back to my own.

  4. #4
    Join Date
    Feb 2011
    Location
    Coquitlam, B.C. Canada
    Beans
    2,852
    Distro
    Ubuntu Development Release

    Re: CPU load average spikes every hour with CPU 55% and memory 82% only

    Readers, F.Y.I.: also posted on askubuntu
    Any follow-up information on your issue would be appreciated. Please have the courtesy to report back.

  5. #5
    Join Date
    Jul 2020
    Beans
    8

    Re: CPU load average spikes every hour with CPU 55% and memory 82% only

    hi there,

    thank you for your comment. were you talking about the links which are in orange color or something else? sorry, i'm new here so i don't know much about the forums' rules and regulations. apologies.

    br,
    tom

  6. #6
    Join Date
    Jul 2020
    Beans
    8

    Re: CPU load average spikes every hour with CPU 55% and memory 82% only

    hi thefu,

    thanks for your suggestion!

    but it doesn't sound like a very good option at this point because we got 31 websites sitting on this DO droplet and many of the domain names are not even managed by us.

    br,
    tom

  7. #7
    Join Date
    Jun 2006
    Location
    UK
    Beans
    Hidden!
    Distro
    Ubuntu 20.04 Focal Fossa

    Re: CPU load average spikes every hour with CPU 55% and memory 82% only

    Welcome to the forum!

    Quote Originally Posted by azinity-tom View Post
    thank you for your comment. were you talking about the links which are in orange color or something else? sorry, i'm new here so i don't know much about the forums' rules and regulations. apologies.
    QIII was not referring to the hyperlinks, but the fact that you added a mass of BBCode rendering your first post in a difficult to read grey colour.

    This was what it looked like.

    You also changed the font to Arial. That is a lesser problem, but unnecessary as the forum software defaults to a very clear font, and defaults to back text. As QIII says, please do not use non-standard font/colour/size/attribute formatting unless you need to highlight something.
    Ubuntu 18.04 Desktop Guide - Ubuntu 20.04 Desktop Guide - Forum Guide to BBCode - Using BBCode code tags - IRC #ubuntuforums

    Member: Not Canonical Team

    Please do not PM me about your forum account unless you have been asked to. The correct place to contact an admin about your account is here.

  8. #8
    Join Date
    Jul 2020
    Beans
    8

    Re: CPU load average spikes every hour with CPU 55% and memory 82% only

    hi doug s,

    thanks for your comment.

    first, no one from DO support replied to me even after i had opened two tickets. then i got some comments from the DO community but i still can't figure out why so many apache child processes were running and killing mysql. it's been almost two weeks since this problem first cropped up so i've become so desperate to get this fixed.

    any suggestions would be highly appreciated!

    br,
    tom

  9. #9
    Join Date
    Jul 2020
    Beans
    8

    Re: CPU load average spikes every hour with CPU 55% and memory 82% only

    hi coffeecat!

    thank you so much for pointing me to the right guides.

    it was probably because i just cut and pasted the content directly from the DO community website which carried the code forward. i should have hit the "remove format" button. once again, thank you for the reminder. have a good one.

    br,
    tom

  10. #10
    Join Date
    Feb 2011
    Location
    Coquitlam, B.C. Canada
    Beans
    2,852
    Distro
    Ubuntu Development Release

    Re: CPU load average spikes every hour with CPU 55% and memory 82% only

    Quote Originally Posted by azinity-tom View Post
    hi doug s,

    thanks for your comment.

    first, no one from DO support replied to me even after i had opened two tickets. then i got some comments from the DO community but i still can't figure out why so many apache child processes were running and killing mysql. it's been almost two weeks since this problem first cropped up so i've become so desperate to get this fixed.

    any suggestions would be highly appreciated!

    br,
    tom
    I simply wanted to make readers on both forums aware of the other posting so they wouldn't waste time if one or the other was making progress on your issue.
    Any follow-up information on your issue would be appreciated. Please have the courtesy to report back.

Page 1 of 2 12 LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •