Page 1 of 3 123 LastLast
Results 1 to 10 of 24

Thread: Web server crashing every few days

  1. #1
    Join Date
    Jun 2007
    Beans
    95
    Distro
    Ubuntu 7.10 Gutsy Gibbon

    Web server crashing every few days

    Hi guys I've got an Ubuntu web server that keeps crashing on me every few days but don't even know where to start looking for the problem.
    I've already updated and upgraded it but no luck.

    When I took it out of the rack and hooked it up to a monitor and a keyboard it booted. So I stuck it back in and first time it booted ok but then a couple of days later went down and didn't want to boot up anymore. It has happened the same way a couple of times no.

    So can anyone give me some advice how to start or what to look for?

    Thanks and best regards

  2. #2
    Join Date
    Feb 2007
    Location
    West Hills CA
    Beans
    8,353
    Distro
    Ubuntu 12.10 Quantal Quetzal

    Re: Web server crashing every few days

    Look in /var/log and check every file for clues. If you don't find anything, then that usually points to a hardware problem--power supply or dying motherboard. While you have the rack out, reseat RAM, cables, and blow out the dust.

    Post any messages that you don't understand, don't bother posting entire log files.

    ssh in from another terminal and run top or htop and keep it running. If it freezes, you sometimes get a snapshot of what was running, take a picture with a digital camera, or cut and paste if possible.
    Last edited by tgalati4; March 12th, 2013 at 03:29 AM.
    -------------------------------------
    Oooh Shiny: PopularPages

    Unumquodque potest reparantur. Patientia sit virtus.

  3. #3
    Join Date
    Dec 2012
    Location
    Canada
    Beans
    187
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Web server crashing every few days

    Quote Originally Posted by self-defence View Post
    I've got an Ubuntu web server that keeps crashing on me every few days
    [...]
    I stuck it back in and first time it booted ok but then a couple of days later went down and didn't want to boot up anymore. It has happened the same way a couple of times no.

    So can anyone give me some advice how to start or what to look for?
    I had a similar experience with a Ubuntu 8.x virtual server. It ran fine for a couple of years and then it started "winking out". A hardware problem was ruled out fairly quickly because the host machine was not having any problems at all.

    It turns out that the client's "web guy" had just given the company's main website a makeover, changing it from (mostly) static web pages to a WordPress monstrosity with 40 or 50 plugins installed. Prior to that change an inconsiderate bot could establish multiple connections and "scarf down" the entire site without overwhelming the server. After the change one of those bots would come along and lock up the server so tight that it would not respond at all; the only thing I could do was reset (i.e., "power cycle") the VM.

    So, +1 for starting by looking at the logs. That's how I came to the conclusion that those bots were killing us, and once I installed `mod_limitipconn` and capped the number of simultaneous connections from a given IP the server stopped tanking.

  4. #4
    Join Date
    Jun 2011
    Location
    Las Vegas
    Beans
    Hidden!
    Distro
    Ubuntu 8.04 Hardy Heron

    Re: Web server crashing every few days

    There's a strong change its related to Apache. Can you run to command the press Shift + M and paste screenshot here?

    What type of website(s) are on this server? Wordpress, Drupal, custom code?

  5. #5
    Join Date
    Jun 2007
    Beans
    95
    Distro
    Ubuntu 7.10 Gutsy Gibbon

    Re: Web server crashing every few days

    Sorry guys had some stuff and couldn't get to the server.

    I'll be able to get it out tomorrow since it doesn't even boot after the last failure.

    The websites running are mostly Wordpress and a couple of static ones and some very basic custom work.

    You think it might be something to do with Apache?

    Thanks for the help I'll check the logs and top. I've already tried to look at the logs but wasn't able to decode much.

  6. #6
    Join Date
    Jun 2007
    Beans
    95
    Distro
    Ubuntu 7.10 Gutsy Gibbon

    Re: Web server crashing every few days

    Hi guys.

    Thanks for helping me so quickly but I was unable to access the server quicker because of my scheduled.

    The server is a Dell PowerEdge 2400 if it makes any difference.

    As always the server became unresponsive after a couple of days and this was the error on the screen when I turned on the monitor:


    So I switched it off and booted it up again and it booted fine and is working for the time being but there was an error during boot up regarding Apache just before the log on screen:


    I'm guessing there's something wrong in the Apache configuration file. Although I have no idea what since I used Webmin to configure everything.

    But I don't know if the two errors are connected?

    I logged in and left TOP running for when the server fails if something of significance can be caught there.

    Thanks for the help and best regards.

  7. #7
    Join Date
    Feb 2007
    Location
    West Hills CA
    Beans
    8,353
    Distro
    Ubuntu 12.10 Quantal Quetzal

    Re: Web server crashing every few days

    Linux is unforgiving when it runs out of memory. Did you not see the "Out of memory . . ." message? Linux will start killing processes to survive, but that won't help with a remote server.

    Instead of htop, run an ssh session and watch free. Do you have a large enough swap file? Do you have large enough memory footprint in php.ini? How much RAM in the machine? What is the output of free right after boot and starting all services.
    -------------------------------------
    Oooh Shiny: PopularPages

    Unumquodque potest reparantur. Patientia sit virtus.

  8. #8
    Join Date
    Jun 2007
    Beans
    95
    Distro
    Ubuntu 7.10 Gutsy Gibbon

    Re: Web server crashing every few days

    The sever has 512MB of RAM installed. But it had been working fine for the last three years before any problems showed up. I even ran Memtest to see if the memory was defective but it all passed.

    It crashed again during the night.

    Here is the free right after boot up:
    Code:
    free
                 total       used       free     shared    buffers     cached
    Mem:        515488     509140       6348          0       2468     324280
    -/+ buffers/cache:     182392     333096
    Swap:      1020116        896    1019220
    And here's the top from when it crashed. It was still on the screen:


    I don't think it has anything to do with increased activity on the web server if any there is less activity now that there was two years ago. So it should have been crashing then.

    Or should I just download the whole web server and db content and do a fresh install?

    Thanks for the help.

  9. #9
    Join Date
    Feb 2011
    Location
    Coquitlam, B.C. Canada
    Beans
    1,520
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Web server crashing every few days

    Look at your load average, the 15 minute time constant one is 76!!!! Your swap space is full, and the memory is consumed. Your server appears to be very very severly overloaded. Of course, once it starts swapping a lot, things gets worse, as more and more time is spent just swapping.
    Any follow-up information on your issue would be appreciated. Please have the courtesy to report back.

  10. #10
    Join Date
    Nov 2008
    Location
    Metro Boston
    Beans
    8,907
    Distro
    Kubuntu 14.04 Trusty Tahr

    Re: Web server crashing every few days

    I see so many apache and postgresql instances there that I have to wonder what this machine is doing. If it's a web server, and those are requests for pages that use the database, this machine is way underpowered for the amount of traffic it is handling. Increasing swap will help, but you need a lot more physical memory. I get worried when I have machines with load averages above 3-4. Values like 90 are off the charts. Upgrade to at least 2 GB of physical memory and a swap file of at least equal size.
    If you ask for help, please have the courtesy to check for responses and thank the people who helped you.

    Blog · Linode System Administration Guides · Android Apps for Ubuntu Users

Page 1 of 3 123 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •