Page 1 of 8 123 ... LastLast
Results 1 to 10 of 73

Thread: Recent Ubuntuforums Downtime

  1. #1
    Join Date
    Oct 2004
    Location
    USA
    Beans
    3,774

    Recent Ubuntuforums Downtime

    This was posted on my blog which will hit the planet soon, but wanted you all to know as well.

    I just wanted to take a few minutes and comment on some of the recent issues we’ve had over at the UbuntuForums. Last week Thursday around 4pm EST, our main database server went into a “crashed” state. The result of this state was over 2000 queries that were lingering without completion for over 8000 seconds which resulted in resource issues, this caused the mysqld process to go into an unresponsive state and would not shutdown effectively. After the crash it was realized we had some inconsistent data in the database and a check/repair was ran to potentially correct the issues.

    On Sunday evening around 11:30pm EST the forums were down again but this time with table corruption on our largest tables which is our post table (it has around 6.5 million rows) and the thread table (over 1 million rows). I’m assuming everything “wasn’t” ok, after the crash on Thursday even though the checks said they were. Monday was spent cleaning up the tables, fixing corruption issues etc. Unfortunately, because of the massive size of our tables a check and repair takes upwards of 4-5 hours per check and repair until the issues were mostly resolved, this had to be completed a few times. We’ll be monitoring the data integrity over the next couple days to ensure everything stays consistent.

    When the forums were ready to come online last night around 7pm EST, we ran into issues with the amount of traffic the site was getting (benefits of being popular eh?). To help ease the pressure on our main database server we have utilized the “slave” feature in vbulletin which will send queries to a slave server to help reduce load on the master. So far this has been working pretty well except for a few hiccups which were easily corrected by changing mySQL variables.

    James Troup from Canonical was a great help making sure the proxies were stable last night and will be getting us additional hardware to help with our explosive growth as we have outgrown our current servers. James, thank you for your help, it is appreciated.

    Until things stable out we have disabled a couple of popular features on the forums namely the “Thank you” and the “Solved Post” features. We hope to re-enable them again soon. I expect we’ll have a some rough patches ahead of us until the new hardware is in place and we look forward to getting things more stable in the near future. We’ll also be implementing a better method of communication to our users when the forums are offline, expect an announcement on the forums in the near future about that. We appreciate everyones patience during this time.
    My Blog | Forum Search | Forum Guidelines
    Please don't PM support questions--post a thread so everyone can benefit!

  2. #2
    Join Date
    May 2007
    Location
    The New Forest
    Beans
    Hidden!
    Distro
    Xubuntu Development Release

    Re: Recent Ubuntuforums Downtime

    thanks for letting us know - I for one appreciate what you do

  3. #3
    Join Date
    Feb 2007
    Location
    Cincinnati, OH
    Beans
    1,433

    Re: Recent Ubuntuforums Downtime

    Good to hear. Certainly been missing the forums throughout my slower times at work. =] Here's to hoping you get the needed hardware and get everything back up & running as should be soon. *cheers*
    Quote Originally Posted by *snip*
    I love it when people stop trying to help me and I just uninstall ubuntu because of it . . .

  4. #4
    Join Date
    Feb 2008
    Location
    Cape Town, South Africa
    Beans
    Hidden!
    Distro
    Ubuntu 8.04 Hardy Heron

    Re: Recent Ubuntuforums Downtime

    Thanks again for your hard work on this regard.

  5. #5
    Join Date
    Apr 2008
    Location
    London
    Beans
    1,873
    Distro
    Kubuntu 12.04 Precise Pangolin

    Re: Recent Ubuntuforums Downtime

    Yay new hardware. The forums have been generally slow and unresponsive for as long as I've been using them so more hardware is a great thing. Thanks!


    .

  6. #6
    Join Date
    Aug 2007
    Location
    Manchester, UK
    Beans
    10,281
    Distro
    Ubuntu

    Re: Recent Ubuntuforums Downtime

    First off, thanks to U-G (and James over at Canonical), for managing to minimize the downtime

    Secondly, as I think this is going to be popular for the next couple of days, I've stuck the thread

  7. #7
    Join Date
    Nov 2008
    Beans
    9,635
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Recent Ubuntuforums Downtime

    Hi Ubuntu-geek,

    Thank you for your hard work. And all the forum work you do. Hopefully soon I'll be able to "thank you"

  8. #8
    Join Date
    Apr 2005
    Location
    Parts Unknown
    Beans
    9,213
    Distro
    Ubuntu Development Release

    Re: Recent Ubuntuforums Downtime

    Thank you, ubuntu-geek, and thank you, James, Canonical, etc.
    what's a troll? | my blog | my writing | Ubuntu Unleashed

    Don't ask support questions in PMs--post a thread so everyone can benefit!

  9. #9
    Join Date
    Oct 2007
    Location
    Auckland, New Zealand
    Beans
    Hidden!
    Distro
    Ubuntu 10.10 Maverick Meerkat

    Re: Recent Ubuntuforums Downtime

    Thank you so much ubuntu-geek and James for all the hard work you guys have put in.
    First they ignore you, then they laugh at you, then they fight you, then you win ~ Mahatma Gandhi

  10. #10
    Join Date
    Mar 2008
    Beans
    424

    Re: Recent Ubuntuforums Downtime

    whatever any of that means. get well soon.

Page 1 of 8 123 ... LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •