Page 1 of 3 123 LastLast
Results 1 to 10 of 22

Thread: How Probable is Hard Drive Failure?

  1. #1
    Join Date
    Jul 2010
    Beans
    28

    How Probable is Hard Drive Failure?

    I want to build a personal home server. I am still planning which way I want to go (Windows, Linux, or NAS). However, during my planning, I keep coming to the notion of RAID an or data protection in case of hard drive failure.

    I know for sure I won't be using a hardware RAID. I'll be going with the software RAID route if I do use RAID.

    My main question is, how probable is it that a hard drive will fail? Do server hard drives (regular 3.5inch hdd compared to 2.5inch laptop hdd) fail regularly?

    Do I really need to consider using a RAID during my early stages of building a server? If I build my server without the use of RAID, can I convert it to a RAID in the future when I upgrade my hardware to have more hard drive slots (since at the moment I only have 2 hard drive slots available on my old desktop and only 2x 3TB hard drive)?

  2. #2
    Join Date
    Apr 2008
    Location
    LOCATION=/dev/random
    Beans
    5,767
    Distro
    Ubuntu Development Release

    Re: How Probable is Hard Drive Failure?

    I would say that the probability of a drive failing is 100%.

    All hard drives fail, it's not a matter of if just a matter of when.
    I've got some 40GB drives kicking about that are still working but I've also had drives fail in the first week.

    Also bear in mind that RAID will only protect you in the case of hard drive failure, it doesn't protect against user error, hardware failure, software bugs etc...
    You should always have a full backup strategy in place as well as your RAID setup.
    Last edited by Cheesemill; January 21st, 2013 at 10:14 PM.
    Cheesemill

  3. #3
    Join Date
    Jan 2007
    Location
    Location: Location:
    Beans
    1,246
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: How Probable is Hard Drive Failure?

    A safe rule of thumb is that the drive will fail as soon as you really need it. If you go the RAID route, do it hardware with a mirror (RAID 6 or my favorite RAID 1+0). Mdadm is useful but horrendous if anything goes wrong with the OS. Best to keep your raid on a separate chip if your data is mission critical.
    clear && echo paste url and press enter; read paste; (youtube-dl $paste) | zenity --progress --title="" --text "Downloading, please wait" --auto-close --pulsate && ans=$(zenity --file-selection); gnome-terminal -x mplayer "$ans"

  4. #4
    Join Date
    Apr 2008
    Location
    LOCATION=/dev/random
    Beans
    5,767
    Distro
    Ubuntu Development Release

    Re: How Probable is Hard Drive Failure?

    Quote Originally Posted by |{urse View Post
    Mdadm is useful but horrendous if anything goes wrong with the OS. Best to keep your raid on a separate chip if your data is mission critical.
    Only if you can afford to keep spare RAID cards lying around.

    I tend to prefer mdadm for the reason that the drives can be moved to any machine running Linux and the array can be brought back up.

    I've been in the situation more than once where a hardware RAID card has died and I've been unable to recover the array as the card was no longer in production or unavailable second-hand.
    Cheesemill

  5. #5
    Join Date
    Feb 2009
    Location
    Dallas, TX
    Beans
    7,472
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: How Probable is Hard Drive Failure?

    Hi Zythyr.

    Yes, drives fail.

    If your data, or the service it provides it is important, you'd need either some sort of duplication (RAID, zfs, etc), or comprehensive backup procedures.

    Having said that, they don't die like flies. This is my experience (coincides with I've read): drives have a considerable chance to fail in their first weeks of operation. Once you get to a month or so, the chances remain lower and flat for at least a couple of years. Starting the forth year, the probability of failure becomes significantly.

    Just my thoughts.
    Regards.

  6. #6
    Join Date
    Nov 2007
    Location
    Newry, Northern Ireland
    Beans
    1,258

    Re: How Probable is Hard Drive Failure?

    For home use, mostly as a file server, I'd have a small disk or even a USB stick for the OS, and a mdadm RAID5 array for the data.

    I'd have a RAID5 array of at least 3 same size disks from different batches if possible, so that you have a good compromise between space (N-1 x capacity) while still maintaining redundancy. This way, if one disk dies, you can still get to your data, although access will be slower when the array is degraded.

    In a production server in the workplace, I'd also have a hot-spare in place, but this is probably overkill for a home server, unless money is no object.

    As Cheesemill said, use RAID and a good backup strategy, it's not an either/or scenario.

    Lastly, a virtual environment is the perfect place to learn about mdadm and what to do when a disk dies. Try creating a VM with 6 x 2GB disks and create an array with or without hotspares and see what happens when you remove one.
    Can't think of anything profound or witty.
    My Blog: http://gonzothegeek.blogspot.co.uk/

  7. #7
    Join Date
    Jan 2007
    Location
    Location: Location:
    Beans
    1,246
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: How Probable is Hard Drive Failure?

    Quote Originally Posted by Cheesemill View Post
    Only if you can afford to keep spare RAID cards lying around.
    Doesn't everyone?
    clear && echo paste url and press enter; read paste; (youtube-dl $paste) | zenity --progress --title="" --text "Downloading, please wait" --auto-close --pulsate && ans=$(zenity --file-selection); gnome-terminal -x mplayer "$ans"

  8. #8
    Join Date
    Jul 2010
    Beans
    28

    Re: How Probable is Hard Drive Failure?

    Quote Originally Posted by |{urse View Post
    A safe rule of thumb is that the drive will fail as soon as you really need it. If you go the RAID route, do it hardware with a mirror (RAID 6 or my favorite RAID 1+0). Mdadm is useful but horrendous if anything goes wrong with the OS. Best to keep your raid on a separate chip if your data is mission critical.
    Quote Originally Posted by papibe View Post
    Hi Zythyr.

    Yes, drives fail.

    If your data, or the service it provides it is important, you'd need either some sort of duplication (RAID, zfs, etc), or comprehensive backup procedures.

    Having said that, they don't die like flies. This is my experience (coincides with I've read): drives have a considerable chance to fail in their first weeks of operation. Once you get to a month or so, the chances remain lower and flat for at least a couple of years. Starting the forth year, the probability of failure becomes significantly.

    Just my thoughts.
    Regards.
    So after doing more research on RAID and possible backup strategies... I still have few questions.

    I prefer to go the software RAID route (mdadm) using RAID 5. However the issue is, I only have 2x 3TB hdd and 1x 500GB hard drive. Also, I can only fit 2x in my currently desktop enclosure. I can possibly make it fit 3 hdds if I remove the optical drive.

    1) If I do RAID 1 right now, can I switch/convert to RAID 5 in the future when I purchase more hard drives and upgrade my desktop enclosure to fit more hard drives?

    2) I want to do RAID 5, but I don't have 3x 3TB. I only have 2x 3TB. Can I create a RAID 5 with only 2 hdds and pop in the 3rd HD in future?

    3) So I read about ZFS when I was looking into FreeNAS. Its my understanding ZFS is a file system (not a data redundancy method). If I go to FreeNAS route, how do I ensure data redundancy? Is there a RAID for FreeNAS?

    4) Since I have 1x 500GB hdd, I want to partition this hdd to install both Windows (personal use) and Linux (or FreeNAS) for a server. (I want to install Windows for personal use because my desktop is a lot faster than my other devices, so I might use Windows for task that require this performance.) Will this be a problem/

    5) As @papibe, my RAID 5 can lose all the data if my OS is corrupted (which will be installed on my 500GB hard drive) of if this 500GB hard drive fails. Is there a way I can backup the server OS (Linux/FreeNAS) onto another location (ex: external hdd) in case the OS does get corrupted?

    6) @spynappels mentioned using a USB stick to install the OS on. Is this method feasible since USB (flash memory) only have limited number of read/writes and over time they can start to fail?

    7) I have heard people mention when installing the OS, to keep the /var/log or some other directories in a separate partition of the hard drive to prevent "run away logs". What does this mean? How do I do this? Also, when I install a software onto a linux, which directory does it get saved to? The reason I ask is, if I use a USB drive to run the OS, the USB drive will have limited space (~8GB-16GB). Won't this be an issue in the future if I want to install more software onto the Linux server? Won't I run out of space?
    Last edited by Zythyr; February 3rd, 2013 at 10:58 PM.

  9. #9
    Join Date
    Feb 2007
    Location
    West Hills CA
    Beans
    10,050
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: How Probable is Hard Drive Failure?

    3 and 6 go together. Freenas and nas4free have a USB (readonly) boot to RAM option, so the OS runs in RAM and keeps it's log files in RAM. So if you have green drives, your drives will spin down when not serving data and the OS has no disk to write to--and no disk to fail.

    BUT, if there is a problem, there are no log files kept (since they vanish in RAM). Some work arounds--send syslog messages to another linux server which is running a web server for instance. I do this and it works well. Set up text messaging/email for critical problems--this sends warnings out as they happen. Not has good as log files, but you will know when your NAS is not happy.

    The downside of a running in RAM, you will inevitably want to run more and more services and those services need a hard disk installation to store the database and configuration files. There are ways of "respinning" your configuration to a new, bootable USB, but at some point, it is just easier to do a hard disk install and just keep adding services as you need them.

    Yes, you can set up RAIDs on ZFS, but because ZFS has file corruption detection, it's not really necessary, but it depends on what your data availablity requirements are. Remember ZFS and RAID are not a backup strategy. You would still benefit a 3-2-1 strategy: 3 copies of critical data, 2 physical devices, 1 offsite. How you accomplish that is up to you.

    On 7) I have had a server running for 6 years continuously (with only a handful of reboots). Uptime has been over 99%. My logs are not huge. 440 items and 16MB. The standard logrotate cronjobs do a decent job of compressing log files. Now, if you have a runaway process or hardware failure then that could fill up a log file, but that has not been my experience. So a separate hard drive or partition for log files for a home server is overkill.

    4) Windows is always a problem, because if you get a virus, it can destroy your partition tables, which makes getting access to your data problematic.
    Last edited by tgalati4; February 3rd, 2013 at 11:32 PM.
    -------------------------------------
    Oooh Shiny: PopularPages

    Unumquodque potest reparantur. Patientia sit virtus.

  10. #10
    Join Date
    Nov 2007
    Location
    Newry, Northern Ireland
    Beans
    1,258

    Re: How Probable is Hard Drive Failure?

    Just to add that running from a USB is fine also in non read only mode for a system which will remain largely static (doing only a single or a few jobs, so not many OS changes) and logs are written to a partition on the RAID array. I've had several servers run like this for 3 years+ with no problems.

    I have seen logs going mad and filling a disk, so even for a home server, I'd still create a small (~10GB) partition for /var/log/ .

    I would echo tgalati4's 3-2-1 backup strategy as a must for important data.
    Can't think of anything profound or witty.
    My Blog: http://gonzothegeek.blogspot.co.uk/

Page 1 of 3 123 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •