DVDs are not at all a reliable media

[linuxyogi]

Hi,
I have many data DVDs with me with all sorts of files. Today I opened an year old DVD and found that 2 files are not readable.

So what is the most reliable media for storing important files ?

[coldraven]

I have never used it but this may come in handy in the future
http://manpages.ubuntu.com/manpages/gutsy/man1/dvdisaster.1.html

[davetv]

Best I know is a brand new external mechanical hard drive that you only write to and read from - never deleting files/folders.

Keep unplugged in a cool dry place except when you need it.

[ibjsb4]

May want to rethink the whole thing and go with online storage. Dropbox (https://www.dropbox.com/) has served me well for years.

[Lars Noodén]

May want to rethink the whole thing and go with online storage. Dropbox (https://www.dropbox.com/) has served me well for years.

You'd have to have a second online storage site, a backup of your backup, to be sure. What happened to Megaupload can happen to other online providers. Or they can get bought out and shut down like Geocities did.

Home burned DVDs only last a few years because of the oxidation of the dyes (http://www.cci-icc.gc.ca/publications/notes/19-1-eng.aspx). I've lost data that way in as little as a year and a half since recording. On that type of media, the clock starts ticking the minute the discs come off the factory line.

The capacity of DVDs is kind of small by today's measures anyway so I'd second davetv's recommendation of an external mechanical drive. I'm not sure how long those are supposed to last in storage, but they are relatively cheap and if you get two different brands/models they are unlikely to fail at the same time.

Edit: Here's a better diagram of the layers involved in the DVD: http://www.clir.org/pubs/reports/pub121/sec3.html

[linuxyogi]

Best I know is a brand new external mechanical hard drive that you only write to and read from - never deleting files/folders.

Keep unplugged in a cool dry place except when you need it.

Why are you recommending a mechanical drive and not a solid state drive like a usb pen drive ?

[ibjsb4]

You'd have to have a second online storage site, a backup of your backup, to be sure.

Guess thats what I do :) backup to a second hdd and dropbox. Like mention, dvd just don't have enough storage capacity.

[linuxyogi]

May want to rethink the whole thing and go with online storage. Dropbox (https://www.dropbox.com/) has served me well for years.

Are you using the free 2 gb space provided by dropbox ? or are you paying for extra storage ? Coz to backup large files like video files I need good amount of space.

[ibjsb4]

I have 8gig of storage for free. You get your friends to sign up and you will get extra storage space for free. If you want to back up video, thats a different story as you need a good amount of storage. Are you using dvd to backup video? Must have a good size collection.

[linuxyogi]

I have 8gig of storage for free. You get your friends to sign up and you will get extra storage space for free. If you want to back up video, thats a different story as you need a good amount of storage. Are you using dvd to backup video? Must have a good size collection.

I will try n find someone who is willing to sign up. Yes I use DVDs to backup my videos. There must be 40-50 of them with me.

But now I am worried I will loose all that data eventually.

[kurt18947]

There's DVDs and there's DVDs. There are archival quality discs that are suppposed to remain readable for 100 years. They cost more than what you find at the local big box. Whether anyone will be have a means to read DVDs in 100 years is another matter. Here's one example of archival media:

http://www.mediasupply.com/mam-a-gold-archive-dvd-r-jewel-case-10-pack-83998.html

[ibjsb4]

Yes I use DVDs to backup my videos. There must be 40-50 of them with me.

By the time you get that uploaded to online storage you will be old and gray :) Uploading is a slow process.

[Lars Noodén]

Those DVDs and CDs that are claimed to last decades are those that are mechanically pressed in mass production. The ones that are home burned still suffer from relatively rapid degradation regardless of whether they have gold or silver alloy reflective backings, though the gold backings really are more hardy. See the CLIR and CCI links above for the mechanical / chemical details.

Tapes don't last very long either, but they're fortunately not part of the question here.

[Erik1984]

There's DVDs and there's DVDs. There are archival quality discs that are suppposed to remain readable for 100 years. They cost more than what you find at the local big box. Whether anyone will be have a means to read DVDs in 100 years is another matter. Here's one example of archival media:

http://www.mediasupply.com/mam-a-gold-archive-dvd-r-jewel-case-10-pack-83998.html

I've also noticed that there is a lot of quality difference between the cheap disks. I have various audio CDs burned > 10 years ago (most people, including me, often just picked the cheapest pack at the super market) and some of them have become completely unplayable. Sadly you don't know this beforehand when buying those discs. All my orignal CDs still play fine so just to support your point that there is huge difference in durability between optical discs.

[Ender Shadow]

I guess nobody's heard of M-DIscs?

http://www.mdisc.com/mdisc-technology/

[davetv]

Why are you recommending a mechanical drive and not a solid state drive like a usb pen drive ?

Flash/NAND storage integrity degrades relatively quickly over time. After a few years the data will be unreadable.

[buzzingrobot]

The most reliable storage medium is paper. A track record of centuries. Clay tablets are good, but take up a lot of room.

(I'm serious about the paper. No worries about recovery, etc.)

Digitally, I'd guess multiple backups on the best mechanical drives in multiple locations, using an OS-independent approach so your grandkids will be able to get at the data, is as good as we can do.

I've had two expensive SSD's go bad on me. Speedy? Yeah, sorta. Reliable? No.

[asifnaz]

Blu-ray discs are way more reliable and can hold up to 50 gigs of data . But blu ray burners are expensive .

[kurt18947]

The most reliable storage medium is paper. A track record of centuries. Clay tablets are good, but take up a lot of room.

(I'm serious about the paper. No worries about recovery, etc.)

Digitally, I'd guess multiple backups on the best mechanical drives in multiple locations, using an OS-independent approach so your grandkids will be able to get at the data, is as good as we can do.

I've had two expensive SSD's go bad on me. Speedy? Yeah, sorta. Reliable? No.

No question about paper being the best for written words. As long as the language doesn't become extinct and the ink/toner doesn't fade, you're good. Paper isn't real useful for audio & video though. I guess one way forward would be to rewrite data every X number of years to fresh media and resave in file format du jour if necessary. Interesting about SSDs. As I understand it, there is no data recovery if the flash cells themselves die unlike magnetic media.

[CharlesA]

Flash/NAND storage integrity degrades relatively quickly over time. After a few years the data will be unreadable.

I didn't know that.

I think I'll stick to regular hard drives for my data storage needs, at least for now.

@OP: I used to burn a ton of stuff to CDs for storage quite a few years ago - mostly cuz back then we were using 100MB and 250MB ZIP disks and CDs held more information. So far I have been able to pull off most of the data from those burned CDs, but I have also lost quite a lot due to read errors.

I would not recommend anything other than having a backup of a backup using hard drives or online storage if I wanted my data to be safe.

[Erik1984]

Blu-ray discs are way more reliable and can hold up to 50 gigs of data . But blu ray burners are expensive .

The burner is one thing, I guess the writable (big storage) Blu-ray discs aren't cheap either.

[kurt18947]

The burner is one thing, I guess the writable (big storage) Blu-ray discs aren't cheap either.

I haven't paid close attention but I think Blu-ray disk prices have come down. I just checked one source and Blu-ray writers start at around $70 U.S. disks $40 for 50 25GB. disks. Of course this may be another case of you get what you pay for.

[linuxyogi]

@OP:
I would not recommend anything other than having a backup of a backup using hard drives or online storage if I wanted my data to be safe.

I want to backup some videos files to Ubuntu One.

But first I want to compress each of them or all of them together to minimize file size.

I have heard and read that 7zip is the most effective compression format.

So I decided to compress a .flv file with archive manager selecting 7zip as the format.

The original file size is 95.3 mb and after compressing it to 7zip the file size became 94 mb.

As you can see there is hardly any reduction in file size.

What am I doing wrong here?

[Cheesemill]

You're not doing anything wrong. FLV files contain video that is already compressed, you can't really compress it any further.

[linuxyogi]

You're not doing anything wrong. FLV files contain video that is already compressed, you can't really compress it any further.

Understood. But thats a real disappointment.

[mastablasta]

DVD are OK, but the<y need to be stored porpperly. otherwsie i think tape is still good. again it needs to be stored propperly.

DVD have a problem that cracks appear on them. my theory is that they probably appear because of temperature changes. i can't think of any other reason why a DVD that is not moved arround at all or used would get cracks.

it also seems they are of different quality. i have some very old DVD's that work perfectly while some others of another brand that are acting up. so it seems it also depends on their material etc. at the time when they came out (CD and DVD) were hailed as ultimate backup media. their life span was uspposed to be very long. but in practice it should be closer to about 8-10 years i believe.

btw we still have films from super 8 camera form the 70's that are still as good as they were :-) cassets from the 80's that can sitll load a program on spectrum. so i can only wonder what is really wrong with this modern media... i am guessign poor quality control in China is causing the problmes, not their design.

[buzzingrobot]

Two words that being with 'R':

1. Redundancy: All backups are vulnerable for one or more reasons. Having backups of backups kept in different environments may not be practical for many people, but, at least, try to find a way to backup the backups of the files whose loss would have a real, i.e., other than emotional, impact. Keeping hardcopy of essential personal and business documentation in a rented deposit box makes a lot of sense.

2. Restorability: Backups are pointless if you don't know how to find and restore what you need. Say you use rsync to do incremental backups every hour to an external drive. You've zapped a photo from 3 months ago that you want to send to grandma. Your backup disk contains a few hundred thousand files with all kinds of arbitrary filenames. How are you gonna find the photo?

(Linux installs are so easy that I don't think it gains me anything to have some sort of Linux system recovery image. Keep a separate /home partition and back it up, back up any edits made in /etc, don't mess with the rest of the root filesystem, and a reinstall to cope with disaster should go smoothly.)

[slw210]

I would go with a HDD and/or



Optical discs
When it comes to optical discs, there is currently
really only one type on the market that can be trusted for long-term storage,
the Verbatim Gold Archival DVD-R. This particular DVD-R has been made
specifically to ensure long-term data stability and has been rated as the most
reliable DVD-R in a thorough long-term stress test by the well regarded German
c't magazine (c't 16/2008, pages 116-123). According to that test, the Verbatim
Gold Archival DVD-R has a minimum durability of 18 years and an average
durability of 32 to 127 years (at 25C, 50% humidity). No other disc came
anywhere close to these values, the second best DVD-R had a minimum durability
of only 5 years.

One very important factor when burning DVDs that often gets
overlooked, is that burn speed plays a very important role with regards to
recording accuracy and reliability. For archival purposes we strongly recommend
that you burn the DVDs at no more than 4x speed.


From here (http://www.linuxtech.net/tips+tricks/best_safe_long-term_data_storage.html)

Since you are storing lots of Video files, the investment in Blu-Ray and/or M-Disc might be worth it.

Most video sites recommend HDDs just due to file sizes. So time wise and storage space wise HDD, Blu-Ray or M-Disc.

I would also add, the quality of the equipment being used to do the copying is very important as well. Don't go cheap.

Professional CD's, DVD's, etc. are PRESSED. Don't be fooled into thinking you will get any burned media to last as long as pressed.


Realistically all you really need is 10-20 years since current storage technology is usually improved every 10 years or so.

[3rdalbum]

slw20: I'd take that marketing spiel with a pinch of salt. I remember "long-term stress tests" showing that 650mb CD-Rs would retain their data for fifty years.

Interesting enough, my earlier CD-Rs from the late 1990s have held up better than those I burnt in the early 2000s. I always suspected the stronger-coloured dye in the old discs would last longer.

Also, simply saying "Buy a hard disk for long-term data storage" is not necessarily good advice either. I've found WD Green hard disks will start getting bad sectors if you put it through the strain of actually doing anything with your data. One day, the dog next door barked and I lost a whole series of The Vicar of Dibley and the Windows bootloader. If you want to buy a hard disk for backup purposes, you need to buy something that's got a bit of quality behind it. You don't need to buy a server hard disk or a RAID box, but just a decent consumer hard disk will be fine.

[slw210]

I was just showing the OP the options. As stated, these numbers are also under ideal storage conditions. Do not see how Linuxtech is MARKETING SPIEL.

I'll agree with the older stuff being much higher quality, My PIII from around 2000 still has the same 40GB and 19 GB well used hard drives (WDs) running strong, I wouldn't expect that these days.

I fully expect anyone purchasing an item to check the reviews on said product themselves. As storage, you would think even a midrange HDD would last 10-20 years.

[rai_shu2]

I've been using a set of 1TB drives for the past 5 years. I haven't had any problems, but then I don't trust any one drive with all my data.

LOL I use TB drives the way I used to use floppy disks about 20 years ago.

[jedispork]

I have been debating over back up solutions for a while. Not many long term solutions. For reliability and size most people seem to prefer hdd or flash memory over standard dvd recordables and keeping multiple copies. Still not great for archiving though.

I have a drive and some m-discs on order. If the claims about them are even partially true they should be far superior to anything else we have available right now. The size limitation is a hassle but less so than loosing data. I only store important pics or videos. If these work out I could be happy keeping my stuff on flickr and m-disc while getting by with a small ssd on my pc. If I wanted more copies it would make more sense to buy more m-discs vs storage that wears out.

They also have a bluray version coming out and I think for a lot of people writing data in 25 gb chunks should be more than adequate. Maybe someday there will be a better archival medium but a stone like surface seems like it could be difficult beat.