Bad Sectors - CAN NOT save the data on the HDD

[ActionParsnip]

This:
https://www.datanumen.com/blogs/hard...ses-solutions/



That did NOT solve my problem. Kindly read the last paragraph in my OP.

"Soft Bad Sectors" are not "Software Bad Sectors"

[mikewhatever]

Bad sectors are bad sectors. Data corruption due to a power failuer is not it.
I am afraid that article is pure nonsense.

[ActionParsnip]

Bad sectors are bad sectors. Data corruption due to a power failuer is not it.
I am afraid that article is pure nonsense.

Exactly

[amjjawad]

Hi and thank you for your reply.

Have you considered Clonezilla?
https://clonezilla.org/

No, I haven't thought about it. Appreciate the suggestion.
However, with the current situation, I am not even sure if this is going to help/work or not?

I will give it a go!

Prepare a USB stick with Clonezilla
Clone your /home partition

I was reading the website and it mentioned:

The destination partition must be equal or larger than the source one.

I am not sure what does it mean?
Does it mean, if the drive I'm trying to clone is 1TB so the destination should be equal or larger?
Or, it means the size of 'files' that I'd like to clone so the destination drive should be equal or larger?

I have never used this software before.
Whenever I wanted to do a backup, I simply copy and paste. Old school.

Beg, borrow, steal another PC
Restore your cloned partition to a different PC
Check your data for damage/corruption etc.

Steal?
I don't have to steal.

If and only if I managed to do that successfully, that would be great.
The most important part is, I check the data, if saved, for damage or corruption.

I'll try and report back.

[tea for one]

In this regard, cloning is not strictly a back-up, it is just a means of transferring data from a possibly damaged drive to a healthy one.

You could first try partition to partition clone using devices (rather than images)
Boot up a live session of Clonezilla
Select only your /home partition as source (i.e. less than 1TB)
Use your new 1TB disk as destination (i.e. greater than your 877GB home partition)

When the process has finished, use another Ubuntu PC to see if you can access the files as a normal external drive.

Fingers crossed.......

[amjjawad]

Hi and thank you for your reply.

If a LiveCD or Try Ubuntu environment running from a flash drive are failing, then the computer hardware is dying. Probably something inside the motherboard. Initially, it looked like it could just be the original HDD, but the alternate boot media having problems moved the issue.

At the beginning, I thought it was the RAM. Never thought it could be the HDD. Never thought a machine that was running only for 10 weeks might fail so miserably like this.

To get data off a failing HDD,
a) stop using it for anything other than safe copying to a new disk.
b) use a different computer.
c) Take the HDD out that has the data you want and connect it to a different computer that supports 2 storage devices and can boot from a USB-flash Try Ubuntu environment.

I don't have any machine here that support (2) storage devices.
Any Plan B?

d) install ddrescue - sudo apt install ddrescue
e) use ddrescue to perform a bit-for-bit copy FROM the source HDD to the target HDD. Don't get those backwards. ddrescue is designed to get as much data as possible from storage. The target drive will be completely wiped. The target drive needs to be at least as large as the source drive.

If that is the case, I don't even a HDD that is as large as the current one.
If 'ddrescue' will copy the entire failing HDD, regardless of the used sectors, then I can't offered that.
If it will only move the used sectors, I can handle this.

Let ddrescue run over night - perhaps longer. There is output showing progress of the copying and how the errors are being handled. Multiple passes can be used to attempt to get more data from difficult sectors copied.

Yes, I will leave it overnight, that in case it will work. Let's see.

Just because ddrescue can copy the data, that doesn't mean it is fully recovered. On really bad disks, the drive might fail. This was going to happen already. All storage devices fail. Our goal is to have good backups BEFORE that happens. For really important data, we need it stored in 3 different places.

I deserve this. I have learned the lesson the hardest way.

If you do get a copy onto a new HDD, that is now the primary storage for that data and you need a new, fresh, backup ASAP. Cloning isn't really an efficient backup. It is a way to move from one disk to another, if you don't have a better option. There are many, many, better options than cloning, unless the source HDD is failing. Then the options are limited - or cost $3000.

3,000$?
I can buy 3 new laptops with 3 new eternal HDD!
Oh well, I am learning the lesson yet again the hardest possible way.

I am not sure if this is going to work or not?
One way to find out.

However, I just need to know:
If I have no laptop that support (2) storage devices, what am I supposed to do?

[TheFu]

Hi and thank you for your reply.

Nobody is born knowing these things. It took me over a decade as a programmer before most of this sunk into my bonehead. Then it took a few massive data-loss events before I actually did something - which was not perfect, but better than nothing. Then I had an other massive data loss event and got backup religion. Since that time, I've never lost more than 1 day of new data.

At the beginning, I thought it was the RAM. Never thought it could be the HDD. Never thought a machine that was running only for 10 weeks might fail so miserably like this.

Run time says nothing about treatment. If it has been sitting on a comfy shelf with no vibrations and was never moved, that's very different than riding a bus or in a subway or in a backpack. Hibernation, standby, and other modes that don't completely shutdown the OS and remove power could have an impact too. IDK. When ever I move my laptop from 1 building to another - driving, walking, whatever, it is powered off, 100%, completely.

I don't have any machine here that support (2) storage devices.
Any Plan B?

How is this possible? No USB ports? Really?

If that is the case, I don't even a HDD that is as large as the current one.
If 'ddrescue' will copy the entire failing HDD, regardless of the used sectors, then I can't offered that.
If it will only move the used sectors, I can handle this.

Clonezilla and all imaging bit-for-bit tools work this way. So does ddrescue. They are dumb. They want to copy 1 byte at a time from Disk A --> Disk B. So ... disk B needs to be at least as large as Disk A, the source. No way around that when data recovery is the goal. Now, if you are doing file-based backups, then just the amount of storage necessary to hold the files is required. Cloning a disk to an image file can be sent through compression tools, but as new as you appear to be, I'm loath to attempt to explain that. It would be a 90% chance of failure.

Yes, I will leave it overnight, that in case it will work. Let's see.

If your target disk isn't at least as large as the source, don't.

I deserve this. I have learned the lesson the hardest way.

We all do. That's life. Learn from this.
Story ... not important:

3,000$?
I can buy 3 new laptops with 3 new eternal HDD!
Oh well, I am learning the lesson yet again the hardest possible way.

$3000 is for a professional data recovery service. They many need to replace controllers, platters, most of the insides before they can attempt recovery, assuming the media/platters aren't harmed already.

I am not sure if this is going to work or not?
One way to find out.

Yep. But it probably won't if the target disk isn't larger than the source disk. You could try file-based backups - and I would. We don't really know what the problem is. If it is the motherboard, then you'll need to borrow someone else's computer for this stuff. BTW, you can run a desktop on a $50-raspberry Pi, if you are really stuck for cash.

We zoomed in on the HDD being the issue based on the subject of this thread. Did you happen to check the SMART data yet? That is normally how people check that. It could be just a bad cable too. SMART usually shows that as ECC errors and Read-Errors.

However, I just need to know:
If I have no laptop that support (2) storage devices, what am I supposed to do?

Again - your laptop doesn't have USB ports? You can put almost any storage HDD/SSD into a cheap external USB enclosure. I've seen these for $9. I have a number of them - all sorts. I prefer the "Dock" type, which support 2 HDDs over 1 USB3 connection. Then I can load raw drives into the "dock" and gravity holds it in place. Works with 3.5 and 2.5 inch SATA drives. They run as fast as the drive would internally (almost always), since the "dock" has an external power and USB3 bus speeds are faster than all but the fastest NVMe drives anyway. Think I saw one of these dual Docks for $25 a few months ago during a sale. That's USD.

I'm not convinced the HDD is really the issue. A cracked motherboard (physically cracked) can create short circuits.

Story ... not important:
I've seen this a few times over my career. I once dropped my personal laptop about 8 inches from a low couch to the floor. That system never booted again. The motherboard power connection was harmed. It was just over 1 yr old, so out of warranty. At the time, my credit card doubled the warranty on purchases, so I just needed to provide an official estimate to fix the problem and they'd reimburse me. It was a $850 laptop. They gave me $650, since it has been used over a year. I shopped about for a replacement about 4 months - I have multiple computers and didn't need a laptop at the time. This was in 2011 ... and I still have that replacement laptop today. It is on the 3rd HDD now and I use it almost every day. It hasn't moved from the location since about 2013 when I moved to a very light, 2lb, laptop.

[amjjawad]

Screenshot from 2021-05-19 10-40-19.png

Is this normal?
Have you ever seen something like this before?!

P.S.
I'll get back to the replies on this thread hopefully soon. I'm not ignoring anything.

[amjjawad]

Nobody is born knowing these things. It took me over a decade as a programmer before most of this sunk into my bonehead. Then it took a few massive data-loss events before I actually did something - which was not perfect, but better than nothing. Then I had an other massive data loss event and got backup religion. Since that time, I've never lost more than 1 day of new data.

Learn by doing, +1 for that approach. However, it is really painful. I had stress and anxiety over this issue, that is why I didn't reply earlier.

Run time says nothing about treatment. If it has been sitting on a comfy shelf with no vibrations and was never moved, that's very different than riding a bus or in a subway or in a backpack. Hibernation, standby, and other modes that don't completely shutdown the OS and remove power could have an impact too. IDK. When ever I move my laptop from 1 building to another - driving, walking, whatever, it is powered off, 100%, completely.

I don't know but I think the excessive use of Hibernate might be the real cause, not sure and can't verify that.

Lesson learned. I'll not use Hibernate ever again whether it was the cause or not!

How is this possible? No USB ports? Really?

I thought you were referring to internal '2nd' drive.
Yes, of course I have USB ports.
However, I don't have anything (hardware) to connect any drive to the laptop via the USB ports.
I'm trying to use whatever I have at home instead of buying new stuff.

Clonezilla and all imaging bit-for-bit tools work this way. So does ddrescue. They are dumb. They want to copy 1 byte at a time from Disk A --> Disk B. So ... disk B needs to be at least as large as Disk A, the source. No way around that when data recovery is the goal. Now, if you are doing file-based backups, then just the amount of storage necessary to hold the files is required. Cloning a disk to an image file can be sent through compression tools, but as new as you appear to be, I'm loath to attempt to explain that. It would be a 90% chance of failure.

I'm not new but I've never faced this very issue ever before.
Since 1999 until now, never had to deal with a similar situation!

I personally prefer file-based recovery or backup.
However, in my case, I am not sure if that is even possible or not?!

If your target disk isn't at least as large as the source, don't.

Yes, until this moment, I have no drive that is as large as the dying/dead drive.

We all do. That's life. Learn from this.

Undoubtedly, I will sure do.

$3000 is for a professional data recovery service. They many need to replace controllers, platters, most of the insides before they can attempt recovery, assuming the media/platters aren't harmed already.

I found a company here in Australia and I'll try to call them tomorrow. At least, a rescue plan!

Yep. But it probably won't if the target disk isn't larger than the source disk. You could try file-based backups - and I would. We don't really know what the problem is. If it is the motherboard, then you'll need to borrow someone else's computer for this stuff. BTW, you can run a desktop on a $50-raspberry Pi, if you are really stuck for cash.

I took the HDD out, connected (internally) to another laptop. Same problems. Can't do much.
I posted a screenshot on my previous comment.

By 'file-based backup', are we talking about copying and pasting?
If yes, that failed and didn't work.

We zoomed in on the HDD being the issue based on the subject of this thread. Did you happen to check the SMART data yet? That is normally how people check that. It could be just a bad cable too. SMART usually shows that as ECC errors and Read-Errors.

I posted screenshots earlier from 'Disk' program/application, unless you need me to follow a different path?

Again - your laptop doesn't have USB ports? You can put almost any storage HDD/SSD into a cheap external USB enclosure. I've seen these for $9. I have a number of them - all sorts. I prefer the "Dock" type, which support 2 HDDs over 1 USB3 connection. Then I can load raw drives into the "dock" and gravity holds it in place. Works with 3.5 and 2.5 inch SATA drives. They run as fast as the drive would internally (almost always), since the "dock" has an external power and USB3 bus speeds are faster than all but the fastest NVMe drives anyway. Think I saw one of these dual Docks for $25 a few months ago during a sale. That's USD.

No internal slot for yet another drive but it sure does have USB ports.
The main issue here is, I have no such things as you mentioned. Even if I order that online, it will take days to arrive. I don't go out usually unless it's urgent. If that is our one and only option, I must find where these things can be bought.
I prefer to use whatever available I have here first.

I'm not convinced the HDD is really the issue. A cracked motherboard (physically cracked) can create short circuits.

But, no one and nothing happened to the motherboard as far as I'm aware.

Story ... not important:
I've seen this a few times over my career. I once dropped my personal laptop about 8 inches from a low couch to the floor. That system never booted again. The motherboard power connection was harmed. It was just over 1 yr old, so out of warranty. At the time, my credit card doubled the warranty on purchases, so I just needed to provide an official estimate to fix the problem and they'd reimburse me. It was a $850 laptop. They gave me $650, since it has been used over a year. I shopped about for a replacement about 4 months - I have multiple computers and didn't need a laptop at the time. This was in 2011 ... and I still have that replacement laptop today. It is on the 3rd HDD now and I use it almost every day. It hasn't moved from the location since about 2013 when I moved to a very light, 2lb, laptop.

It's important to me because you are sharing your stories with me. It means a lot, thank you so much.

[TheFu]

The key to file-based backups is running them BEFORE htey are needed and for them to be "versioned." That means a cp isn't the best choice. Making a weekly "cp" backup means you need as much backup storage as 5-200x the source files. That's not very efficient. With a real backup tool, it is easy to have 90 days of versioned backups that uses 20-30% more storage than the original. Also, each increment backup takes just a few minutes, so it isn't a big deal and you'll allow them to run more often. For example, last night backups for 3 system here:

=== Time for Backups to regulus === 
StartTime 1621401303.00 (Wed May 19 01:15:03 2021)
EndTime 1621401497.20 (Wed May 19 01:18:17 2021)
ElapsedTime 194.20 (3 minutes 14.20 seconds)
TotalDestinationSizeChange 196530021 (187 MB)


=== Time for Backups to hadar === 
StartTime 1621402504.00 (Wed May 19 01:35:04 2021)
EndTime 1621403066.37 (Wed May 19 01:44:26 2021)
ElapsedTime 562.37 (9 minutes 22.37 seconds)
TotalDestinationSizeChange 81611277 (77.8 MB)


=== Time for Backups to romulus === 
StartTime 1621403127.00 (Wed May 19 01:45:27 2021)
EndTime 1621403261.35 (Wed May 19 01:47:41 2021)
ElapsedTime 134.35 (2 minutes 14.35 seconds)
TotalDestinationSizeChange 142401 (139 KB)

My backups are all automatic. My laptop has been powered down the last few days, or I'd show it, but ... here are the backups for the last 90+ days:

        Time                       Size        Cumulative size
-----------------------------------------------------------------------------
Sun May 16 00:07:04 2021         24.5 GB           24.5 GB   (current mirror)
Sat May 15 00:07:03 2021         15.9 MB           24.5 GB
Fri May 14 00:07:03 2021         17.5 MB           24.6 GB
Wed May 12 00:07:04 2021         15.7 MB           24.6 GB
...
Wed Feb 17 00:07:03 2021         47.6 MB           29.5 GB
Tue Feb 16 00:07:03 2021         32.2 MB           29.6 GB
Mon Feb 15 00:07:04 2021         69.0 MB           29.6 GB

Basically, I backup 24.5G of data and all the daily versions of that data going back to Feb 15 only use 29.6G. When you see this, doesn't it seem like a no-brainer choice to run daily, versioned, backups? The trick, of course, is to pick the data/information to be backed up. I can restore any files or directories or a complete set from any of those dates. The last backup version looks like a mirror, so getting the latest copy of a file is just a cp .... or copy/paste in a GUI. I've used rsync for restores. The backup tool I use doesn't use hidden file types or funky tools. It uses rsync + gzip + diff. Standard tools, so if I really need the data, it is there. Of course, using the --restore-as-of {DATE} option for any older versions is much easier.

My backups for systems at home all go to a single, USB3 1.5TB HDD, sitting in a $20 Dock. Considering that a 2TB USB HDD is only $45 these days, seems like cheap insurance to me. I don't backup OSes, just the settings and config data to put that all back in about 30-45min for each system. If my 10 systems all died at the same time, it would be a bad day. 30min x 10 ... a full day, once replacement storage is installed. I'd certainly need to prioritize. So far, only 1 system has ever been impacted at a time the last 30 yrs.

Anyway, enough about that. It doesn't help today.

Temperature changes can cause lose connections on all electronics. Stuff designed to work over wider temperatures uses larger connections, more solder. Any little bump can make an iffy connection just lose enough to stop working.

For SMART data, I use smartctl. I don't trust "disks". Plus, since I need to run smart testing weekly and long smart testing monthly on all the storage here, a GUI isn't exactly efficient. If we know how to do things from the CLI/shell, then we don't have to re-re-re-relearn the new GUIs every 2-3 yrs. But I'm not against all GUI tools. gparted completely rocks!

As for trying to use stuff you already have, that's great unless it don't allow doing what is necessary. Clearly, this has been unsolved for 6 days already, so the data really isn't that important.

Another story:
One of my work systems (I was the project enterprise architect) was down for 6 hrs due to database corruption. 25K workers couldn't work. It cost the company about $50M in lost work and lost client implementation schedules. What's sad is that the system design would have made it a 10 minute outage with no more than 1 hour of data loss, had the client implemented that part. The client had already PAID for everything, but just didn't allow the hourly BCV snapshots to be setup, created, tested during a maintenance window. They always prioritized anything else higher - new features, other parts of the system, just not the BCV snapshots. We were 3 yrs into the implementation. Nearly every weekend, something new was being added - I think they didn't do changes when there were 5 huge holidays, but that was about it. How'd you like to be the admin on that - every weekend - 11pm-6am, having to make system changes. Glad I wasn't in operations. The IT guys were great, but never really had any weekend off.
Even after the outage where it became clear who's fault it was, they still didn't prioritize doing it for over 3 months. And rather than doing it all in 1 night, which was very possible, they decided to do it the way that caused the 3 full nights of effort and had almost as much risk.
This outage had visibility to the company leadership for a few weeks.

When you call that data recovery company, I hope they can help, but it won't be $200, unless nothing is really wrong with the hardware. It is going to be expensive.