Recover Lost or Corrupted GPT Partition

[Aritsugu]

Good Morning, Afternoon, Evening... delete as appropriate

I was hoping someone might take a look at a problem I'm having with an old drive I re-purposed to shuffle some data around on. Sadly it won't mount as it did before and disk utility (Palimpsest) is saying there is just free space, no partition. I've been reading up on this stuff myself, but, I think I'm missing something, or I'm out of luck.

Not sure how this has happened, drive was formatted with a guid table and a single ext4 partition created on it, all within disk utility (Palimpsest). I moved the data over to it, powered down, moved drive to new machine, booted, partition gone, returned to first machine, still missing.

Note this isn't the boot disk.

The following sums up my attempts so far (not including my fruitless foray with testdisk)

gdisk (gpt fdisk) and fdisk with the following results:

dan@zeus:~$ sudo gdisk -l /dev/sda
[sudo] password for dan: 
GPT fdisk (gdisk) version 0.8.4

Warning! Disk size is smaller than the main header indicates! Loading
secondary header from the last sector of the disk! You should use 'v' to
verify disk integrity, and perhaps options on the experts' menu to repair
the disk.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! One or more CRCs don't match. You should repair the disk!

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: damaged

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
Disk /dev/sda: 625140335 sectors, 298.1 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 4FF348B9-D041-49A6-AD98-18C15F055F2D
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 625142414
Partitions will be aligned on 8-sector boundaries
Total free space is 0 sectors (0 bytes)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              34       625142414   298.1 GiB   0700  
dan@zeus:~$ sudo fdisk -lu /dev/sda

WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sda: 320.1 GB, 320071851520 bytes
255 heads, 63 sectors/track, 38913 cylinders, total 625140335 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1   625142447   312571223+  ee  GPT

Delving further into gdisk and following its suggestion I selected the 'v' (verify) option:

Command (? for help): v

Caution: The CRC for the backup partition table is invalid. This table may
be corrupt. This program will automatically create a new backup partition
table when you save your partitions.

Problem: The secondary header's self-pointer indicates that it doesn't reside
at the end of the disk. If you've added a disk to a RAID array, use the 'e'
option on the experts' menu to adjust the secondary header's and partition
table's locations.

Problem: Disk is too small to hold all the data!
(Disk size is 625140335 sectors, needs to be 625142448 sectors.)
The 'e' option on the experts' menu may fix this problem.

Problem: partition 1 is too big for the disk.

Caution: Partition 1 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.

Consult http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/
for information on disk alignment.

Identified 4 problems!

Its clearly a mess, for more info I've used the 'i' and 'p' options:

Command (? for help): i
Using 1
Partition GUID code: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Microsoft basic data)
Partition unique GUID: 9E79F802-02B9-4097-8CD2-F9FFE96136BD
First sector: 34 (at 17.0 KiB)
Last sector: 625142414 (at 298.1 GiB)
Partition size: 625142381 sectors (298.1 GiB)
Attribute flags: 0000000000000000
Partition name: ''

Command (? for help): p   
Disk /dev/sda: 625140335 sectors, 298.1 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 4FF348B9-D041-49A6-AD98-18C15F055F2D
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 625142414
Partitions will be aligned on 8-sector boundaries
Total free space is 0 sectors (0 bytes)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              34       625142414   298.1 GiB   0700

Following the suggestion of the 'v' (verify) command which said to use the 'e' option from the experts menu:

Command (? for help): x

Expert command (? for help): e
Relocating backup data structures to the end of the disk

Expert command (? for help): w

Warning! Secondary partition table overlaps the last partition by
2113 blocks!
You will need to delete this partition or resize it in another utility.

Problem: partition 1 is too big for the disk.
Aborting write operation!
Aborting write of new partition table.

Since I'm not going to follow the advice to delete the partition (I need the data) I'll have a go at resizing it:

root@zeus:/home/brendan# parted /dev/sda
GNU Parted 2.3
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print                                                            
Error: Invalid argument during seek for read on /dev/sda                  
Retry/Ignore/Cancel? R                                                    
Error: Invalid argument during seek for read on /dev/sda                  
Retry/Ignore/Cancel? R                                                    
Error: Invalid argument during seek for read on /dev/sda                  
Retry/Ignore/Cancel? I                                                    
Error: The backup GPT table is corrupt, but the primary appears OK, so that will be used.
OK/Cancel? OK                                                             
Backtrace has 8 calls on stack:
  8: /lib/libparted.so.0(ped_assert+0x31) [0x7f8e8fe0c351]
  7: /lib/libparted.so.0(+0x444c1) [0x7f8e8fe3d4c1]
  6: /lib/libparted.so.0(ped_disk_new+0x75) [0x7f8e8fe13505]
  5: parted() [0x4070de]
  4: parted(interactive_mode+0xf3) [0x40e183]
  3: parted(main+0x8f) [0x40b1df]
  2: /lib/libc.so.6(__libc_start_main+0xfd) [0x7f8e8f62bc8d]
  1: parted() [0x405709]
                                                                          

You found a bug in GNU Parted! Here's what you have to do:

Don't panic! The bug has most likely not affected any of your data.
Help us to fix this bug by doing the following:

Check whether the bug has already been fixed by checking
the last version of GNU Parted that you can find at:

    http://ftp.gnu.org/gnu/parted/

Please check this version prior to bug reporting.

If this has not been fixed yet or if you don't know how to check,
please visit the GNU Parted website:

    http://www.gnu.org/software/parted

for further information.

Your report should contain the version of this release (2.3)
along with the error message below, the output of

    parted DEVICE unit co print unit s print

and the following history of commands you entered.
Also include any additional information about your setup you
consider important.

Assertion (last_usable <= disk->dev->length) at ../../../libparted/labels/gpt.c:718 in function _parse_header() failed.

Aborted

Which causes parted to crash. Now I'm stuck, I also tried testdisk, and am willing to try anything to get this solved. I can see that the partition seems to be too big for the disk, so if I could just chop it smaller? I don't mind if that loses the files at the end as long as I get some of them back.

[oldfred]

I have not had to adventure into repairs of my gpt drives.

Have you backed up and tried these commands?

repair gpt:
http://www.rodsbooks.com/gdisk/repairing.html

[Aritsugu]

I did read the information on the link you provided. The most relevant part is "probably"

"If the original disk used an unusual partition table size, though, you'll need to resize your in-memory table before you can successfully load the table from disk."

I say "probably" as they might mean the size of the table or the partition, its the partition in my case that appears to be a unusual size.

Like I say I'm confused now, if I can resize the partition like the gdisk utility suggests, great, but parted crashes simply loading the information.

if I resize will it ruin the information on the drive or will it crop the end off? the later I can cope with.

I'm guessing I should 'dd' the drive. I'll work out the command, it'll have to be to a file as I don't have a spare drive.

I'm off to bed now, I'll check in again at 0900 UTC, thanks guys.

[Aritsugu]

Warning: don't do what I'm doing, because I don't have a clue what I'm doing!

Not sure if backing up a broken partition table makes sense, but I guess its all I've got so I guess I'll keep it... despite all the warnings it says it completed successfully so I guess I should trust it... (note its sgdisk I'm using, not sfdisk which didn't work on gpt disks).

After this I'm off to find/buy a spare harddrive to clone the broken disk to, I guess using ddrescue, something like:

ddrescue /dev/sda /dev/sdb logfile

However from the ddrescue manual :

"Note: you do not need to partition /dev/hdb beforehand, but if the partition table on /dev/hda is damaged, you'll need to recreate it somehow on /dev/hdb. "

This applies to me I guess, not sure what to do about it though. If I created a new partition etc on the new disk and copied the damaged one over would I be back in business? can I even copy the partition off the damaged drive?

Off to find a harddrive to clone onto, output of sgdisk partition table backup follows:

root@zeus:/home/dan# sgdisk -b sda-backup.gpt /dev/sda
Warning! Disk size is smaller than the main header indicates! Loading
secondary header from the last sector of the disk! You should use 'v' to
verify disk integrity, and perhaps options on the experts' menu to repair
the disk.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! One or more CRCs don't match. You should repair the disk!

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
The operation has completed successfully.

[Aritsugu]

Ok I found a dusty old drive to clone onto, smart is saying it has a uncorrectable sector, and a pending sector, but, hey worth a shot. Plus side its the same size as my problem drive.

I've followed the same approach to partitioning this new drive as the problem drive and then had a look at it with gdisk.

The disk utility has partitioned it the same, starting (34) and ending (625142414) on the same sectors. But this drive claims to have 625142448 sectors, which is enough to fit the old partition in, while the current drive is reporting 625140335 sectors, too small for the partition to end on 625142414. I'm guessing this is the problem?! The problem drive is no longer reporting the same number of sectors as it did when I partitioned it? Hence the backup partition table has been lost, causing the drive not to mount, the crc errors, can't be rewritten and etc.

Ok so I'm waiting for the result of:

root@zeus:/home/brendan# dd_rescue -l ddlogfile /dev/sda /dev/sdd

I realise this will overwrite the partition I just created but hey! hopefully because this drive is reporting that it is big enough for the partition it will be file and dandy...

we are at 6% (that's the royal we)

root@zeus:/home/brendan# gdisk /dev/sdd
GPT fdisk (gdisk) version 0.8.4

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sdd: 625142448 sectors, 298.1 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 58F7115C-D8E0-4D86-B93A-1421EE9F14B8
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 625142414
Partitions will be aligned on 8-sector boundaries
Total free space is 0 sectors (0 bytes)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              34       625142414   298.1 GiB   0700  

Command (? for help): i
Using 1
Partition GUID code: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Microsoft basic data)
Partition unique GUID: 472AD041-C293-4BDE-BF64-46E5A6AB964A
First sector: 34 (at 17.0 KiB)
Last sector: 625142414 (at 298.1 GiB)
Partition size: 625142381 sectors (298.1 GiB)
Attribute flags: 0000000000000000
Partition name: ''

Command (? for help): v

Caution: Partition 1 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.

Consult http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/
for information on disk alignment.

No problems found. 0 free sectors (0 bytes) available in 0
segments, the largest of which is 0 (0 bytes) in size.

Command (? for help): q

[oldfred]

I think that srs5694 (same fellow as the rodbooks site) said not to use dd with gpt partitions. Something about the internal structure of the partition and the gpt tables. You can use dd on the entire drive as then all the internal structures and tables are also copied.

[Aritsugu]

Thanks, oldfred

I think dd'ing /dev/sda to /dev/sdd where sda and sdd are both 320GB drives should copy all of the gpt and partition data (byte for byte?). creating an exact copy???

My only concern is I should have used dd not dd_rescue? as is suggested on the rodbooks site, also, the rodbooks repair site states w.r.t. dd

Note that if the backup disk is larger than the source, even by a single sector, the GPT backup data structures will be misplaced on the backup disk. This can make GPT data recovery harder, so you may need to work on the original disk rather than the backup.

Now I know it is bigger, by about 2000+ sectors but the backup GPT structures are already broken/missing on the original and need to be replaced anyway. I think. and since the old disk seemed to shrink (was possibly the same size as the new one initially ) and not allow the backup structures to be written to the end of the partition, the new drive being bigger is surely a good thing??

I'm enjoying making all this up as I go along.

the backup using dd_rescue is at 62% complete, so its going to be about another hour and a half before I can have a play with this.

PS. I'm surprised the log file is empty.

[oldfred]

I thought dd_rescue was just a front end for dd, so I would not worry about that.

Keep us posted. All I can offer is moral support as you are well past what I know.

[Aritsugu]

The following is the result of the dd_rescue command looks good to me...

root@zeus:/home/dan# dd_rescue -l ddlogfile /dev/sda /dev/sdd
dd_rescue: (info): ipos: 312570112.0k, opos: 312570112.0k, xferd: 312570112.0k
                   errs:      0, errxfer:         0.0k, succxfer: 312570112.0k
             +curr.rate:    17727kB/s, avg.rate:    29535kB/s, avg.load:  5.9%
dd_rescue: (info): /dev/sda (312570176.0k): EOF
Summary for /dev/sda -> /dev/sdd:
dd_rescue: (info): ipos: 312570176.0k, opos: 312570176.0k, xferd: 312570176.0k
                   errs:      0, errxfer:         0.0k, succxfer: 312570176.0k
             +curr.rate:     2897kB/s, avg.rate:    29535kB/s, avg.load:  5.9%

On the off chance I've checked the disk utility and its not saying anything new about the drive.

I'm going to try a windows trick, reboot....

[Aritsugu]

oldfred,
simply your presence let alone your technical and moral support is most welcome. During times of stress its generally advisable to have someone there for you. I'd have talked to the missus but since the photos of her sister's wedding are on this drive I thought it would be best to fix the problem, rather than discuss it

The good news is I think I've dodged a bullet on this one, I can mount the backup, and I can copy a file off it ...............

"sudo gdisk /dev/sdd" however is not entirely happy, noting that the disk guid in the backup header is different from the main header (see output below). The one in the main header is from the original 'problem' disk, whereas the backup header contains the guid for the new disk. Makes sense to me, dd didn't overwrite the backup header on the new disk as it finished copying the old 'shorter' disk first. Now I guess I can get rid of these warnings by over-writing the backup with the main header. However I would think that would leave me with two disks with the same disk guid? A problem perhaps?

First things first I'm going to copy the files off onto a newer drive, after that I don't care to fix it any further, both drives are going in the not trusted pile, for eventual sledgehammering, but once I'm sure the files are off, I could

a) use gdisk to overwrite the backup header on the backup drive and see what happens... will the warnings disappear?

EDIT: The following is wrong there isn't a resize partition option, there is a resize partition table option. (Aritsugu)
b) I could brave the resize option (I just noticed) on the gdisk utility to shrink the partition on the original drive so it fits on the disk, see if it mounts then, and how much data is lost....

Regardless that's for doing once the copying is done, see you in a few hours.

BTW I'm hoping that my degree of verbosity is going to be helpful rather than annoying, let me know if its the later...

Caution: The CRC for the backup partition table is invalid. This table may
be corrupt. This program will automatically create a new backup partition
table when you save your partitions.

Problem: main header's disk GUID (4FF348B9-D041-49A6-AD98-18C15F055F2D) doesn't
match the backup GPT header's disk GUID (58F7115C-D8E0-4D86-B93A-1421EE9F14B8)
You should use the 'b' or 'd' option on the recovery & transformation menu to
select one or the other header.

Caution: Partition 1 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.