Using sfdisk to fix partition table problems

[unutbu]

Introduction:

This is not a tutorial for beginners. Using sfdisk incorrectly can make a computer unbootable, or your data inaccessible. Please be careful.

Occasionally people end up with a partition table where a primary partition is located inside of an extended partition. At least from GParted's point of view, this is a heinous error which makes the entire partition table invalid. If you try to run GParted on this drive, it will simply show the entire drive as unallocated space. (See attached image, below.)

This can be quite worrisome if you know you have data on the drive, and just can't access it. It can also cause boot and installation problems.

Often people don't realize there is a problem with their partition table because Windows may seem to deal with it just fine. (In fact, the problem is most likely caused by a maverick Windows partition editor which allowed the user to write a partition table which did not conform to specifications).

Below I describe how sfdisk can be used to fix this problem.

Although I only show how to fix this single problem, the technique is generalizable. Once you understand how sfdisk works, you can use it to do just about anything fdisk can do. (One thing sfdisk can not do is change C/H/S disk geometry.)

The attraction of sfdisk is that you can ask someone to post their partition table as a file, you can edit and repost that file, then tell the person a single command to fix their partition table.

Please note I am not an expert. Most of what I know comes from watching caljohnsmith and meierfra use sfdisk:

http://ubuntuforums.org/showthread.php?t=1095606
http://ubuntuforums.org/showthread.php?t=1008458
http://ubuntuforums.org/showpost.php...6&postcount=22
http://ubuntuforums.org/showthread.php?t=1036239
http://ubuntuforums.org/showpost.php...&postcount=716
http://art.ubuntuforums.org/showthre...224351&page=74
http://ubuntuforums.org/showpost.php...&postcount=847

Many thanks to caljohnsmith and meierfra for generously sharing their knowledge. All mistakes are my own. I welcome corrections.

Background:

• Each drive has 1 partition table.
  
• A partition table can have a maximum of 4 primary partitions. If the drive is called sdc, the the primary partitions are called sdc1, sdc2, sdc3, sdc4.
  
• A partition table can have at most 1 extended partition. The extended partition must also have a name whose numerical part is between 1 and 4: that is, the extended partition must be named sdc1 or sdc2 or sdc3 or sdc4.
  
• Logical partitions always have device names whose numerical part is greater than or equal to 5. (e.g. sdc5, sdc6, etc.)
  
• The partition table is located at sectors 447--512 on the drive.
• A sector = 512 bytes.
  
• You can save the partition table in its native binary format with the command
  

  sudo dd if=/dev/sdc of=PT_sdc.img bs=1 count=66 skip=446

• and you can restore the partition table with the command
  

  sudo dd of=/dev/sdc if=PT_sdc.img bs=1 count=66 skip=446

  I mention this so you can have a picture in your mind about where the partition table is located. We won't be using dd to manipulate the partition table, however. We'll use sfdisk instead.

The sfdisk commands:

You can save the partition table in an ascii format with the command

sudo sfdisk -d /dev/sdc > PT.txt

This saves the partition table on /dev/sdc to a file called PT.txt.
What's particularly lovely is that is file is in ASCII format.

You can edit it in a normal text editor, then tell sfdisk to write a new partition table based on our edited PT.txt:

sudo sfdisk --no-reread -f /dev/sdc -O PT.save < PT.txt

"--no-reread" means don't check if disk is unmounted
-f force
"-O PT.save" means save a backup of original partition table in PT.save. PT.save is in binary format.

To restore the partition table using PT.save:

sudo sfdisk --force -I PT.save /dev/sdc

An example use case:
We start with a valid partition table:

sudo fdisk -l /dev/sdc

Disk /dev/sdc: 2004 MB, 2004877312 bytes
255 heads, 63 sectors/track, 243 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0006aba6

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1          80      642568+   b  W95 FAT32
/dev/sdc2              81         160      642600   83  Linux
/dev/sdc3             161         243      666697+   5  Extended
/dev/sdc5             161         200      321268+   7  HPFS/NTFS
/dev/sdc6             201         243      345366   82  Linux swap / Solaris

Here we dump the partition table to PT.txt:

sudo sfdisk -d /dev/sdc > PT.txt
cat PT.txt 

# partition table of /dev/sdc
unit: sectors

/dev/sdc1 : start=       63, size=  1285137, Id= b
/dev/sdc2 : start=  1285200, size=  1285200, Id=83
/dev/sdc3 : start=  2570400, size=  1333395, Id= 5
/dev/sdc4 : start=        0, size=        0, Id= 0
/dev/sdc5 : start=  2570463, size=   642537, Id= 7
/dev/sdc6 : start=  3213063, size=   690732, Id=82

Now we do some math to find the end points:

        start         end    math   
sdc1       63     1285200    63+1285137=1285200
sdc2  1285200     1413722    1285200+128522=1413722
sdc3  2570400     3903795    2570400+1333395=3903795
sdc5  2570463     3213000    2570463+642537=3213000
sdc6  3213063     3903795    3213063+690732=3903795

If you wish to edit PT.txt, here are some things you should note:

• sdc3 is an extended partition. sdc5 and sdc6 are logical partitions that sit inside sdc3. You can confirm that by looking at the start and end positions of the three partitions.
  
• there is no partition called sdc4.
  Notice that sfdisk writes (in PT.txt) a blank entry in this case:
  

  /dev/sdc4 : start=        0, size=        0, Id= 0

• sdc3 starts at 2570400 but sdc5 starts at 2570463. There is a 63 sector gap between the start of an extended partition and the start of a logical partition.
  
• sdc5 ends at 3213000 and sdc6 starts at 3213063. Again, there is a 63 sector gap between the end of a logical partition and the start of the next logical partition.
  
• In contrast, notice that there is no gap at all between primary partitions.
  It is possible to have larger gaps between partitions, but never smaller than 63 sectors for logical partitions, (and obviously) never smaller than 0 for primary partitions. Partitions must not overlap.

Now we create a messed up partition table:

cat PT_messed_up.txt

# partition table of /dev/sdc
unit: sectors

/dev/sdc1 : start=       63, size=  1285137, Id= b
/dev/sdc2 : start=  1285200, size=  1285200, Id=83
/dev/sdc3 : start=  2570400, size=  1333395, Id= 5
/dev/sdc4 : start=  2570463, size=   642537, Id= 7
/dev/sdc5 : start=        0, size=        0, Id= 0
/dev/sdc6 : start=  3213063, size=   690732, Id=82

I edited PT_messed_up.txt by changing sdc5 to sdc4.

This makes a partition inside of an extended partition into a primary partition.
This is a no-no.

Here we write the messed up partition table to disk:

sudo sfdisk --no-reread -f /dev/sdc -O PT.save < PT_messed_up.txt
...
Successfully wrote the new partition table

Now we are in a pickle. How do we fix it?

Or, imagine you find a partition table in this invalid state. Below I show how to fix it.

sudo fdisk -l /dev/sdc

omitting empty partition (5)

Disk /dev/sdc: 2004 MB, 2004877312 bytes
255 heads, 63 sectors/track, 243 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0006aba6

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1          80      642568+   b  W95 FAT32
/dev/sdc2              81         160      642600   83  Linux
/dev/sdc3             161         243      666697+   5  Extended
/dev/sdc4             161         200      321268+   7  HPFS/NTFS
/dev/sdc5             201         243      345366   82  Linux swap / Solaris

Notice that "sudo fdisk" reports

omitting empty partition (5)

Whenever you see this, it means there is an error in the partition table.

By virtue of having a number between 1 and 4, sdc4 is a primary partition.
It starts at cylinder 161, inside the extended partition sdc3. This is an error.

GParted refuses to operate on invalid partition tables:

gksu gparted /dev/sdc

======================
libparted : 1.8.9
======================
Can't have overlapping partitions.

See the attached image below.

GParted displays the entire drive as being unallocated.

Given PT_messed_up.txt, we need to move the primary partition sdc4 to a logical partition. Since the name sdc5 is free, we could simply change "sdc4" to "sdc5" in PT_messed_up.txt.
Indeed, this gives us back PT.txt.

So, to fix the problem, we simply run

sudo sfdisk --no-reread -f /dev/sdc < PT.txt

sudo fdisk -l /dev/sdc

Disk /dev/sdc: 2004 MB, 2004877312 bytes
255 heads, 63 sectors/track, 243 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0006aba6

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1          80      642568+   b  W95 FAT32
/dev/sdc2              81         160      642600   83  Linux
/dev/sdc3             161         243      666697+   5  Extended
/dev/sdc5             161         200      321268+   7  HPFS/NTFS
/dev/sdc6             201         243      345366   82  Linux swap / Solaris

Here we see the partition table is back in a valid state, and GParted successfully parses the partition table and once again allows us to operate on it.

If we make a mistake in PT.txt, we can revert to the previous partition table
by running

sudo sfdisk --force -I PT.save /dev/sdc

Caveats:
sfdisk can be finicky about the format of PT.txt.
Spaces and end-of-line characters are important to sfdisk.
Here are some technical caveats:

• In PT.txt, you can't have trailing spaces at the ends of lines, after the "ID" field
• In PT.txt, the end of lines (EOLs) have to be unix-style line feeds (LF), and can't be Windows/DOS style carriage return + line feed (CR+LF).
• You can check for either problem by doing "cat -A partition_table.txt"
  It puts a $ at the end of lines that end with unix-style LF and
  it puts an M at the end of lines that end with Windows-style CR+LF and

Some further info:
Example of using sfdisk to change a logical partition into a primary partition:
http://ubuntuforums.org/showthread.php?t=1036239

Example of using sfdisk to change a primary partition into a logical partition:
http://ubuntuforums.org/showpost.php...&postcount=847

Here is the specification of the partition table format:
http://www.win.tue.nl/~aeb/partition...bles.html#toc3

[wieman01]

Nice, unutbu. Thank you!

[tedrogers]

CAVEAT: Readers might want to read my initial post concerning my goals. 

It should help shed light on some of what I'm saying below.

http://ubuntuforums.org/showthread.php?t=1366638

Hi all (and hi there Wieman01...it's been a while!),

I'm trying to reorder my partitions using sfdisk, and I would welcome some assistance so I get it right.

This is the output of my PT.txt:

# partition table of /dev/sda
unit: sectors

/dev/sda1 : start=       63, size= 20964762, Id=83, bootable
/dev/sda2 : start=115105725, size=  2104515, Id=82
/dev/sda3 : start=        0, size=        0, Id= 0
/dev/sda4 : start= 20964825, size= 94140900, Id=83

As you can see, /sda2 is at the end of the disk (this is my swap space), /sda3 has no physical size, and /sda4 (my /home space) is before /sda2 (swap) and after /sda1 (boot).

What I want to do is remove sda3 completely, physically move sda2 on the disk to just after the boot space, and move sda4 to directly after the swap space and rename it to sda3.

To do all the physical moving of partitions (i.e. move sda2 and sda4), I will use gparted, but to rename them can I use sfdisk to change the partition table like this (assuming that it would have new START and SIZE information based on the fact I had moved the partitions using gparted):

# partition table of /dev/sda
unit: sectors

/dev/sda1 : start=       63, size= 20964762, Id=83, bootable
/dev/sda2 : start=115105725, size=  2104515, Id=82
/dev/sda3 : start= 20964825, size= 94140900, Id=83

Thanks for your assistance. I'm happy to answer any questions you might have.

All the best.

[meierfra.]

You can reorder the partition with fdisk:

sudo fdisk /dev/sda
x  # enter expert mode
f  # fix  partition table
w  #  write the changes to disk

Reordering the partition table requires reinstall grub. Asked for help if needed.

Also instead of mowing the swap partition, I would just delete it. Then create a new swap partition after you moved /dev/sda3. That's much faster.

Only drawback, the UUID of the swap partition will change. But that is easy to fix. After you repartitioned and reordered

sudo mkswap -U the_old_uuid /dev/sda3

(you can look up the old UUID in /etc/fstab, or before repartitioning via "sudo blkid")

[wieman01]

Hi all (and hi there Wieman01...it's been a while!)...

Hey there, Ted! Nice seeing you around!

[tedrogers]

Okay, thanks for your replies!

If I do this (I haven't written the partition table yet), it produces this partition list:

Disk /dev/sda: 255 heads, 63 sectors, 7296 cylinders

Nr AF  Hd Sec  Cyl  Hd Sec  Cyl     Start      Size ID
 1 80   1   1    0 254  63 1023         63   20964762 83
 2 00 254  63 1023 254  63 1023   20964825   94140900 83
 3 00   0   0    0   0   0    0          0          0 00
 4 00 254  63 1023 254  63 1023  115105725    2104515 82

So as you can see I still have a zero size partition. How would I resolve this?

Also, is there any problem with leaving my swap at the end of the disk like it would be if I reordered the partitions using fdisk as described. I might just leave it at the end if it won't do any harm or hinder the performance.

Thanks.

[SecretCode]

That "partition of zero size" isn't a problem - there are 4 slots for primary partitions and you don't have to use them all. Some tools display that empty slots as zeroes.

[tedrogers]

That "partition of zero size" isn't a problem - there are 4 slots for primary partitions and you don't have to use them all. Some tools display that empty slots as zeroes.

That partition of zero size used to be a partition of 10GB.

Are you saying that if I rename sda4 as sda3 then it will no longer display that "slot"?

[SecretCode]

I'm confused. You only have three partitions shown in any of your posts in this threads.

What I'm saying is that sfdisk will always display a row for every one of the 4 possible primary partitions. E.g. here's one of my external drives, which has only one partition:

$ sudo sfdisk -d /dev/sdc
# partition table of /dev/sdc
unit: sectors

/dev/sdc1 : start=       63, size=976768002, Id= 7
/dev/sdc2 : start=        0, size=        0, Id= 0
/dev/sdc3 : start=        0, size=        0, Id= 0
/dev/sdc4 : start=        0, size=        0, Id= 0

I can't see any reason why you need to 'rename' sda4 to be sda3, or have the partition table entries in disk order. What are you trying to achieve?

[tedrogers]

I'm confused. You only have three partitions shown in any of your posts in this threads.

What I'm saying is that sfdisk will always display a row for every one of the 4 possible primary partitions. E.g. here's one of my external drives, which has only one partition:

$ sudo sfdisk -d /dev/sdc
# partition table of /dev/sdc
unit: sectors

/dev/sdc1 : start=       63, size=976768002, Id= 7
/dev/sdc2 : start=        0, size=        0, Id= 0
/dev/sdc3 : start=        0, size=        0, Id= 0
/dev/sdc4 : start=        0, size=        0, Id= 0

I can't see any reason why you need to 'rename' sda4 to be sda3, or have the partition table entries in disk order. What are you trying to achieve?

Ahh, I see what you mean now.

This is what I'm aiming for...it's quite fussy really...but that's how we learn.

http://ubuntuforums.org/showthread.php?t=1366638