18.04 How to repair BTRFS partitions from a live CD?

[Robbyx]

At the moment I am running off a live CD because I can not get into my OS.


Starting from my HD I get a diversion to the emergency mode before being able to log into Ubuntu.


The system partition and the separate home partition were both formatted to BTRFS because of its advanced checking and correction ability. I am not running a rade.


sudo btrfs check /dev/nvme0n1p1
Checking filesystem on /dev/nvme0n1p1
UUID: ff16fdbe-9a30-4c73-89a4-bf10cd1b1974
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
found 17087352832 bytes used, no error found
total csum bytes: 12078132
total tree bytes: 406142976
total fs tree bytes: 373637120
total extent tree bytes: 16924672
btree space waste bytes: 71695136
file data blocks allocated: 28555075584
referenced 15398830080



ubuntu@ubuntu:~$ sudo btrfs check /dev/nvme0n1p2
Checking filesystem on /dev/nvme0n1p2
UUID: 5bb8aa95-b657-4297-be5b-4352a09596b0
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
found 20158885888 bytes used, no error found
total csum bytes: 19491032
total tree bytes: 163266560
total fs tree bytes: 121552896
total extent tree bytes: 16056320
btree space waste bytes: 31167715
file data blocks allocated: 28492115968
referenced 19452403712




sudo btrfs scrub /dev/nvme0n1p1
btrfs scrub: unknown token '/dev/nvme0n1p1' btrfs scrub: unknown token '/dev/nvme0n1p1'

sudo btrfs scrub start -Bd /dev/nvme0n1p1
ERROR: '/dev/nvme0n1p1' is not a mounted btrfs device

sudo btrfs scrub start -Bd nvme0n1p1
ERROR: cannot check nvme0n1p1: No such file or directory
ERROR: 'nvme0n1p1' is not a mounted btrfs device




What is the correct syntax for this type of scrub?

What should I do next?

I thought of reinstalling Ubuntu but I would prefer to use btrfs's error correction rather than have to do a clean install, of just the system partition. I do not want wipe out the existing home unless I really have no choice

[1fallen]

First Check.
What dose this show:

btrfs scrub status /btrfs -d
If this does not work, start the repair procedure


btrfs check —repair /dev/<device_name>
btrfs scrub start -Bf /dev/<device_name>

If this does not fix the problem using the tools from a live installer might be a better choice.
As a last resort cleaning the transaction log can be done with:


btrfs rescue zero-log /dev/<device_name>

If a scrub does not correct errors, you can use the following mount option to roll back to a known good B-tree, given that the rest of the tree is available because of copy-on-write:


# mount –o recovery /dev/sda /btrfs

[Robbyx]

Of course thank you very much for your help. This is what has happened so far by running your proposals:

First check:


sudo btrfs scrub status -d /dev/nvme0n1p1
ERROR: '/dev/nvme0n1p1' is not a mounted btrfs device
ubuntu@ubuntu:~$


It wants it to be mounted? I had the same response to my 3rd set of code in my first post.

If this does not work, start the repair procedure
Code:

btrfs check —repair /dev/<device_name>
btrfs scrub start -Bf /dev/<device_name>


ubuntu@ubuntu:~$ sudo btrfs check -—repair /dev/nvme0n1p1
usage: btrfs check [options] <device>

Check structural integrity of a filesystem (unmounted).

Check structural integrity of an unmounted filesystem. Verify internal
trees' consistency and item connectivity. In the repair mode try to
fix the problems found.
WARNING: the repair mode is considered dangerous

-s|--super <superblock> use this superblock copy
-b|--backup use the first valid backup root copy
--force skip mount checks, repair is not possible
--repair try to repair the filesystem
--readonly run in read-only mode (default)
--init-csum-tree create a new CRC tree
--init-extent-tree create a new extent tree
--mode <MODE> allows choice of memory/IO trade-offs
where MODE is one of:
original - read inodes and extents to memory (requires
more memory, does less IO)
lowmem - try to use less memory but read blocks again
when needed
--check-data-csum verify checksums of data blocks
-Q|--qgroup-report print a report on qgroup consistency
-E|--subvol-extents <subvolid>
print subvolume extents and sharing state
-r|--tree-root <bytenr> use the given bytenr for the tree root
--chunk-root <bytenr> use the given bytenr for the chunk tree root
-p|--progress indicate progress
--clear-space-cache v1|v2 clear space cache for v1 or v2


ubuntu@ubuntu:~$ sudo btrfs scrub status -d /dev/nvme0n1p1
ERROR: '/dev/nvme0n1p1' is not a mounted btrfs device

ubuntu@ubuntu:~$ btrfs scrub start -Bf /dev/nvme0n1p1
ERROR: '/dev/nvme0n1p1' is not a mounted btrfs device

[1fallen]

EDIT:Do this first:
This might help show what filesystems you have and what disks are part of them.

btrfs fi show

Then if needed:

btrfs device ready [device]
Check device to see if it has all of its devices in cache for mounting
btrfs device stats [options] [path]|[device]
Show device IO error statistics
btrfs device usage [options] [path] [path..]
Show detailed information about internal allocations in devices.
btrfs device ready [device]
Check device to see if it has all of its devices in cache for mounting
btrfs device stats [options] [path]|[device]
Show device IO error statistics
btrfs device usage [options] [path] [path..]
Show detailed information about internal allocations in devices.
My only question is how did you get "nvme0n1p1", but it should get sorted with the options Above^^^
I only seen "sda1" in my short and limited usage.

[Robbyx]

Have I misunderstood the way the partition should have set up? The first 2 partitions were btrfs ie / and home. 3= extended, 4=btrfs for data, 5= linux-swap, 6=fat32, balance unallocated as it is an sdde drive.

I am finding the names as stated in Gparted.

[Robbyx]

I had no choice about the name of the btrfs partitions they were automatically allocated those long names when I created the partitions

[Robbyx]

I am unclear what to do next.

[Skaperen]

what do you get from this command: cat /proc/partitions

[1fallen]

Ok this is a bit different then, for my usage but did you ever show:

btrfs fi show

[Robbyx]

ubuntu@ubuntu:~$ btrfs fi show
ERROR: cannot open /dev/nvme0n1p1: Permission denied
ERROR: cannot open /dev/nvme0n1p2: Permission denied
ERROR: cannot open /dev/nvme0n1p4: Permission denied



ubuntu@ubuntu:~$ sudo btrfs fi show
Label: none uuid: ff16fdbe-9a30-4c73-89a4-bf10cd1b1974
Total devices 1 FS bytes used 15.91GiB
devid 1 size 32.60GiB used 18.55GiB path /dev/nvme0n1p1

Label: none uuid: 5bb8aa95-b657-4297-be5b-4352a09596b0
Total devices 1 FS bytes used 18.77GiB
devid 1 size 69.85GiB used 24.02GiB path /dev/nvme0n1p2

Label: 'Data' uuid: f5ab46ce-b3da-4e59-8f18-3418fe272a09
Total devices 1 FS bytes used 405.79GiB
devid 1 size 685.67GiB used 421.02GiB path /dev/nvme0n1p4




ubuntu@ubuntu:~$ cat /proc/partitions
major minor #blocks name

7 0 1820752 loop0
7 1 88964 loop1
7 2 35472 loop2
7 3 144260 loop3
7 4 2376 loop4
7 5 13300 loop5
7 6 14840 loop6
7 7 3796 loop7
259 0 937692504 nvme0n1
259 1 34179072 nvme0n1p1
259 2 73242624 nvme0n1p2
259 3 1 nvme0n1p3
259 4 718979072 nvme0n1p4
259 5 1772544 nvme0n1p5
259 6 178176 nvme0n1p6
8 0 244198584 sda
8 2 29750000 sda2
8 3 41667584 sda3
8 5 39936000 sda5
8 6 113150976 sda6
8 7 203776 sda7
8 8 17189888 sda8
8 16 732574584 sdb
8 17 676393984 sdb1
8 18 56178688 sdb2
11 0 1907584 sr0
ubuntu@ubuntu:~$

[Skaperen]

try the command this way: sudo btrfs fi show

[1fallen]

I need to narrow down a bit more, can you show me this:

sudo df -h /btrfs

[Robbyx]

ubuntu@ubuntu:~$ sudo btrfs fi show
Label: none uuid: ff16fdbe-9a30-4c73-89a4-bf10cd1b1974
Total devices 1 FS bytes used 15.91GiB
devid 1 size 32.60GiB used 18.55GiB path /dev/nvme0n1p1

Label: none uuid: 5bb8aa95-b657-4297-be5b-4352a09596b0
Total devices 1 FS bytes used 18.77GiB
devid 1 size 69.85GiB used 24.02GiB path /dev/nvme0n1p2

Label: 'Data' uuid: f5ab46ce-b3da-4e59-8f18-3418fe272a09
Total devices 1 FS bytes used 405.79GiB
devid 1 size 685.67GiB used 421.02GiB path /dev/nvme0n1p4



ubuntu@ubuntu:~$ sudo df -h /btrfs
df: /btrfs: No such file or directory
ubun

[Robbyx]

Some tests we have run give results for a btrfs system others say the system is not btrfs. Any ideas how to clear this up?

[1fallen]

Until I get firm reply in the terminal, I can't tell you what to do next:
Hopefully this will show something:

sudo btrfs filesystem df /home
I'm assuming that this below is where /home resides:

Label: 'Data' uuid: f5ab46ce-b3da-4e59-8f18-3418fe272a09
Total devices 1 FS bytes used 405.79GiB
devid 1 size 685.67GiB used 421.02GiB path /dev/nvme0n1p4

[Robbyx]

Hopefully this will show something:

sudo btrfs filesystem df /home
ERROR: not a btrfs filesystem: /home


I am running off a live CD therefore /home is different to the one on the HD.

I also ran, but should there be an args to indicate uuid?

$ sudo btrfs filesystem df /5bb8aa95-b657-4297-be5b-4352a09596b0
ERROR: cannot access '/5bb8aa95-b657-4297-be5b-4352a09596b0': No such file or directory


When I use Nautilus I can open up both the home and the system partition and they do not look damaged. Is it possible that the grub2 has become corrupted? I was surprised to note that I was being thrown into emergency mode on startup.


I'm assuming that this below is where /home resides:
Label: 'Data' uuid: f5ab46ce-b3da-4e59-8f18-3418fe272a09
Total devices 1 FS bytes used 405.79GiB
devid 1 size 685.67GiB used 421.02GiB path /dev/nvme0n1p4[

No that is a data storage partition. The home is in p2 the system in p1


sudo blkid
/dev/nvme0n1p1: UUID="ff16fdbe-9a30-4c73-89a4-bf10cd1b1974" UUID_SUB="8bbadc12-83ab-476b-92c3-a09c49dc1484" TYPE="btrfs" PARTUUID="36bdb116-01"
/dev/nvme0n1p2: UUID="5bb8aa95-b657-4297-be5b-4352a09596b0" UUID_SUB="12d8a1ae-9e45-4467-99d1-bae0e01b1c53" TYPE="btrfs" PARTUUID="36bdb116-02"
/dev/nvme0n1p4: LABEL="Data" UUID="f5ab46ce-b3da-4e59-8f18-3418fe272a09" UUID_SUB="62eb831e-e336-455d-9804-afed243f3c28" TYPE="btrfs" PARTUUID="36bdb116-04"

[1fallen]

Anything special you added to grub?

cat /etc/default/grub

EDIT: This is up to you to try, as I'm not sure if any good will come from it:


sudo mount /dev/nvme0n1p1 /mnt
ls /mnt
sudo umount /mnt
Don't umount untill From your Live session try to run a scrub again.
Even better if you could show me a Gparted screenshot of all that's on the Drive/s

[Robbyx]

]
Anything special you added to grub?

cat /etc/default/grub

No. Nothing special.

EDIT: This is up to you to try, as I'm not sure if any good will come from it:
[

sudo mount /dev/nvme0n1p1 /mnt
ls /mnt
sudo umount /mnt[/


ubuntu@ubuntu:~$ sudo mount /dev/nvme0n1p1 /mnt
ubuntu@ubuntu:~$ ls /mnt
@
ubuntu@ubuntu:~$ sudo btrfs filesystem df /mnt
Data, single: total=17.01GiB, used=15.54GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=1.51GiB, used=387.31MiB
GlobalReserve, single: total=30.94MiB, used=0.00B


Is the @ response to the c/L normal?




Don't umount untill From your Live session try to run a scrub again.

I will do that after you advise that the output shown above of the following is normal:
sudo btrfs filesystem df /mnt



Even better if you could show me a Gparted screenshot of all that's on the Drive/s


ubuntu@ubuntu:~$ sudo blkid
/dev/nvme0n1p1: UUID="ff16fdbe-9a30-4c73-89a4-bf10cd1b1974" UUID_SUB="8bbadc12-83ab-476b-92c3-a09c49dc1484" TYPE="btrfs" PARTUUID="36bdb116-01"
/dev/nvme0n1p2: UUID="5bb8aa95-b657-4297-be5b-4352a09596b0" UUID_SUB="12d8a1ae-9e45-4467-99d1-bae0e01b1c53" TYPE="btrfs" PARTUUID="36bdb116-02"
/dev/nvme0n1p4: LABEL="Data" UUID="f5ab46ce-b3da-4e59-8f18-3418fe272a09" UUID_SUB="62eb831e-e336-455d-9804-afed243f3c28" TYPE="btrfs" PARTUUID="36bdb116-04"
/dev/nvme0n1p6: LABEL="EFI" UUID="C649-2797" TYPE="vfat" PARTUUID="36bdb116-06"
/dev/sda2: UUID="0414cae8-b4fe-48d6-b06f-9509c3f9f8cb" TYPE="ext4" PARTUUID="00083267-02"
/dev/sda3: UUID="bf95a4fd-9e7a-4a06-afed-e82fdb82ac63" TYPE="ext4" PARTUUID="00083267-03"
/dev/sda5: LABEL="mydocs" UUID="8d99bfc8-90ab-4488-b7da-87e7676aff19" TYPE="ext4" PARTUUID="00083267-05"
/dev/sda6: LABEL="Data inc AV" UUID="1a2e8d10-7b93-4de1-b551-31db6a621276" TYPE="ext4" PARTUUID="00083267-06"
/dev/sda7: UUID="B1BA-7E06" TYPE="vfat" PARTUUID="00083267-07"
/dev/sdb1: LABEL="hitachi 750gb" UUID="3c4f4688-4bd7-4262-a2b8-86f71bbd6d0b" TYPE="ext4" PARTUUID="0008ffd0-01"
/dev/sdb2: LABEL="Deju_dup_backup" UUID="c79b648e-48c5-45db-949f-a835928f3816" TYPE="ext4" PARTUUID="0008ffd0-02"
/dev/sr0: UUID="2018-07-25-03-21-56-00" LABEL="Ubuntu 18.04.1 LTS amd64" TYPE="iso9660" PTUUID="663eb4c4" PTTYPE="dos"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"
/dev/loop4: TYPE="squashfs"
/dev/loop5: TYPE="squashfs"
/dev/loop6: TYPE="squashfs"
/dev/loop7: TYPE="squashfs"
/dev/nvme0n1p5: LABEL="SWAP" UUID="4ec66d43-75bf-4f9e-8f36-13275a0f3258" TYPE="swap" PARTUUID="36bdb116-05"
/dev/sda8: UUID="02c1d09e-54db-4346-863d-1097ff0ed637" TYPE="swap" PARTUUID="00083267-08"
/dev/sdc1: UUID="B3E7-6FA0" TYPE="vfat" PARTUUID="20cd881c-01"
/dev/nvme0n1: PTUUID="36bdb116" PTTYPE="dos"

[1fallen]

Well all that shows us something, Hip Ho-ray! :)
And yes that's what I was looking for>>"the @ response to the c/L "
Now let's see if we get lucky:

sudo su

mount -o subvol=@ -t btrfs /dev/nvme0n1p1
now try and run your scrub.

btrfs scrub start -Bf /dev/nvme0n1p1
Best of luck, I have to go to work but I'll check back later to-nite.

[Robbyx]

A. It seems to have worked. (I did not see your last comment #19 when I gave this answer. See #21 below in response to your #19)


ubuntu@ubuntu:~$ sudo btrfs filesystem df /mnt
Data, single: total=17.01GiB, used=15.54GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=1.51GiB, used=387.31MiB
GlobalReserve, single: total=30.94MiB, used=0.00B
ubuntu@ubuntu:~$ sudo btrfs scrub start -Bd /mnt
scrub device /dev/nvme0n1p1 (id 1) done
scrub started at Sat Sep 28 21:12:01 2019 and finished after 00:00:07
total bytes scrubbed: 15.91GiB with 0 errors
ubuntu@ubuntu:~$ sudo umount /mnt/

ubuntu@ubuntu:~$ sudo mount /dev/nvme0n1p2 /mnt
ubuntu@ubuntu:~$ ls /mnt
@home
ubuntu@ubuntu:~$ sudo btrfs scrub start -Bd /mnt
scrub device /dev/nvme0n1p2 (id 1) done
scrub started at Sat Sep 28 21:17:14 2019 and finished after 00:03:02
total bytes scrubbed: 18.77GiB with 0 errors
ubuntu@ubuntu:~$ sudo umount /mnt/




B. Here is the original fstab. I notice I am using subvol. Is there a special adjustment that needed to be made when running btrfs to take into account subvols. Is that the reason why our btrfs c/L did not work?
see https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-subvolume


# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
# SDDe_980GiB
# / was on /dev/nvme0n1p1
UUID=ff16fdbe-9a30-4c73-89a4-bf10cd1b1974 / btrfs defaults,noatime,subvol=@ 0 0
# /home was on /dev/nvme0n1p2
UUID=5bb8aa95-b657-4297-be5b-4352a09596b0 /home btrfs defaults,noatime,subvol=@home 0 0
#/dev/nvme0n1p4: LABEL="Data"
UUID=f5ab46ce-b3da-4e59-8f18-3418fe272a09 /media/robins/data_sdde btrfs defaults,noatime 0 0
# swap was on /dev/nvme0n1p5
UUID=4ec66d43-75bf-4f9e-8f36-13275a0f3258 none swap sw 0 0
# /dev/nvme0n1p6 LABEL="EFI"
UUID=C649-2797 /media/robins/windows vfat utf8,umask=007,gid=46 0 0

#----------------------------------------------------------------------------SDD 250GiB
#/dev/sda5: LABEL="mydocs" on SDD_250
UUID=8d99bfc8-90ab-4488-b7da-87e7676aff19 /media/robins/mydocs_sdd ext4 defaults,noatime 0 0
#/dev/sda6: LABEL="Data inc AV" on SDD_250
UUID=1a2e8d10-7b93-4de1-b551-31db6a621276 /media/robins/data_inc_av_sdd ext4 defaults,noatime 0 0

# ----------------------------------------------------------------------------Hitachi HD 750GiB
# sdb1 hiachi 750GiB
UUID=3c4f4688-4bd7-4262-a2b8-86f71bbd6d0b /media/robins/hitachi_750GiB ext4 defaults,noatime 0 2
#/dev/sdb2: LABEL="Deju_dup_backup"
UUID=c79b648e-48c5-45db-949f-a835928f3816 /media/robins/hitachi_deju_dup ext4 defaults,noatime 0 2

#------------------------------------------------------------------------------------Samsung 1.8TiB
# /dev/sdc2: LABEL="xhd system data"
UUID=23fcb4e3-c94e-44de-b7a9-105da58918c6 /media/robins/samsung_xhd/sam_xhd_system_data ext4 defaults,noatime 0 2
#/dev/sdc3: LABEL="xhd old bacs"
UUID=c99be12f-3a2d-4411-ab7c-5696167482dc /media/robins/samsung_xhd/sam_xhd_old_bacs ext4 defaults,noatime 0 2
#/dev/sdc4: LABEL="spare4"
UUID=84a9cb0b-6006-4996-8b4a-2edc07cc25c4 /media/robins/samsung_xhd/sam_xhd_spare4 ext4 defaults,noatime 0 2



C. What should I do next? Assuming the answer is to restart the computer without the liveCD, and I find that I am still thrown into emergency mode on startup. What commands do you suggest I use load the os? Mount?

[Robbyx]

Response to #19:


ubuntu@ubuntu:~$ sudo umount /mnt/
ubuntu@ubuntu:~$ sudo su
root@ubuntu:/home/ubuntu# mount -o subvol=@ -t btrfs /dev/nvme0n1p1
mount: /dev/nvme0n1p1: can't find in /etc/fstab.
root@ubuntu:/home/ubuntu#


Is the error due to me running in liveCD mode?

[Robbyx]

I have good news:
I am able to boot and access the system via the HD.

Background:
After scrubbing the btrfs root and home partitions and finding no errors, I tried to log in via the HD. I was quickly put into emergency mode. I then attempted to mount the two partitions manually, but an error message came up that they were not in the fstab. Next, I did a clean install, formatting the root directory but not home or any other partitions. At the end, I started from the HD. This time emergency mode did not popup and I was able to boot up successfully. I now have to reinstall my missing programs and printer driver.

[1fallen]

Great News!
BTRFS aka butter-fs is a finicky thing (eg: names paths dev vols, and subvols) but with more experience, it is a nice "copy-on-write" (or "COW") filesystem.
I'm waiting for some other things to get fixed and let it mature or simmer before I give it another go.

[Robbyx]

It is unclear why the installation failed, especially as it has the advanced error checking built in via BTRFS. However thank you very much for your support in getting the system up and running.

[The Cog]

Glad to see you got it running.
For other readers, I think a lot of the problems in this thread were caused by trying to refer to the /dev/* device file. But a lot of the btrfs commands require the partition to be mounted and they reed to be given the path to the mount point, not the device file. Maybe this is because the same device file can contain lots of subvolumes and lots of different mount points. Pointing to the device file is not specific enough. But it certainly feels odd when you are used to only mounting the whole partition.