"ALERT! /dev/disk/by-uuid/X-X-X-X does not exist." when booting Ubuntu with Xen

[Mikael_Niemel]

Well, I decided to go ahead and try Ubuntu server with Xen to virtualize stuff. This is mostly because my server is a bit older Opteron machine, which doesn't support full hardware virtualization, so KVM is out and VirtualBox doesn't have as good performance as Xen. Also, I'd like to use zfs (as it's finally supported properly in Linux, yay!) from Dom0 as the storage for various virtual machines.

So, I tried to install Xen according to instructions (minus LVM part): https://help.ubuntu.com/community/Xen

But I get this error when trying to boot Ubuntu with Xen. When I choose plain Ubuntu from the Grub menu, then it boots just fine.


Gave up waiting for root device. Common probleMs:
— Boot args (cat /proc/cmdline)
— Check rootdelay= (did the systeM wait long enough?)
— Check root= (did the systeri wait for the right device?)
— Missing Modules (cat /proc/Modules; ls /dev)
ALERT? /dev/disk/by-uuid/X—X—X—X—X does not exist.
Dropping to a shell!


Any idea how to solve this?

[Mikael_Niemel]

..anybody?

[MAFoElffen]

Please post the linux boot line argument from your Xen boot section. The line will start with "linux".

[Mikael_Niemel]

I didn't find a line starting with "linux" in the Xen boot section in /boot/grub/grub.cfg. This is what that section has:

menuentry 'Ubuntu GNU/Linux, with Xen hypervisor' --class ubuntu --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-simple-X-X-X-X-X$ insmod part_msdos
insmod ext2
set root='hd5,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd5,msdos1 --hint-efi=hd5,msdos1 --hint-baremetal=ahci5,msdos1 X-X-X-X-X
else
search --no-floppy --fs-uuid --set=root X-X-X-X-X
fi
echo 'Loading Xen 4.6-amd64 ...'
if [ "$grub_platform" = "pc" -o "$grub_platform" = "" ]; then
xen_rm_opts=
else
xen_rm_opts="no-real-mode edd=off"
fi
multiboot /boot/xen-4.6-amd64.gz placeholder ${xen_rm_opts}
echo 'Loading Linux 4.4.0-22-generic ...'
module /boot/vmlinuz-4.4.0-22-generic placeholder root=UUID=X-X-X-X-X ro
echo 'Loading initial ramdisk ...'
module --nounzip /boot/initrd.img-4.4.0-22-generic
}

And this is the normal Ubuntu section, which works fine, and has the "linux" line

menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-X-X-X-X-X' { recordfail
load_video
gfxmode $linux_gfx_mode
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_msdos
insmod ext2
set root='hd5,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd5,msdos1 --hint-efi=hd5,msdos1 --hint-baremetal=ahci5,msdos1 X-X-X-X-X
else
search --no-floppy --fs-uuid --set=root X-X-X-X-X
fi
linux /boot/vmlinuz-4.4.0-22-generic root=UUID=X-X-X-X-X ro
initrd /boot/initrd.img-4.4.0-22-generic
}

[MAFoElffen]

Please post file /ect/default/grub...

EDIT--
Didn't hear back from you yet... but this is what I am looking for / if you addedd Xen boot option to your command line in your default file such as this:

# Xen boot parameters for all Xen boots
GRUB_CMDLINE_XEN="dom0_mem=1024M,max:1024M"

[Mikael_Niemel]

I have been a bit busy, so I'm taking a look it just now.

Anyways, I didn't find a line like that at all, I'll try adding it..

EDIT: That didn't work.. also, here's my current /ect/default/grub. Line starting with GRUB_CMDLINE_XEN was added by me just now.

# If you change this file, run 'update-grub' afterwards to update# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'


GRUB_DEFAULT=0
#GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=2
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_CMDLINE_LINUX=""
GRUB_CMDLINE_XEN="dom0_mem=1024M,max:1024M"


# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"


# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console


# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480


# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true


# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"


# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

[Mikael_Niemel]

Any further ideas how to solve this?

[MAFoElffen]

Yes. I had to look further back into this specific error. I found it under some Ubuntu Precise 12.04 LTS error codes. It doesn't specifically have to do with Xen itself. It has to do with the boot process. This error usually comes up when it reads the fstab and tries to mount filesystems for parts of the default directories. For instance, where "/" (root) and swap is, identified by their UUID's ... and when it hits a UUID that it can not find, it thorows this message, and usaully drops down to a grub rescue or busybox prompt.

The thing to do is to look at, and/or post your /etc/fstab file and the results of

sudo blkid

then you/we would match up the UUID's and ensure that what is in the fstab matches up with the output of what is physically there.

Does that make sense as a plan?

[Mikael_Niemel]

Here's my /etc/fstab file:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    nodev,noexec,nosuid 0       0
# / was on /dev/sda1 during installation
#UUID=88bae243-a8d2-4435-913a-eca6e6807646
/dev/sda1       /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
#UUID=a41a19e5-80f2-4715-8062-56f3b9915a46
/dev/sda5       none            swap    sw              0       0
/dev/fd0        /media/floppy0  auto    rw,user,noauto,exec,utf8 0       0

IIRC those commented out UUIDs were there before I replaced them with /dev/sda1 and /dev/sda5 as I tried to solve the issue. Anyways, the odd thing is that Ubuntu boots just fine without Xen, doesn't it use the same fstab file?

Also, here's what blkid gave:

/dev/sda1: UUID="88bae243-a8d2-4435-913a-eca6e6807646" TYPE="ext4" PARTUUID="0009e8cc-01"
/dev/sda5: UUID="a41a19e5-80f2-4715-8062-56f3b9915a46" TYPE="swap" PARTUUID="0009e8cc-05"
...
...

[MAFoElffen]

I do not see anything wrong with the original filesystem UUID's. After you update an fstab file, then you should update your initramfs files... so the initial boot ram disk matches what is physically there.

But you bring up an interesting point. one thing that is different between the normal boot on a kernel and a Xen boot, (as it relates to finding storage devices) is the initramfs images.

If update the boot images for 4.4.0-22-generic via:

sudo update-initramfs -u 4.4.0-22-generic

^That was the kernel version, right?