Why isn't my LiveUSB persistent

[Vinze]

ANOTHER UPDATE: The problem concerned Feisty, if you want to put Gutsy or anything non-Feisty on a USB drive you can follow the normal guide at https://wiki.ubuntu.com/LiveUsbPendrivePersistent

UPDATE: My problem (a bug in Feisty) is now solved (or rather: worked around). Instructions on how to do this yourself with Ubuntu Feisty can be found here on my blog. Or if you'd rather use Xubuntu, I also have this guide.

Hey, I followed these instructions (fairly similar to these, but geared towards Feisty) to put Ubuntu on my USB drive. I finally managed to get it working, it booted and all, but after I rebooted I noticed that everything I had configured and installed was gone! Same when I tried it later with Xubuntu.

Why? I've properly named the partition casper-rw

[daristia]

This post could be related to an Ubuntu bug filed at: https://bugs.launchpad.net/ubuntu/+s...per/+bug/84591
----------------------------


You may want to look here:

https://bugs.launchpad.net/ubuntu/+s...per/+bug/84591

It's a known bug, hopefully it will be fixed soon. Meanwhile, you can use a former version.

Daniel

[Vinze]

Originally Posted by daristia This post could be related to an Ubuntu bug filed at: https://bugs.launchpad.net/ubuntu/+s...per/+bug/84591 ---------------------------- You may want to look here: https://bugs.launchpad.net/ubuntu/+s...per/+bug/84591 It's a known bug, hopefully it will be fixed soon. Meanwhile, you can use a former version. Daniel

Lol, I had already found it.

Of what should I use a former version? Casper?

[daristia]

No, a former version of Ubuntu; 6.10 is supposed to work.

[Vinze]

Originally Posted by daristia No, a former version of Ubuntu; 6.10 is supposed to work.

Right, I'll try with Feisty Herd 4 when I have the time, thanks

[calinb]

I think the reason persistence is not working in 7.04 may be due to a Unionfs / kernel version incompatibility. As I reported on the bug listing, persistence works okay when vmlinuz and initrd.gz are copied from a 6.10 liveCD to the USB stick. Other things break, however.

https://bugs.launchpad.net/ubuntu/+s...per/+bug/84591

I read the Unionfs notes in the kernel source.

sudo apt-get install linux-source-2.6.20

Check out the notes in linux-source-2.6.20/ubuntu/fs/unionfs (or /usr/src/linux/ubuntu/fs/unionfs, if you've already setup a sym link to build the kernel).

Study the "README," "NEWS," and "INSTALL" files.

README: "This Unionfs release supports only one 2.6 kernel version (run 'make kvers' to find out which)."

NEWS: * Unionfs 1.3 (intended for use with 2.6.17)
- Fixed makefile release & build target
- Fixed up uniondbg & unionimap man pages
- Updated RPM specfile
- Implemented export functions (for NFS)
- Updated to compile with 2.6.17

Gee--persistence works when I hack the 6.10 kernel into the USB stick and which kernel does 6.10 use? It uses 2.6.17! That's a match with Unionfs 1.3. However, 7.04 uses 2.6.20 and it appears to be incompatible with Unionfs 1.3. I see no indication that the module has been patched for 2.6.20. No wonder unionfs won't load on 7.04.

The compatibility table at
http://www.fsl.cs.sunysb.edu/project-unionfs.html

also pairs 1.3 with the 2.6.17 kernels and indicates the correct stable unionfs code for 2.6.20 kernels is Unionfs version 2.0, 2.6.20-u1.

From the Unionfs INSTALL file:
"You should be able to just type "make" and Unionfs will build itself for the
running kernel. The Makefile will look for your running kernel sources in
/lib/modules/`uname -r`/build/include."

So maybe we can use the 6.10 hack to get persistence working and build a Unionfs version 2.0, 2.6.20-u1 module using source downloaded from the Unionfs website. Once the new Unionfs module is built and installed, we can switch back the the 7.04 / 2.6.20 kernel. Of course this is still somewhat of a hack and the right way to do it would be to create a new live CD / USB stick with the correct Unionfs module, rather than relying on casper-rw to carry it, but I don't know how to build a new liveCD yet.

From what I've seen, networking may not work during a build with the hacked-in 2.6.17 kernel so a possible workflow might be something like the following, but I've not had time to try the method.

1. Boot 7.04 (either USB or live CD) and download the Unionfs source and Ubuntu kernel source to a hard disk. Actually, it might be a good idea to download everything you need to build the kernel and save the .deb files to the hard disk too. Don't use FAT for kernel builds. You may run into problems later. It's probably best to put the kernel source on the hard disk and setup the /usr/src/linux symlink to the hard disk and go ahead and build a kernel--just to make sure you're got all the .deb files you'll need later. Save the .deb files from the cache. Put everything on the hard disk because it will probably fill your USB stick if you expand the sources there. Keep in mind that persistence is not yet working. Everything must be on the hard disk for later use. In general follow the guide:

http://ubuntuforums.org/showthread.p...t=kernel+build

but don't use vanilla sources. Just use the sources from apt-get install linux-source-2.6.20 and copy /boot/config-2.6.20-15-generic from the 7.04 CD to .config.

2. Copy vmlinz and initrd.gz from a 6.10 CD to /usb and /usb/casper on a fresh 7.04 USB stick.

3. Boot the hacked USB stick. Persistence should now be working (but many other things may be broker.) Hopefully, enough stuff is working to build a new unionfs module.

4. Install all the .deb packages that you previously saved to the hard disk.

5. Point the unionfs build to the 2.6.20 kernel source on the hard disk, as described in the INSTALL notes. It should not be necessary to actually build the entire kernel. I only recommended building the kernel in step 1 to make sure we have all the packages that are needed for a complete kernel build environment but who knows? It may be necessary to actually build a new kernel but I don't really know how to do a build for the live CD. Obviously it would be a good idea to copy the config file from /boot and re-use it.

Wow--I could've done all that in the time it took to type it. Nevertheless, it would be good to have others working on it because I think there will be a few hurdles to overcome.

Let's use this thread to post progress.

Thanks,

-Cal

[calinb]

Update:

It's not as easy as the Unionfs 1.3 notes, contained in the Ubuntu kernel source, indicate. For 2.0, you will git the entire 2.6.22 kernel tree when you download the Unionfs source. There's no mention of building unionfs modules into an operating kernel. It seems you must merge the unionfs/fs... branch you downloaded into /usr/src/linux/ubunbu/fs... and probably merge some .h files elsewhere (I moved some around before I got the unionfs patch installed correctly). You must apply the unionfs linux-2.6.20-u1.diff patch to patch the kernel tree back to Feisty's 2.6.20 kernel. Presumably this is how to proceed.

The build is proceeding under a stock 7.04 live boot (no persistence). Even if it completes, I have no idea if I can just reboot to a hacked 7.04/6.10 live USB (to enable persistence) and install the resulting deb packages from the build, rename the initrd and kernel files and copy them to /usb and /usb/casper.

I also have a strong speculative notion that an Ubuntu dev was here before me, had problems with unionfs, and gave up to meet the release schedule. The old unionfs files were simply left on the distro. Until unionfs makes it into the kernel.org linux kernel, I suspect it's a maintenance nightmare for any distro and I can certainly sympathize if that's what happend, or something similar.

-Cal

P.S. Wow, I'm on to quickcam module in my build:

..
..
CC [M] ubuntu/media/quickcam/qc-vv6410.o
CC [M] ubuntu/media/quickcam/qc-formats.o
..
..
maybe this thing will at least complete!

[calinb]

One more thing...

We could wait until Unionfs version 1.x (legacy code) is released. Version 1 is what Ubuntu uses. The Unionfs site says, "2.6.20 N/A (1.6 coming soon"

Of course Ubuntu will be probably be down the road at 2.6.22+ by then. Maybe Ubuntu will intercept Unionfs 2 in the next release.

-Cal

[jerrylamos]

Originally Posted by calinb Update: I also have a strong speculative notion that an Ubuntu dev was here before me, had problems with unionfs, and gave up to meet the release schedule. The old unionfs files were simply left on the distro. Until unionfs makes it into the kernel.org linux kernel, I suspect it's a maintenance nightmare for any distro and I can certainly sympathize if that's what happened, or something similar.

Yep, one of the key developers tried just before 2007-04-15 to do a fix and left some changes in:

" Colin Watson said on 2007-04-12: (permalink)

I tried as hard as I could to fix this, but I couldn't get it to work for me so couldn't recommend an upload; at this point I'm afraid it's post-feisty. The fixes I attempted are in the main Ubuntu casper bzr branch if anyone wants to try them out.

(Specifically, while my fixes correct the casper-rw label handling, the USB device still doesn't appear until somewhere just shortly after the attempt to detect casper-rw devices. Adding udevtrigger/udevsettle didn't help.)"

It looked to me at the time that the USB device was getting recognized later than it had in 6.10, likely because of the Feisty very slow very complex SCSI simulation of SATA, IDE, CD's, USB, Floppy, and who knows what else. I would have no idea how to delay the "attempt to detect casper-rw devices" until the SCSI simulation has finally got the USB device to appear, since udevsettle didn't work for Colin.

Cheers, Jerry

[calinb]

Thanks, Jerry. I'll try to find time to look at the fixes Colin attempted--especially since my first attempt at building Unionfs 2.0 has failed:

root@ubuntu:/usr/src/linux# make-kpkg -initrd --revision=386 kernel_image kernel_headers modules_image
...
...
...
WARNING blah blah blah
...
WARNING blah blah blah
...
WARNING: vmlinux - Section mismatch: reference to .init.text: from .text between 'iret_exc' (at offset 0xc02f2220) and '_etext'
WARNING: vmlinux - Section mismatch: reference to .init.text:start_kernel from .paravirtprobe between '__start_paravirtprobe' (at offset 0xc03c2790) and '__stop_paravirtprobe'
WARNING: "drop_pagecache_sb" [ubuntu/fs/unionfs/unionfs.ko] undefined!
WARNING: "pathget" [ubuntu/fs/unionfs/unionfs.ko] undefined!
WARNING: "pathput" [ubuntu/fs/unionfs/unionfs.ko] undefined!
WARNING: "krealloc" [ubuntu/fs/unionfs/unionfs.ko] undefined!
make[2]: *** [__modpost] Error 1
make[1]: *** [modules] Error 2
make[1]: Leaving directory `/mnt/sdc1/linux-source-2.6.20'
make: *** [debian/stamp-build-kernel] Error 2

Looks like it might be path problem as unionfs in the 2.6.20 Ubuntu source tree is under /usr/src/linux/ubuntu/fs/... but the unionfs 2.0 kernel.org fork is at /usr/src/linux/fs/...

Someone more adept at building kernels than I might know better.

-Cal