One pendrive for all PC (Intel/AMD) computers - single-boot dual-boot multi-boot

[sudodus]

The script runs well from another Persistent drive but not from a Full install drive.
Attached screen shots, a red warning and gparted.

Have you tested the new version, which works for me also from an active persistent live drive?

How to find how much space left in home?

df -h  # and look for the partition mounted at /home

Script seems to move everything from home in the casper-rw partition to the home partition.

This is a surprise. I did nothing for that to happen. For me the new home partition will be empty until I reboot and start using it.

I'm still playing will let you know if I notice anything else.
I'm not sure I am too hot on UDF though.
It is only a small pain for me when windows wants to reformat, except if I have a lot of partitions then it takes a while.

I can see what you mean, particularly because the casper-rw partition needs a file system, that the current Windows 10 system wants to format.

- A small(?) advantage is that the data in a home partition with UDF will not be subject to such attacks from Windows 10, so it is more likely to survive.

- Another advantage is that partition #1 will be available for Ubuntu as /home and at the same time for Windows as external storage of files.

- A disadvantage is that there is no well known tool to repair the UDF file system.

So to summarize, unless I get positive feedback from other users, I will not push this issue further.

I really like the idea of having an flexible sdx4 that could be used for ISO9660 as is, or for a Full installed system, or maybe for multibootin ISO's. I'm wondering if there should be a basic page in mkusb and maybe a pro page or advanced page.
But I would hate to see mkusb messed up, it is pretty good as is. There are lots of uses for Persistent and Live installs.
I think mkusb would be the first USB tool to make Full installs, which is the future.

OK, let us agree that both [persistent] live and full installs are useful, and we need methods and tools for both

Edit:
There is a new unknown partition where usbdata used to be but stuff dumped on the desktop is still going to home in casper-rw, I think.

Please describe this new unknown partition and how you see it! You can run the same commands as I show in the following code window and compare your result.

xubuntu@xubuntu:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            1.9G     0  1.9G   0% /dev
tmpfs           383M  1.3M  382M   1% /run
/dev/sdb4       663M  663M     0 100% /cdrom
/dev/loop0      580M  580M     0 100% /rofs
/cow            6.9G  101M  6.5G   2% /
/dev/sdb1       7.1G  2.3M  7.1G   1% /home
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs           1.9G  8.0K  1.9G   1% /tmp
tmpfs           383M   12K  383M   1% /run/user/999
xubuntu@xubuntu:~$ sudo lsblk -fm
NAME   FSTYPE   LABEL                      UUID                                 MOUNTPOINT   SIZE OWNER GROUP MODE
loop0  squashfs                                                                 /rofs      579.8M root  disk  brw-rw----
sda                                                                                        111.8G root  disk  brw-rw----
├─sda1 vfat                                1357-5B02                                         300M root  disk  brw-rw----
└─sda2 ext4                                2e1f47a4-0518-4429-ad23-65893d154ebe            111.5G root  disk  brw-rw----
sdb                                                                                         14.9G root  disk  brw-rw----
├─sdb1 udf      home-rw                    5b57ab9c1ab1c194                     /home          7G root  disk  brw-rw----
├─sdb2                                                                                         1M root  disk  brw-rw----
├─sdb3 vfat     usbboot                    A178-47CB                                         244M root  disk  brw-rw----
├─sdb4 iso9660  Xubuntu Core 18.04 - amd64 2018-04-26-21-58-14-00               /cdrom       682M root  disk  brw-rw----
└─sdb5 ext4     casper-rw                  c5b2ce97-a934-4de2-acc8-6a5fa0080a70                7G root  disk  brw-rw----
sr0                                                                                         1024M root  cdrom brw-rw----
xubuntu@xubuntu:~$ sudo parted -ls
Model: ATA KINGSTON SUV400S (scsi)
Disk /dev/sda: 120GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: 

Number  Start   End    Size   File system  Name  Flags
 1      2098kB  317MB  315MB  fat32              boot, esp
 2      317MB   120GB  120GB  ext4


Model: SanDisk Extreme (scsi)
Disk /dev/sdb: 16.0GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name     Flags
 2      1049kB  2097kB  1049kB               primary  bios_grub
 3      2097kB  258MB   256MB   fat32        primary  boot, esp
 4      258MB   973MB   715MB                primary
 5      973MB   8493MB  7520MB  ext2         primary
 1      8493MB  16.0GB  7520MB               primary  msftdata


xubuntu@xubuntu:~$

When running the script from an active persistent live drive, stuff dumped on the desktop is still going to home in casper-rw. But at least for me (tested with Lubuntu 16.04.1, 18.04 and Xubuntu-Core 18.04) after rebooting twice, the new 'home-rw' partition with UDF will store the persistence for /home.

[C.S.Cameron]

Oops, had a senior moment trying to recall how to post screen shots in the Forums.
Showing Warning and GParted.

[sudodus]

@C.S.Cameron,

I intend to reproduce your error and warning output, and need details in order to do it.

The persistent live system is Ubuntu 18.04 LTS amd64. What is the running operating system? The controls are on the left side indicating Unity. Is it an Ubuntu 16.04 LTS system (live or installed)?

Edit:

What is /dev/sdf ? Is the 'Datatraveler 3.0' the 'Kingston DT50 16GB' USB 3 pendrive?

I'm guessing here: Maybe there are other USB devices connected, that confuse the USB system. Can you disconnect [at least some of] them so that the 'Datatraveler 3.0' can be seen as /dev/sdb or /dev/sdc? Then the script might work better.

[C.S.Cameron]

I was running 16.04.4. Persistent.
I tried 16.04 Full install but could not get the script to run.
I then booted the 18.04 disk and the script gave the same results as 16.04 drive.
(I'm working on an old P5, no internal disk, only USB is mouse, keyboard and bootdrive).
The 4GB DataTraveler is 16.04 (sde), the DT50 16GB is the 18.04 (sdf).
For some reason there is no sda - sdd, only sde and sdf.
I've tried clearing the target casper-rw partition before running, (so there would be no existing home), but this did not make an active home-rw partition.
I will freshen up these drives and test on another computer.

[sudodus]

It is not your fault. There was a bug in the script.

I could reproduce your result and found the bug: In 16.04.4 persistent, the universe repository must be activated, because that's where the package udftools resides. So it was simply not installed, and there was no way to create a UDF file system.

I have updated post #158 with a new version of the shellscript, that works for me in 16.04.4 persistent. I hope that you will have better luck with this new version

ubuntu@ubuntu:~$ ./udf2home-rw 
udf2home-rw
live system or temporary superuser permissions
mpcrw=/tmp/tmp.aTHgQeQxv2
Device      Model            Size     Persistent live OS
/dev/sdb    Extreme          29.8G    Ubuntu 16.04.4 LTS amd64
/dev/sdc    Extreme          14.9G    Ubuntu 18.04 LTS amd64
cntcrw=2
srcdev=/dev/sdc5
 Device: /dev/sdc  OS: Ubuntu 18.04 LTS amd64 
bupfile=
umount: /dev/sdc1: not mounted
umount: /dev/sdc3: not mounted
umount: /dev/sdc4: not mounted
umount: /dev/sdc5: not mounted
'universe' distribution component enabled for all sources.
Ign:1 cdrom://Ubuntu 16.04.4 LTS _Xenial Xerus_ - Release amd64 (20180228) xenial InRelease
Hit:2 cdrom://Ubuntu 16.04.4 LTS _Xenial Xerus_ - Release amd64 (20180228) xenial Release
Hit:4 http://archive.ubuntu.com/ubuntu xenial InRelease                    
Hit:5 http://archive.ubuntu.com/ubuntu xenial-updates InRelease                                      
Hit:6 http://security.ubuntu.com/ubuntu xenial-security InRelease                                    
Get:7 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7,532 kB]
Get:8 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [361 kB]
Get:9 http://archive.ubuntu.com/ubuntu xenial/universe Translation-en [4,354 kB]
Get:10 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [135 kB]
Get:11 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 DEP-11 Metadata [107 kB]         
Get:12 http://security.ubuntu.com/ubuntu xenial-security/universe DEP-11 64x64 Icons [142 kB]            
Get:13 http://archive.ubuntu.com/ubuntu xenial/universe amd64 DEP-11 Metadata [3,410 kB]              
Get:14 http://archive.ubuntu.com/ubuntu xenial/universe DEP-11 64x64 Icons [7,448 kB]
Get:15 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [645 kB]
Get:16 http://archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [260 kB]
Get:17 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 DEP-11 Metadata [246 kB]
Get:18 http://archive.ubuntu.com/ubuntu xenial-updates/universe DEP-11 64x64 Icons [331 kB]
Fetched 25.0 MB in 3s (6,511 kB/s)                                 

** (appstreamcli:4193): CRITICAL **: Error while moving old database out of the way.
AppStream cache update failed.
Reading package lists... Done
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  dvd+rw-tools pmount
The following NEW packages will be installed:
  udftools
0 upgraded, 1 newly installed, 0 to remove and 284 not upgraded.
Need to get 62.1 kB of archives.
After this operation, 230 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu xenial/universe amd64 udftools amd64 1.0.0b3-14.4 [62.1 kB]
Fetched 62.1 kB in 0s (243 kB/s)    
Selecting previously unselected package udftools.
(Reading database ... 195541 files and directories currently installed.)
Preparing to unpack .../udftools_1.0.0b3-14.4_amd64.deb ...
Unpacking udftools (1.0.0b3-14.4) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu21.1) ...
Processing triggers for ureadahead (0.100.0-19) ...
ureadahead will be reprofiled on next reboot
Setting up udftools (1.0.0b3-14.4) ...
Processing triggers for systemd (229-4ubuntu21.1) ...
Processing triggers for ureadahead (0.100.0-19) ...
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141386 s, 74.2 MB/s
start=0, blocks=64, type=RESERVED 
start=64, blocks=12, type=VRS 
start=76, blocks=180, type=USPACE 
start=256, blocks=1, type=ANCHOR 
start=257, blocks=16, type=PVDS 
start=273, blocks=1, type=LVID 
start=274, blocks=13491693, type=PSPACE 
start=13491967, blocks=1, type=ANCHOR 
start=13491968, blocks=239, type=USPACE 
start=13492207, blocks=16, type=RVDS 
start=13492223, blocks=1, type=ANCHOR 
Please wait for sync (flushing file system buffers to the device)
Partprobe: 
Umount: 
umount: /dev/sdc[1345]: mountpoint not found

MODEL            NAME   FSTYPE  LABEL                  MOUNTPOINT  SIZE
Extreme          sdc                                              14.9G
                 ├─sdc2                                              1M
                 ├─sdc5 ext4    casper-rw                          6.4G
                 ├─sdc3 vfat    usbboot                            244M
                 ├─sdc1 udf     home-rw                            6.4G
                 └─sdc4 iso9660 Ubuntu 18.04 LTS amd64             1.8G
 Created UDF file system on home-rw partition '/dev/sdc1' :-) 
If modifying an active persistent live drive, you may need to reboot twice
ubuntu@ubuntu:~$

[C.S.Cameron]

OK, that is running with no warnings. making an unknown file system partition in GParted labeled home-rw.
Plugged the drive into a Win 10 machine and Windows still wanted to format three partitions but not home-rw.
Having Ubuntu home partition visible in Windows is kinda neat, and hopefully works for a Windows - Linux partition, but it is also kinda spooky.
I made a few changes to the next flash drive, background image, and dumped some big files on the desktop.
The script ran OK from the Full install flash drive.
None of the data in home moved to the new location.
The drive booted to Live session user and asked for password.
The second, third and fourth try froze.
I think home folder needs to be empty when running script.
The script should probably be run as part of the install process.
Maybe a toggle NTFS vs UDF home.

[sudodus]

@C.S.Cameron,

Thanks for your patience during these tests

OK, that is running with no warnings. making an unknown file system partition in GParted labeled home-rw.
Plugged the drive into a Win 10 machine and Windows still wanted to format three partitions but not home-rw.
Having Ubuntu home partition visible in Windows is kinda neat, and hopefully works for a Windows - Linux partition, but it is also kinda spooky.
I made a few changes to the next flash drive, background image, and dumped some big files on the desktop.
The script ran OK from the Full install flash drive.
None of the data in home moved to the new location.

OK, this is what I would expect.

The drive booted to Live session user and asked for password.

This is what happens for me too, when modifying the currently running persistent live system.

The second, third and fourth try froze.

This should not happen. But I made it happen, when booting with more than one mkusb-persistent-live pendrives connected when booting. I think the problem is confusion concerning which casper-rw (and home-rw if available) partition to use, and I think it it a general problem with mkusb-persistent-live systems (not specific to home-rw with UDF).

If this is the cause also in your case, you can work around it by booting with only one mkusb-persistent-live pendrive connected. After booting (when booting has finished and you are at the desktop), you can plug in the target pendrive and do the things to convert 'usbdata' to 'home-rw' with UDF.

I think home folder needs to be empty when running script.
The script should probably be run as part of the install process.
Maybe a toggle NTFS vs UDF home.

In my tests, the home folder in the 'casper-rw' partition need not be empty. But it will not be transferred to the new UDF home.

---

Edit 1: The drive booted to Live session user and asked for password.

This is what happens for me

- when modifying the currently running persistent live system

- when there is content in the 'casper-rw' partition (I tested it right now).

The solution is a second reboot.

Edit 2: You wrote that the script should probably be run as part of the install process. Maybe a toggle NTFS vs UDF home.

You may have some other things 'on the wishlist' for this script too. If you think it can be useful, please let discuss it now

For example: should the content in the 'casper-rw' partition be copied or moved to the new home-rw partition?