ubuntu-desktop-installer Staffing shift

[MAFoElffen]

I have bugs, enhancements, and questions filed in the issues at the GitHub for ubuntu-desktop-installer... (And an unanswered question at Launchpad)

They were not getting answered or addressed, so I filed this as a bug:
Issues, Questions and Enhancements Not being Addressed #2383

I figured it might at least get their attention... After over a month (today), it did:

The installer project is being handed over between developers so there will be a period of quiet.

Since this issue is not anything specific about the code I am going to close it.

This is still the correct place to discuss installer issues. And no I am not one of the new installer developers. Your patience is appreciated.

"there will be a period of quiet" <-- Is that like a moment of silence?

So it took someone who is loosely related to the project to review and close it? Curious that it took a "staffing change" to even trigger looking at issues. I admit, that doesn't specifically relate to the code, but is an "Issue" with their Project, that they are not addressing anything related to the code, any replies to enhancements or questions.

I know, it was filed tongue in cheek... I do sort of have a twisted sense of humor. But my original question there is still open and unanswered, like it's corresponding question at Launchpad.

Which are these:
https://github.com/canonical/ubuntu-...er/issues/2361 <-- Which directly relates to Issue #2360.
https://answers.launchpad.net/ubuntu...uestion/708132

Where do I find those answers and/or who to talk to that might know where to look? I am still looking for answers to those questions and it has been 2-3 months now. No-one can even tell me "what" the new installer's 'partitioner' is named, to try to research it myself. I'm sort of at a dead-end with that.

FYI -- After I filed bugs at Launchpad on the TPM backed installer script (https://bugs.launchpad.net/ubuntu-de...r/+bug/2039741), and added reasoning and support to users at the ubuntu-desktop-installer GitHub Issues (and linked those back to that bug report) that were having problems with that installer scripts results... I can see they dropped that Experimental option for Noble.

[Claus7]

Hello,

some questions to the above:
1) is the new installation a snap package?
2) could you please explain what do you mean "change partitioning on the fly"?
I do know about manual partitioning and that's it.
3) if legacy installation will be available as well, is it not possible for you to continue doing what you have been doing all this time long?

I do know that the latter is not the "answer" you would expect from someone, yet my speculation is that, the new installer seems to have issues. I do not remember in the past an ubuntu amd iso to take so much time in order to be available and to remain to pending state for a reasonable amount of time. From my experience, non latin language installations have some issues that are hard to address, so I suppose that the new installation task is not so trivial. Once again, not answers to your questions, yet having a small experience, I could share it with you.

Regards!

[rayanclass]

So, there's been this interesting staffing shift happening with the "ubuntu-desktop-installer" team, and it's got everyone talking. The folks who were initially responsible for managing this essential component of Ubuntu's desktop installation process have handed off their duties to a new group of talented individuals. It seems like a breath of fresh air for the project, as these new recruits bring in a wealth of expertise and passion for streamlining the desktop installer experience. From what I've gathered, they've been hard at work brainstorming creative ways to enhance user friendliness and improve performance. This change in personnel has injected a newfound sense of energy into the team, and it's exciting to see how it will shape the future development of Ubuntu. Here's hoping that their efforts result in an even more user-friendly and efficient desktop installer!

[MAFoElffen]

@Claus7 --

1) Yes, ubuntu-desktop installer is a Snap Application, installed in the Subiquity installer. This has some advantages, in that the installer can be updated and refreshed, without requiring a respin of the whole image. It can also be set to check for an update at the github, and/or to use a different channel (Stable, beta. etc)...

2) The old installer was Ubiquity , and used partman as the partitioner. First, partman "IS" volume manager aware, so it recognises LVM2, LUKS containers, mdadm RIAD arrays, BTRFS volumes, etc... So it allowed us to set those up from the commandline, start up the installer, choose other > the partitioner saw the layout > you configured the mounts and let the installer continue to install into it...

The new installer's partitioner in not Volume Manager aware. So we lose that ability above...

In the old partitioner, if you used one the the installer scripts... You could change the layout, example /tmp/zsys-setup/layout. You could change sizes of the partitons in the scripts in either /usr/lib/partman/recipes/ (for legacy boot) or /usr/lib/partman/recipes-amd64-efi/ (for UEFU). You could modify the fstab entries in /usr/lib/partman/fstab.d/. And the mount options in /usr/lib/partman/mount.d and /usr/lib/partman/mount.d. So, it the past, we could tweak things even with the canned installers scripts...

I used to jump out to the commandline, setup my own sized partitions, LVM with my own PV's, VG's, LV's... Separate Home's LVM RAID arrays... Jump back into the installer, let the partitioner see it, point the mounts to where i wanted them, then conitinue the installer to install into the filesystem framework.

The partitioner in Flutter... No one will tell me what it is using. Or where it is using files from. It does not even see LVM2: Bug Report: https://bugs.launchpad.net/ubuntu-de...r/+bug/2014977

One of the big complaints about the new installer it that if you use a canned script for the install, you have no control over the size of the EFI partition or boot partition. Because the partiotner is not Volume Manager aware, the only thing you can create manually and still have the installer's partitioner 'see' to use it to install in an "other" option, it just flat partitions, with no Volume Manager definitions.

#3) Eventually, AFAIK, the plans are for the old installer to go away.

[Claus7]

Hello,

thank you for your response.
1) even if it has advantages, I guess that it might be a little bit slower.
2) so you could tamper with the volume manager aware scripts and continue manually with the installations. According to this:
https://wiki.ubuntu.com/Ubiquity/Adv...itionerRewrite
I consider partman and the procedure you are following to be an alternative to gparted, command line oriented vs graphically oriented, in order to setup your installation.
If you have time, could you please explain the term canned script? I have never heard about it. I searched and found that it can be used in screen sharing and other procedures, yet from the name, I suspect that is a somehow routine script.
3) as long as it is called legacy, it seems normal for this to happen, so readjusting seems necessary.

Thank you for your response, hope that soon will be clarified.

Regards!

[MAFoElffen]

@Claus7 --

RE: On the link given in #2 of Post #5...

No. that was Ubiquity and partman, and written in 2008, when they were adding more feature to partman.

This Snap installer app is Flutter installer based is running on top of Subiquity. Partman is not present there any more on the Image. I do not know exactly what it is using... Reviewing installer logs again today.

"Canned default installer scripts" <-- Not sure what the official name is for them, but the scripts the installer uses for:

• Erase and install Ubuntu
• Install Ubuntu alongside (something else)
• Advanced- Erase all & Install LVM
• Advanced- Erase all & Install LVM with Encryption
• Advanced- Experimental Erase all and Install ZFS
• Advanced- Experimental Hardware backed Encryption

[Claus7]

Hello,

...

"Canned default installer scripts" <-- Not sure what the official name is for them, but the scripts the installer uses for:

• Erase and install Ubuntu
• Install Ubuntu alongside (something else)
• Advanced- Erase all & Install LVM
• Advanced- Erase all & Install LVM with Encryption
• Advanced- Experimental Erase all and Install ZFS
• Advanced- Experimental Hardware backed Encryption

Thanks, crystal clear!

Regards!

[MAFoElffen]

I found it...

Buried in the snap image is example autoinstall.yaml scripts for different scenarios...

ubuntu@ubuntu:~$ ls /snap/ubuntu-desktop-installer/1276/bin/subiquity/examples/
ai-zfs-efi-simple.yaml  prefill-system-setup-complete.yaml
ai-zfs-efi.yaml         prefill-system-setup-missing-identity.yaml
ai-zfs-guided.yaml      prefill-system-setup-missing-locale.yaml
answers                 prefill-system-setup-missing-realname.yaml
autoinstall             prefill-system-setup-missing-username.yaml
curtin-events           prefill-system-setup-missing-welcome.yaml
dry-run-configs         snaps
lsb-release-focal       sources
lsb-release-impish      uaclient-status-expired.json
machines                uaclient-status-valid.json
postinst.d              umockdev

ubuntu@ubuntu:~$ ls /snap/ubuntu-desktop-installer/1276/bin/subiquity/examples/autoinstall
ad.yaml                invalid.yaml            system-setup-no-shutdown.yaml
apt-legacy.yaml        most-options.yaml       system-setup.yaml
fallback-offline.yaml  reset-only.yaml         user-data.yaml
hybrid.yaml            short.yaml
interactive.yaml       system-setup-full.yaml

When the Snap first came out, I thought these were just example yaml files, but the installer actually uses them to build the directives for the installer... So, I'm thinking thse can be tweaked to change the outcome of the installs (bigger EFI & Boot partitions).

The installer uses and interacts with those files to build the autoinstall-user-data file, which it uses as if it is the autoinstall.yaml file

sudo grep . /var/log/installer/autoinstall-user-data
[sudo] password for mafoelffen: 
#cloud-config
autoinstall:
  apt:
    disable_components: []
    fallback: abort
    geoip: true
    mirror-selection:
      primary:
      - country-mirror
      - arches: &id001
        - amd64
        - i386
        uri: http://archive.ubuntu.com/ubuntu/
      - arches: &id002
        - s390x
        - arm64
        - armhf
        - powerpc
        - ppc64el
        - riscv64
        uri: http://ports.ubuntu.com/ubuntu-ports
    preserve_sources_list: false
    security:
    - arches: *id001
      uri: http://security.ubuntu.com/ubuntu/
    - arches: *id002
      uri: http://ports.ubuntu.com/ubuntu-ports
  codecs:
    install: true
  drivers:
    install: true
  identity:
    hostname: noble-d02
    password: [redacted-filter]
    realname: [redacted-filter]
    username: mafoelffen
  kernel:
    package: linux-generic-hwe-22.04
  keyboard:
    layout: us
    toggle: null
    variant: ''
  locale: en_US.UTF-8
  network:
    ethernets:
      enp1s0:
        dhcp4: true
    version: 2
  oem:
    install: auto
  source:
    id: ubuntu-desktop
    search_drivers: true
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: false
  storage:
    config:
    - ptable: gpt
      path: /dev/vda
      wipe: superblock-recursive
      preserve: false
      name: ''
      grub_device: false
      id: disk-vda
      type: disk
    - device: disk-vda
      size: 1127219200
      wipe: superblock
      flag: boot
      number: 1
      preserve: false
      grub_device: true
      offset: 1048576
      path: /dev/vda1
      id: partition-0
      type: partition
    - fstype: fat32
      volume: partition-0
      preserve: false
      id: format-0
      type: format
    - device: disk-vda
      size: 2147483648
      number: 2
      preserve: false
      offset: 1128267776
      path: /dev/vda2
      id: partition-1
      type: partition
    - device: disk-vda
      size: 4454350848
      wipe: superblock
      flag: swap
      number: 3
      preserve: false
      offset: 3275751424
      path: /dev/vda3
      id: partition-2
      type: partition
    - fstype: swap
      volume: partition-2
      preserve: false
      id: format-1
      type: format
    - path: ''
      device: format-1
      id: mount-1
      type: mount
    - device: disk-vda
      size: 19112394752
      number: 4
      preserve: false
      offset: 7730102272
      path: /dev/vda4
      id: partition-3
      type: partition
    - vdevs:
      - partition-1
      pool: bpool
      mountpoint: /boot
      pool_properties:
        ashift: 12
        autotrim: 'on'
        feature@async_destroy: enabled
        feature@bookmarks: enabled
        feature@embedded_data: enabled
        feature@empty_bpobj: enabled
        feature@enabled_txg: enabled
        feature@extensible_dataset: enabled
        feature@filesystem_limits: enabled
        feature@hole_birth: enabled
        feature@large_blocks: enabled
        feature@lz4_compress: enabled
        feature@spacemap_histogram: enabled
        version: null
      fs_properties:
        acltype: posixacl
        atime: null
        canmount: 'off'
        compression: lz4
        devices: 'off'
        normalization: formD
        relatime: 'on'
        sync: standard
        xattr: sa
      default_features: false
      id: zpool-0
      type: zpool
    - pool: zpool-0
      volume: BOOT
      properties:
        canmount: 'off'
        mountpoint: none
      id: zfs-0
      type: zfs
    - vdevs:
      - partition-3
      pool: rpool
      mountpoint: /
      pool_properties:
        ashift: 12
        autotrim: 'on'
        version: null
      fs_properties:
        acltype: posixacl
        atime: null
        canmount: 'off'
        compression: lz4
        devices: 'off'
        dnodesize: auto
        normalization: formD
        relatime: 'on'
        sync: standard
        xattr: sa
      default_features: true
      id: zpool-1
      type: zpool
    - pool: zpool-1
      volume: ROOT
      properties:
        canmount: 'off'
        mountpoint: none
      id: zfs-2
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t
      properties:
        canmount: 'on'
        mountpoint: /
      id: zfs-3
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var
      properties:
        canmount: 'off'
      id: zfs-4
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/lib
      properties:
        canmount: 'on'
      id: zfs-5
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/lib/AccountsService
      properties:
        canmount: 'on'
      id: zfs-6
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/lib/apt
      properties:
        canmount: 'on'
      id: zfs-7
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/lib/dpkg
      properties:
        canmount: 'on'
      id: zfs-8
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/lib/NetworkManager
      properties:
        canmount: 'on'
      id: zfs-9
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/srv
      properties:
        canmount: 'on'
      id: zfs-10
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/usr
      properties:
        canmount: 'off'
      id: zfs-11
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/usr/local
      properties:
        canmount: 'on'
      id: zfs-12
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/games
      properties:
        canmount: 'on'
      id: zfs-13
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/log
      properties:
        canmount: 'on'
      id: zfs-14
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/mail
      properties:
        canmount: 'on'
      id: zfs-15
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/snap
      properties:
        canmount: 'on'
      id: zfs-16
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/spool
      properties:
        canmount: 'on'
      id: zfs-17
      type: zfs
    - pool: zpool-1
      volume: ROOT/ubuntu_nshf9t/var/www
      properties:
        canmount: 'on'
      id: zfs-18
      type: zfs
    - pool: zpool-0
      volume: BOOT/ubuntu_nshf9t
      properties:
        canmount: 'on'
        mountpoint: /boot
      id: zfs-1
      type: zfs
    - path: /boot/efi
      device: format-0
      id: mount-0
      type: mount
    swap:
      size: 0
  timezone: America/Los_Angeles
  updates: security
  version: 1

So playing with that and seeing how and what can be done with that...

EDIT: Reading through these files, I actually learned that Subiquity got it's name "[S]ubiquity" from "Ubiquity for Servers"...

[lammert-nijhof]

I already try for many weeks to install Ubuntu 24.04 in Virtualbox, but the new installer is crap! It freezes at the the user / password form all the time. I tried everything; safe graphics, 3D graphics ON/OFF; VBoxVGA. I tried it on my Ryzen 3 2200G desktop and on my old laptop with an i5-2520M. Maybe in desperation I will clone Ubuntu 23.10 and try a very early system upgrade to 24.04.
I have installed 4 flavors without any issue in VBox, fortunately they still use the old installer.
I completely agree with the reorganization of the Ubuntu installer development team. If that has no results in the next fortnight, cancel the whole project and go back to the proven installer still used by the flavors.

[MAFoElffen]

@lammert-nijhof --

Keep an eye on this thread: https://ubuntuforums.org/showthread.php?t=2492322

Everyone has been pretty good on updating it with which dailys are working or not, with which bugs noticed on each.

Also check the ISO tracker, where we log the daily ISO QA tests: https://iso.qa.ubuntu.com/ <-- quiverc is really on top of tracking that.

I do daily, then have other that were debian-upgraded (via sources.list) from Mantic, that I keep around to track daily updates to them.