Windows-based Ubuntu Installer Development Thread (merged parts 1 and 2)

[tuxcantfly]

I agree, at any time there should be 4 files

"Stable" installer
Experimental installer (same as the above, but maybe make it red or something)
Latest Backend (tgz) (required for other platforms)
Latest Source (tgz)

Well what I was thinking of was keeping all the source files, uncompressed (around 10 MB) on the cutlersoftware server as an cvs-like repository (only in http), then combine Ago's initrd-builder with my installer-builder and add in a wget -r portion that downloads the latest source, so we essentially have an installer-autobuilding mechanism, so what we'll have is:

"Stable" installer
Experimental installer that is automatically built using the scripts
Uncompressed source on the server

If you ask me, this would be way easier to maintain than a bunch of tarballs in the server; instead of having to download, uncompress, recompress, create an installer, upload, you can just edit it directly on the server with an ftp client, and run the script to automatically build an installer

[Hello1024]

it was mainly the installing packages that took a long time, as it seemed to take a few minutes for every package, with the HD light on solidly throughout, so I suspect some PIO/DMA problem as ago suggested. I think the same issue might be continuing after installation, because it still takes 3 min 50 sec to get from the boot menu to the desktop. The creation of the image files by nsis takes next to no time now (< 1 sec per file), although I don't know about formatting them, although I suspect thats quick as well.

I'm still unclear about which way you're planning on modularizing it - are you planning to have the nsis setup exe extract the batch file and execute it hidden, which in turn executes the nsis setup.exe once for every dialog, with different command line parameters so it knows which dialog to show? Or are you using nsis as the base, which then executes different batch files (hidden) to do parts of the hardware detection/installation?

Either way, I still think it's best to modularize it at the source code level (different script files linked with includes etc.), but not at the end user side (because using batch files are really fairly inflexable when it comes to doing a lot of complex tasks, and it would be better to have different plugin dlls, if we're seperating it to allow parts to be compiled seperately).

As far as a build process goes, I think it would be good to good to be able to add the nsis installer compilation to the make process, although it would require wine I think because the linux port of the nsis installer isn't fully tested yet.

[Hello1024]

I don't know how much of a priority everyone thinks hardware detection is, but it would be quite easy to get windows to spit out a list of all the hardware id's of every piece of PnP hardware. That then needs to be checked against some kind of online database to check for compatibility out of the box, with special drivers/tweaks, partially, or not at all. However there don't seem to be any good hardware databases around (I don't mean wiki lists, but rather searchable databases so a script can search it and highlight to the user any potential issues before installation). To start with, it should just highlight potential flaws (eg. your disk controller isn't supported at all), as most driver issues are best sorted from within ubuntu itself, unless of course it's a vital driver.

We either need to find an ubuntu hardware database and customise it to do the searches we need, or create our own (which wouldn't be as much work as it seems if users submit the data). It would be a very good way to see which hardware needs supporting, because on every installation, with the users permission, unsupported hardware can be logged and then a centralized system counts the most common problem hardware. Every PC does have a LOT of hardware, usually many hundreds of devices, so we certainly need an auto-search.

All that sounds great, but the problem is while I can make and support the hardware detection code, I don't think I have time to make a server-side searchable user friendly hardware database. (php and mysql projects always seem to take longer than I expect), so the real question is does any web developer out there like the idea of this as a project? It would probably encompass all the other main linux distros as well, with the possible creation of a compatibility checker that can run on windows/linux to show which linux distros are supported on any given hardware.

[tuxcantfly]

I'm still unclear about which way you're planning on modularizing it - are you planning to have the nsis setup exe extract the batch file and execute it hidden, which in turn executes the nsis setup.exe once for every dialog, with different command line parameters so it knows which dialog to show? Or are you using nsis as the base, which then executes different batch files (hidden) to do parts of the hardware detection/installation?

The way I plan on having it happen is to have a 7-zip auto-self-extractor run first (takes roughly 10 sec here) then that will launch a batch script in the background (with the echo off option, so it isn't seen) and the batch script calls the individual NSIS dialogs that were compressed within the self-extractor. In other words, the user sees it extracting for 10 sec, then a nice nsis dialog pops up, asks a few questions, another one pops up, etc. The main advantage would be that we can easily modularize the installer; just add in another entry into the setup.bat file in the 7z self-extractor, add the new nsisdialog.exe file into the self-extractor, and it'll run. Also, we can have extra functionality outside of NSIS; just add it into the batch file, add the .exe file into self-extractor, and it'll work. Then, whenever you need to build the installer, you just this script: http://cutlersoftware.com/ubuntusetup/buildsfx.sh and it will do it all for you, with a few modifications it can just download the source itself and build automatically

something like this: http://cutlersoftware.com/ubuntusetu...up-di-exe.html only without the command line ugliness; instead, we'll have NSIS dialogs

[tuxcantfly]

by the way, Ago, whenever I try to run your build script, I cd to the directory, run the make command, and I get this:

geza@ubuntu:~/ubuntubuild$ make
Cleaning...
Building...
Error: invalid binary /usr/bin/fusermount
geza@ubuntu:~/ubuntubuild$

I've got fuse-utils installed, though, and /usr/bin/fusermount is there... any idea what could be wrong?

[Hello1024]

It's an idea, but I'm still not sure I like the idea of batch files. How about replacing the batch files in your design with say VBScript files (there is a vbscript interpreter in windows, just like the batch interpreter in windows). That gives a lot more power, while still keeping the open un-compiledness of a batch file.

IMO, it would be better to seperate all the dialogs like you suggest to seperate source files, but then have them all compiled together by the nsis compiler, because that still allows data to be easily passed between windows\forms. It also allows for files to be extracted sequentially to speed up the responsiveness of the installer. There is also the problem that some things the current nsis installer does that are basicly impossible to accomplish with batch files, like calling Kernel32::SetFileValidData to stop the kernel zeroing out a newly created file. One other thing is the batch file must do it's own window management in that case to hide the black dialog box, unless the 7z extractor can do that.

[tuxcantfly]

One other thing is the batch file must do it's own window management in that case to hide the black dialog box

oh yeah, I can just remove the lines that say "echo" to do that, it won't pop up then; it'll just run in the background and only the nsis dialogs themselves will show up

How about replacing the batch files in your design with say VBScript files

why VBScript, though? isn't a batch file good enough for calling .exe files? I don't see the point; the batch file is just meant to wrap together all of the sub-installers, not be the installer itself

There is also the problem that some things the current nsis installer does that are basicly impossible to accomplish with batch files, like calling Kernel32::SetFileValidData to stop the kernel zeroing out a newly created file.

I think you've misinterpreted my statements. If it can't be done in a batch file, we'll do it in an nsis installer that the batch file calls, and if NSIS can't do it either, we can just call a custom executable from the batch file

It also allows for files to be extracted sequentially to speed up the responsiveness of the installer.

No real point, if you ask me; the current version of my 7z-based installer takes 10 sec to extract, and doesn't do any more extracting after that; we can keep the nsis installers uncompressed, as they'll already be compressed by 7-zip in the self-extractor

[ago]

then combine Ago's initrd-builder with my installer-builder and add in a wget -r portion that downloads the latest source, so we essentially have an installer-autobuilding mechanism

I think that back-end and front-end should be kept separate. For one thing: the back-end is platform neutral, the front-end is not (and not only because of the GUI but also because of the modifications required to the host bootloader and the mechanisms for fetching information).

[Hello1024]

I'm just thinking the batch files can't be as simple as you envision, because they've got to make the decisions as to which forms should be displayed, and the installer now isn't as linear as it used to be, since there are lots of routes through the program:

Normal > edition select > username
Expert > Loopback > data source select > destination select > advanced options
Expert > Burn CD > data source select > cd burn options
Expert > Partition install > partition select (as yet not made)

Before deciding which routes to take, or allow the user to take, the code will have to manage, for example handling of resuming partial installations, detecting a broken installation, detecting if bootloader repair is required, detecting if the torrent download is complete etc.

The main issue is that in that case, it has to be the batch file that detects those things, since they have an effect on which forms need to be shown in which order to the user, and in my opinion quite a lot of those things would be easier to do with a more powerfull scripting language (vbscript, nsis, maybe even .net, java, python etc.)

Also, I didn't manage to figure a way without external apps to hide the batch file, because even with @echo off, the black box with flashing cursor is still shown, even if it stays blank.


EDIT: re ago's comment, yep, agreed keep the two seperate, but we still want a building system for the frontend, unless it's a single command as it is now. (just run makensisw)

[ago]

I don't know how much of a priority everyone thinks hardware detection is

I can add that. It was in the previous initrd (I believe v0,0.0.4). Can you please try that? Can you also try to find out what drivers your HD/controller is using?

but it would be quite easy to get windows to spit out a list of all the hardware

But it would be complex to remap all hardware, Linux autodetection is good no need to have windows help for that.It is likely my fault, not Linux's, I must have skipped something.