Howto: NoNetDebs - upgrade Ubuntu without Internet (or with low-bandwidth connection)

[epimeteo]

Hi,

NEW: major changes. New host and new features - see post #49.

For those accessing the Internet from a library, school, college, workplace, from windows, etc.

Just upload the status file from the offline Ubuntu (the complete pathname is /var/lib/dpkg/status) to your user account, select the ubuntu release, check if you want to upgrade or not, add a list of packages you want to install, click "List packages", wait a minute and you will get a list of the packages you need and you still don't have installed on your ubuntu offline, just like if you had done "sudo apt-get install".


http://nonetdebs.homeip.net

or http://nonetdebs.unixpod.com

To see how it works check post #11.
VERY IMPORTANT: To install ubuntu-restricted-extras offline in Gutsy check this post. If you don't do it like posted on that link, you'll get broken packages and a broken system!


================================================
The following is here just for historic curiosity and concept.
Please read post #11 and #49 for about nonetdebs website.
================================================

last few days I've been messing around with this BASH script I've made - nonetdebs. It's fresh (not wide testing done yet, just here) and I need some feedback about if it is useful and if it really works.

nonetdebs can download or list the packages you need to download for your Ubuntu at home with no Internet connection.



It is somehow related/based in this http://ubuntu.wordpress.com/2005/09/...slow-internet/ and this http://beans.seartipy.com/2006/05/06...et-connection/ but done the automatic and easiest way (with a nurcses GUI), plus the independence of which Linux/Ubuntu version you are running (nonetdebs creates a chroot, so nothing is installed on the connected system, the packages are just downloaded) and plus avoids downloading already installed packages (the file you bring from the other computer tells what packages are already installed there).

nonetdebs can be an alternative to APTonCD and/or APT-move:
- APTonCD and APT-move need an equivalent version of Ubuntu on both computers, nonetdebs don't;
- Creating DVD mirrors of the repositories is a big download and needs frequent updates, nonetdebs just downloads upgrades and new packages you want, knowing the packages you already have. It is a more individualized option.


What you need:

1. The status file from your Ubuntu box. You can email if to yourself (~2 MB, maybe 5 MB, 10 MB max, it is located at /var/lib/dpkg/status), or copy it to a CD-RW or USB memory stick/pen.

2. A computer that have Internet connection.

Option 1: A computer, as long as you can boot it with a LiveCD (Ubuntu Desktop?) and have an Internet connection with it. For real Windows, maybe you can use Wubi to install Ubuntu in it and use this script, or use cygwin --- all untested options except for the LiveCD (testers needed). A writeable partition from the LiveCD: ext3, something writeable. Many possible untested options here... (except FAT32, debootstrap apparently doesn't like it)

Option 2: Any Linux box with Internet, as long as you can install debootstrap and dialog and you know its root password (k/x/ubuntu, fedora, whatever). BASH and wget are also needed, but are generally available in almost all Linux distros.

How it works:
* on the following text, "the other computer" is the Ubuntu without Internet.

The complicated way:

1. You copy the file /var/lib/dpkg/status from the other computer to a USB pen/stick or to a CD.

2. You take a LiveCD and a USB pen to the computer with internet and boot it.

3. You install "dialog" and "debootstrap" packages on the LiveCD, then you download and run nonetdebs script on a terminal.

4. You setup nonetdebs (mirrors for downloading, locale, Ubuntu version, status file).

5. You select upgrades, select new packages and make the downloads to the USB memory.

5. You go home.

6. You plug the USB pen on the other computer, open a terminal, type

cd /media/disk
./nonetrepo

sudo apt-get update
sudo apt-get --allow-unauthenticated upgrade
sudo apt-get --allow-unauthenticated install packages_names

The simple way:

1. You have a copy of your status file from the other computer (email attachment, or on CD or USB);

2. You download the packages "dialog" and "debootstrap" and run the nonetdebs script (download it from here and make it executable);

3. nonetdebs downloads, setups and creates a chroot with debootstrap of the Ubuntu version of the other computer on the connected computer (on USB or disk - ext3/something_writeable and with the needed space available - 200 MB minimum ).

This will make a download of a minimal Ubuntu of the Ubuntu version you have choose (50 MB of download, 200 MB max of disk or USB space). This is a one time download. The next time you use nonetdebs, you can select this chroot again, no more chroot downloading needed. You can even create a backup of that chroot from the script;

4. You copy the status file (from the other computer) to the chroot (done from the script). With the status file, the chroot will «know» what packages you have at home and will know which one to upgrade and/or install -- less downloads needed, more accurate upgrades;

5. You select to upgrade and/or install any new packages you want from the repositories. The script will only select the packages you don't have on the other computer, and will select all dependencies except the ones you already have installed (less download volume);

6. once nonetdebs has created a list of the files you need to download (this list will be inside the chroot if, for some reason, you can't download and save the packages with the LiveCD, or if you have a low-bandwith at home and can't/don't_want_to make the downloads there and only have Internet access from a Windows box, etc). The list is the internet address of the packages needed on the repositories (check post #2 to download it from Windows).

7. You download the packages nonetdebs will create a script when you download the packages - nonetrepo - and puts it in the same directory where you saved your packages. This script - nonetrepo - is to add the new repository without too much trouble (easier).

8. OPTIONAL: you can create an ISO file and burn it from Windows ((reboot needed here - look at post #2).

9. You go home (where the other computer is) and run

./nonetrepo

from the directory where the packages are, to had the new repository. You can do this directly on the USB memory stick/pen or copy the files from a CD to the hardisk and run the script from there (a writable directory/partition is needed here for dpkg-scanpackages work, although some tweaking might work to avoid copying the files to disk).

10. You're done. Update APT (Synaptic or apt-get) and install the new packages. Use

sudo apt-get --allow-unauthenticated upgrade

sudo apt-get --allow-unauthenticated install name_of_the_packages

to install them. Update-manager will not work, because the files aren't signed.

More Info
I've tested it upgrading a Ubuntu 7.10 Gutsy Tribe-5 to its latest updates (more than 400 packages upgraded) with a LiveCD and a USB memory pen, then I connected it to internet, add the regular repositories, and no more needed updates were found.

Note to aptitude users: although it is possible to make apt-get download "recommended packages" (to be used with aptitude on the other computer) it doesn't work with Dapper, so I've not enabled this option, but it can be done with Feisty/Gutsy (Edgy not tested) just with some tweaking (with apt.conf on the chroot directory and APT options).

If you try to write to non-writeable partitions, the script will tell you that you can't.

TODO: nonetrepo - check on the other computer if the new repository already exists on source.list (duplicates).
WISHLIST: nonetdebs - gtk frontend (zenity).

The nonetdebs script can be found here: http://ubuntu.no.sapo.pt/nonetdebs/nonetdebs
("dialog" and "debootstrap" also packages required).

[epimeteo]

Thank god nobody found this thread here .

I say that for 2 reasons:

- The script I released yesterday (0.124) had some syntax errors (wasn't working);
- I was claiming that you could run it with a FAT32 partition, my bad.

Forgive me about the syntax errors, I've tested the script before I've made some changes, and found that it wasn't working. But it's all right now.

About FAT32, I should know that FAT32 wouldn't work with debootstrap. I was testing here the "worst case scenario" (saving the chroot to a FAT32 partition) and found that debootstrap changes the owner of some files and FAT32 apparently doesn't support that (FAT32 is a 1 owner filesystem, sorry about that ). At the moment I even can't assure anybody that a debootstrap in NTFS would work either (can somebody confirm this, please?).

So, if you need to upgrade you system and only have access to a computer with a M$ filesystem (FAT32 or NTFS) and you don't have a USB memory pen, I advise you to do the following:

- Run the script and create the chroot in $HOME.

- Save the list of files to http://rafb.net/paste/ or any similar website. The list of files you selected is located in /var/*.txt inside the directory you choose to create the chroot with debootstrap (for example: $HOME/nonetdebs/var/list_upgrades.txt). You can use Firefox from the LiveCD to do that.

- Then use any Windows software you like to download these files (it is a list of addresses of the packages in the repositories, like this: http://archive.ubuntu.com/ubuntu/poo...buntu1_all.deb).

Of course, if you have a USB memory pen/stick, and you formated it with a ext3 filesystem, you will not have any problem and you can download the packages directly into it and install these packages directly from it on the other computer.

This time it's for real, it is working.

Please, any feedback is welcome.

[urukrama]

This could be exactly what I'm looking for. I have a computer that isn't always connected to the internet, and installing something on it can be such a drag. Thank you! I'll give it a try next time I want to install something.

[epimeteo]

I hope it works for you urukrama. Any question or problem, please feel free to ask

I've added a little improvement. Until now the script would copy all the packages that debootstrap used to create the chroot to the download directory (~170 of them) when the user wanted to update/upgrade the other computer. I've corrected that.

Now the script only copies the packages that it finds on the "already downloaded packages" and are not on the download directory and are listed on the list_upgrades.txt file.

This saves some space used to move around the packages (CD or USB) and I think it will avoid any APT/dpkg problems on the other computer (and with nonetrepo) because of unneeded and/or duplicated packages on the other computer.

Also corrected the creation of download directory. It was being created, but the script was saying that the directory already existed.

[blithen]

I haven't tried it yet, but this is defiantly a great idea. Looks very offical. Great job!

[epimeteo]

Hi again,

since the Canonical repository for Gutsy is named "gutsy partner" instead of (for example) "feisty commercial main" on the other releases, I've updated the script to reflect that change.

I've tried to «install» opera in Gutsy and the files were listed ok, so the script works fine with Canonical repos and Gutsy.

[dhruva023]

thanks, i will be trying it

[epimeteo]

Some corrections today.

Apparently I wasn't using a completely "new" gutsy install during my testings. When I tested the script again on a fresh «target», it didn't had the package dpkg-dev installed (dpkg-scanpackages, used by nonetrepo), so I made the script download it and install it (including any dependencies) if it isn't present on the target.

** tested in a fresh install of alternate desktop cd of gutsy - after upgrading the target (9 packages) with nonetdebs/nonetrepo, I've connected it to the internet and no more upgrades were needed **

[epimeteo]

For those who can't boot a LiveCD on the workplace, library, cybercafe, etc, I've adapted the script to work from a website.

Just register, set your profile, upload the status file, click "List packages" and wait a minute for the list of the packages you need to download.

For now it's just for i386 (dapper/edgy/feisty/gutsy), no medibuntu nor canonical repos.

http://nonetdebs.homeip.net

[Sef]

Seems more appropriate here than Community Cafe.