How To: Make Your Own Ubuntu Repository DVDs

[Sensiva]

I have actually tried to run debmirror in Windows, just out of curiosity, but ran into a number of issues that would take far too much work to sort out.

How about manually doing what debmirror actually does using GnuWin32 utilities (wget grep awk sed..etc)??

With a batch script, or powershell script, I think it can be done.

[luvr]

How about manually doing what debmirror actually does using GnuWin32 utilities (wget grep awk sed..etc)??

With a batch script, or powershell script, I think it can be done.

Should be doable, indeed--but I won't be the one to do it... I'm just not sufficiently motivated to work on a Windows solution.

Your suggestion did make me wonder about developing an alternative to debmirror under Linux, though. If it can be done using the common utilities, then it should not be all that hard to port the solution to Windows. From my past experience with the GnuWin32 utilities, I do remember that the differences in command-line handling between Linux (bash or dash) and Windows require some modifications when copying over scripts between the two systems, though.

[luvr]

Thanks for the idea and the info. Script updated

Here's another neat little command that I have just discovered:

dpkg --print-architecture

This command returns the machine architecture as the Debian package management software sees it (e.g., "amd64," "i386," etc.). It may simplify how your script determines the default architecture.

[Sensiva]

Here's another neat little command that I have just discovered:

dpkg --print-architecture

This command returns the machine architecture as the Debian package management software sees it (e.g., "amd64," "i386," etc.). It may simplify how your script determines the default architecture.

That's Great! Thank you! Script updated

One more thing I really wish to add to that script is checking if the necessary tools are installed, and install them if not. I need to read more about exit codes and stuff in bash.

[k3lt01]

Here is a link to an SIL repository that I would really like to download and put on to my hard drive. In this folder they have all the Ubuntu varieties from Breezy to Hardy and even Debian Etch, Sarge and Sid.

http://packages.sil.org/ubuntu/dists/

Just thought I'd let you know that I have got this downloading successfully. Woo Hoo I took a look through the Man pages for debmirror and modified the code, quite a few times, till I finally got it working. I started it on my laptop but will do a complete download on my desktop tonight.

The original problem seemed to stem from the release and package files, that at least is the details debmirror gave with its error report. So I checked the Man pages for debmirror and remembered that Luvr stated in a thread that there was a difference in a previous version of debmirror that meant the new version needed a patch to enable it to download certain files (packages.gz in preference to packages.bz2 from memory). So I uninstalled the Jaunty>Karmic version (debmirror_20070123ubuntu3_all) and installed the Intrepid and earlier version (debmirror_20070123ubuntu1_all). I then added the "--ignore-missing-release" argument to the script and it worked.

I learned 2 lessons today. 1, the latest isn't always the greatest, sometimes you lose functionality by going "bigger and better". 2, I can understand the Man pages, lol.

[k3lt01]

Has anyone tried using using mirrordir? I have had a little play and it looks really good. However, it literally does mirror an entire directory so if you only want 1, 2 or even 3 dists then you don't want to use mirrordir because you WILL end up with everything on the server.

For those who want to play around check out the mirrordir man page.

Now that reading the mirrordir man page has totally blown your mind with excitement and has convinced you to go ahead with trying this you need to do a couple of things. Before I keep going, by reading further and acting on the instructions below YOU take all responsibility for what happens to your system and your consequent Telephone/Internet Bill. Now we have that out of the way you need to:

1. Download and install mirrordir itself along with other required tools to create the DVDs.

 sudo apt-get install mirrordir liblockfile-simple-perl liblog-agent-perl ruby mkisofs dpkg-dev libdigest-sha1-perl libruby libzlib-ruby

mirrordir is a direct replacement for debmirror. I have them both because debmirror will download what you tell it to download while mirrordir will download the entire directory. Also mirrordir only downloads ftp directories so debmirror is great when you want to do http directories.

2. Create a folder in your /home directory to download the relevant directory into. Mine is setup like so

~/Ubuntu/Repositories/archive.ubuntu.com

the ~ at the beginning of the code indicates the /home folder. When naming your folders please use names that make sense. I find mimicking the names of the native archives helps me to know what is what on my PC. BobSong's original post has an excellent explanation of the reasoning for naming your folders.

3. In a terminal paste the following command.

mirrordir -v ftp://archive.ubuntu.com ~/Ubuntu/Repositories/archive.ubuntu.com

This will now start downloading EVERYTHING that is in archive.ubuntu.com and place it into your new folder.

NOTES:
1. You MUST, repeat MUST, direct mirrordir to download into your newly created folder (just like I have indicated in the creation of the folder and the command itself) OTHERWISE YOU WILL DELETE EVERYTHING in your /home folder and over-write it with the contents of archive.ubuntu.com. You have been warned.

2. This method replaces part of steps 1 in BobSongs original post. Everything else works the same. As already explained I use both setups.

3. If you need to stop the download or it drops out you can restart it easily by doing the terminal command again, it will not re-download anything that you already have already completely downloaded instead it will continue on from where it left off.

4. Be aware when Canonical remove a distribution from archive.ubuntu.com mirrordir (and debmirror to for that matter) will delete the now obsolete dist from the folder it is contained in. This means IF you want to keep it you will need to copy the relevant files to a separate folder. To accomplish this I copy the old dist (e.g. Intrepid) to my old-releases.ubuntu.com folder about 1 month before its official End-Of-Life. 2 months after that I run my old-releases.ubuntu.com script to catch any updates I missed in the meantime.

[dreamsR4living]

I have a question about step 4 of Bobsongs tutorial. When I use debcopy, I get messages about a lot ignored files and also 2 non-existence files.

Does that mean that some packages will not be included on the DVDs, which means that I cannot have the full repositories on DVD? Or can these messages be ignored?

Bobsongs, thanks alot for your clear tutorial It helped me out so far!

[k3lt01]

Can you post the exact messages you get, or a relevant selection, for us to see?

[dreamsR4living]

sebastiaan@sebastiaan-desktop:~$ ruby debcopy -l /media/MyBook/UbuntuRepos ~/UbuntuDVDs/ubuntu6
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty/restricted/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty/main/binary-i386/Packages.gz... /media/MyBook/UbuntuRepos/libtotem- not found.
done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty/multiverse/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty/universe/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-updates/restricted/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-updates/main/binary-i386/Packages.gz... /media/MyBook/UbuntuRepos/libtotem- not found.
done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-updates/multiverse/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-updates/universe/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-security/restricted/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-security/main/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-security/multiverse/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-security/universe/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-backports/restricted/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-backports/main/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-backports/multiverse/binary-i386/Packages.gz... done
Processing /home/sebastiaan/UbuntuDVDs/ubuntu6/dists/jaunty-backports/universe/binary-i386/Packages.gz... done
Number of Copied Files: 3124
Number of Ignored Files: 98
Number of Non-existence File: 2

This is the output when I use debcopy to create the last (nr. 7) Ubuntu Jaunty repositories DVD. This is the only DVD where I actually get two non-existence files. It is telling "libtotem not found" for 2 times. I also have about 100 ignored files for each of the 7 DVD's. Does this mean that some packages are not downloaded completely, or not at all? Do I have to download all over again?

[dreamsR4living]

Hi, I am trying to install packages from the DVD's in a virtual Ubuntu 9.04 machine, but I am getting Hash Sum mismatches all the time... Is there any way to solve this? I have included a screenshot of the problem.

I am busy to create and test those dvd repositories for two weeks now. I am going to Africa this week to try and set up a computer teaching project, but this hash sum is making me pull my hair out right now. I don't have any internet connection there, so I need these disks to install new packages.

Is there anyone who can help me out?