How To: Make Your Own Ubuntu Repository DVDs

[BobSongs]

I'm not sure if anyone beside me here has done this but...

...but I think I should have a link to this post too.

Everything and anything that's practical and related to improving this thread is welcome.

Thanks!

[Geochelone]

 ! Package libkexiv2-dev (filename ./pool/main/k/kdegraphics/libkexiv2-dev_4.1.4-0ubuntu1~intrepid1_i386.deb) is repeat but newer version;
   used that one and ignored data from ./pool/main/k/kdegraphics/libkexiv2-dev_4.1.3-0ubuntu1~intrepid1_i386.deb !

I ran scanpackages, and terminal returned many similar errors (?) as above. What's happening actually? How's this affect my repo?

[jms1989]

 ! Package libkexiv2-dev (filename ./pool/main/k/kdegraphics/libkexiv2-dev_4.1.4-0ubuntu1~intrepid1_i386.deb) is repeat but newer version;
   used that one and ignored data from ./pool/main/k/kdegraphics/libkexiv2-dev_4.1.3-0ubuntu1~intrepid1_i386.deb !

I ran scanpackages, and terminal returned many similar errors (?) as above. What's happening actually? How's this affect my repo?

This is because it found a newer version then the older one or in this case v4.1.4 is newer than 4.1.3. One solution to use to get rid of the warning is to delete the old version.

rm ./pool/main/k/kdegraphics/libkexiv2-dev_4.1.3-0ubuntu1~intrepid1_i386.deb

Of course, if you chose to keep it, you will continue to see the warning but its safe to keep unless you are running low on space.

Cheers!

[luvr]

And how is this thread --> http://ubuntuforums.org/showthread.php?t=1090731

related to our thread here?

I think the threads are very much related indeed, in that they basically attempt to solve the same problem: How to keep Ubuntu up-to-date on a computer that has limited or no internet access.

BobSongs' approach actually has a few interesting advantages over mine, in that it will download a fully functional repository--including the control files (i.e., the package list, the release file, and even a valid digital signature)--without any further manual intervention. With my approach, you need to update the control files and regenerate your own digital signature whenever you update the local repository. (Note, though, that with my method, you can add your own packages that are not available in any online repositories. You cannot do that unless you're prepared to create your own control files, including the digital signature.)

Also, if I understand correctly, the approach as documented here will automatically clean up your local repository: Any old package versions that are no longer available from the original repository will automatically be removed from your local copy as well. My method, on the other hand, doesn't provide any form of automatic cleanup.

The only potential issue with this method is the humongous amount of data that you will have to download initially, when you want to set up your local copy. But once you're past this potential hurdle, I think it should work great. I think I might give it a try when Jaunty "final" gets released.

As it happens, Ubuntu's parent distribution, Debian, apparently provides pre-built repository CDs and DVDs of its Lenny (5.0) release--cfr. the Debian web site and navigate to the CD or DVD download pages for more info. (In fact, there's even a Blu-Ray disc image available!)

Also, as BobSongs notes in the original post, the debpartial package is no longer available in the Ubuntu repositories. Debian, on the other hand, still does provide it--cfr. search results for debpartial in Debian Lenny.

[Geochelone]

Thanks jms1989 for your very clear explaination.

Why debmirror don't clean the duplicates? Intrepid's scanpackage took significant hours to scan and display those dupes. Any bash script to auto remove them?

[BobSongs]

<snip>Also, if I understand correctly, the approach as documented here will automatically clean up your local repository: Any old package versions that are no longer available from the original repository will automatically be removed from your local copy as well. My method, on the other hand, doesn't provide any form of automatic cleanup.</snip>

You're 100% right on that one. In fact, this cleaning up is why more than one repository cannot be easily "blended" with another. There have been attempts at it... some more successful than others... but I maintain the idea of keeping the repos separated for sanity's sake. Adding a few extra CDs when handing a friend the repos is not all that much more drudgery.

The only potential issue with this method is the humongous amount of data that you will have to download initially, when you want to set up your local copy. But once you're past this potential hurdle, I think it should work great. I think I might give it a try when Jaunty "final" gets released.

True! This is why the tutorial is prefaced with an abundance of warning about how much hard drive space will be consumed. For older machines with smaller drives (60 Gb), this can be a problem.[/quote]

By the way, luvr, just wanted to say that I appreciate your tutorial's layout.

[jms1989]

Thanks jms1989 for your very clear explaination.

Why debmirror don't clean the duplicates? Intrepid's scanpackage took significant hours to scan and display those dupes. Any bash script to auto remove them?

Sorry I didn't get back to you sooner, debmirror is supposed to delete the dupes after it finished downloading the Packages and Sources files. However, you can have debmirror delete them by running (debmirror --cleanup ~/UbuntuRepos) or something to that effect I suppose.

[Geochelone]

I don't think debmirror will remove the duplicates. According to the debmirror description, 2nd step of debmirror is

1. First it downloads all Packages and Sources files for the subset of Debian it was instructed to get.

2. Any files and directories on the local mirror that are not in the list are removed.

3.The Packages and Sources files are scanned, to build up a list of all the files they refer to. A few other miscellaneous files are added to the list. Then the program makes sure that each file in the list is present on the local mirror and is up-to-date, using file size (and optionally md5sum) checks. Any necessary files are downloaded.

So, those old (previously downloaded packages) are there in my mirror, and listed in the newly downloaded Packages list, debmirror won't delete them.

So I read the man again, and I found this option.

"--max-batch=number" : Download at most max-batch number of files (and ignore rest).

What does it mean actually? Is this option set maximum no of each package? For example:
abiword-common_2.6.6-0ubuntu1_all.deb,
abiword-common_2.6.4-4ubuntu4_all.deb, and
abiword-common_2.6.6-0ubuntu1_all.deb

If I set the max no to 1, will debmirror keep ONLY the latest (abiword-common_2.6.6-0ubuntu1_all.deb), and delete the rest (abiword-common_2.6.6-0ubuntu1_all.deb and abiword-common_2.6.4-4ubuntu4_all.deb)?

[Mehashi]

Thank you BobSongs!

This guide is so well written, even I could follow it! I have been wanting to do this for a friend with no internet and now I know how, Thank you!

Your section on pointing apt locally will really help for me with my laptop as well, as it has no wireless and my router is nearly always full, now I can just put the debs on there and not worry anymore!

Thanks again for a well written guide!
Domo arigato!
^_^

[Geochelone]

This is what I got running debmirror in Jaunty

Mirroring to /media/Tortoise/Repositories/Jaunty/Canonical/ from http://archive.canonical.com///
Arches: i386
Dists: jaunty,jaunty-backports,jaunty-proposed,jaunty-security,jaunty-updates
Sections: partner
Passive mode on.
Pdiff mode: use.
Attempting to get lock, this might take 2 minutes before it fails.
Get Release files.
[0%] Getting: dists/jaunty-backports/Release... ok
[0%] Getting: dists/jaunty-proposed/Release... ok
[0%] Getting: dists/jaunty-security/Release... ok
[0%] Getting: dists/jaunty-updates/Release... ok
[0%] Getting: dists/jaunty/Release... ok
Get Packages and Sources files and other miscellany.
[ 93%] Getting: dists/jaunty-backports/partner/binary-i386/Packages.bz2... dists/jaunty-backports/partner/binary-i386/Packages.bz2 failed 404 Not Found
[ 93%] Getting: dists/jaunty-proposed/partner/binary-i386/Packages.bz2... dists/jaunty-proposed/partner/binary-i386/Packages.bz2 failed 404 Not Found
[ 93%] Getting: dists/jaunty-security/partner/binary-i386/Packages.bz2... dists/jaunty-security/partner/binary-i386/Packages.bz2 failed 404 Not Found
[ 93%] Getting: dists/jaunty-updates/partner/binary-i386/Packages.bz2... dists/jaunty-updates/partner/binary-i386/Packages.bz2 failed 404 Not Found
[ 93%] Getting: dists/jaunty/partner/binary-i386/Packages.bz2... dists/jaunty/partner/binary-i386/Packages.bz2 failed 404 Not Found
Missing: .temp/dists/jaunty/partner/binary-i386/Packages
bzip2 -d <.temp/dists/jaunty/partner/binary-i386/Packages.bz2 >.temp/dists/jaunty/partner/binary-i386/Packages
sh: cannot open .temp/dists/jaunty/partner/binary-i386/Packages.bz2: No such file
Failed: bzip2 -d <.temp/dists/jaunty/partner/binary-i386/Packages.bz2 >.temp/dists/jaunty/partner/binary-i386/Packages
WARNING: releasing 1 pending lock...
+\ 
| UPDATING CANONICAL - INTREPID
+/ 
Mirroring to /media/Tortoise/Repositories/Intrepid/Canonical/ from http://archive.canonical.com///
Arches: i386
Dists: intrepid,intrepid-backports,intrepid-proposed,intrepid-security,intrepid-updates
Sections: partner
Passive mode on.
Pdiff mode: use.
Attempting to get lock, this might take 2 minutes before it fails.
Get Release files.
[0%] Getting: dists/intrepid-backports/Release... ok
[0%] Getting: dists/intrepid-proposed/Release... ok
[0%] Getting: dists/intrepid-security/Release... ok
[0%] Getting: dists/intrepid-updates/Release... ok
[0%] Getting: dists/intrepid/Release... ok
Get Packages and Sources files and other miscellany.
[ 83%] Getting: dists/intrepid-backports/partner/binary-i386/Packages.bz2... dists/intrepid-backports/partner/binary-i386/Packages.bz2 failed 404 Not Found
[ 83%] Getting: dists/intrepid-proposed/partner/binary-i386/Packages.bz2... dists/intrepid-proposed/partner/binary-i386/Packages.bz2 failed 404 Not Found
[ 83%] Getting: dists/intrepid-security/partner/binary-i386/Packages.bz2... dists/intrepid-security/partner/binary-i386/Packages.bz2 failed 404 Not Found
[ 83%] Getting: dists/intrepid-updates/partner/binary-i386/Packages.bz2... dists/intrepid-updates/partner/binary-i386/Packages.bz2 failed 404 Not Found
[ 83%] Getting: dists/intrepid/partner/binary-i386/Packages.bz2... dists/intrepid/partner/binary-i386/Packages.bz2 failed 404 Not Found
Missing: .temp/dists/intrepid/partner/binary-i386/Packages
bzip2 -d <.temp/dists/intrepid/partner/binary-i386/Packages.bz2 >.temp/dists/intrepid/partner/binary-i386/Packages
sh: cannot open .temp/dists/intrepid/partner/binary-i386/Packages.bz2: No such file
Failed: bzip2 -d <.temp/dists/intrepid/partner/binary-i386/Packages.bz2 >.temp/dists/intrepid/partner/binary-i386/Packages
WARNING: releasing 1 pending lock...

And this is debmirror in Intrepid

Mirroring to /home/macrochelodina/Repositories/Jaunty/Canonical from http://archive.canonical.com///
Arches: i386
Dists: jaunty,jaunty-backports,jaunty-proposed,jaunty-security,jaunty-updates
Sections: partner
Passive mode on.
Pdiff mode: use.
Attempting to get lock, this might take 2 minutes before it fails.
Get Release files.
[0%] Getting: dists/jaunty/Release... ok
[0%] Getting: dists/jaunty/Release.gpg... ok
[0%] Getting: dists/jaunty-backports/Release... ok
[0%] Getting: dists/jaunty-backports/Release.gpg... ok
[0%] Getting: dists/jaunty-proposed/Release... ok
[0%] Getting: dists/jaunty-proposed/Release.gpg... ok
[0%] Getting: dists/jaunty-security/Release... ok
[0%] Getting: dists/jaunty-security/Release.gpg... ok
[0%] Getting: dists/jaunty-updates/Release... ok
[0%] Getting: dists/jaunty-updates/Release.gpg... ok
Get Packages and Sources files and other miscellany.
dists/jaunty/partner/binary-i386/Packages.gz needs fetch
[ 94%] Getting: dists/jaunty/partner/binary-i386/Packages.gz... ok
Parse Packages and Sources files and add to the file list everything therein.
Cleanup mirror.
deleting pool/partner/a/adobe-flashplugin/adobe-flashplugin_10.0.12.36-1intrepid2_i386.deb
Download all files that we need to get (4 MiB).
[  1%] Getting: pool/partner/a/adobe-flashplugin/adobe-flashplugin_10.0.22.87-2jaunty1_i386.deb... ok
Downloaded 4 MiB in 963s at 4.06 kiB/s
Everything OK. Moving meta files.
All done.
+\ 
| UPDATING CANONICAL - INTREPID
+/ 
Mirroring to /home/macrochelodina/Repositories/Intrepid/Canonical from http://archive.canonical.com///
Arches: i386
Dists: intrepid,intrepid-backports,intrepid-proposed,intrepid-security,intrepid-updates
Sections: partner
Passive mode on.
Pdiff mode: use.
Attempting to get lock, this might take 2 minutes before it fails.
Get Release files.
[0%] Getting: dists/intrepid/Release... ok
[0%] Getting: dists/intrepid/Release.gpg... ok
[0%] Getting: dists/intrepid-backports/Release... ok
[0%] Getting: dists/intrepid-backports/Release.gpg... ok
[0%] Getting: dists/intrepid-proposed/Release... ok
[0%] Getting: dists/intrepid-proposed/Release.gpg... ok
[0%] Getting: dists/intrepid-security/Release... ok
[0%] Getting: dists/intrepid-security/Release.gpg... ok
[0%] Getting: dists/intrepid-updates/Release... ok
[0%] Getting: dists/intrepid-updates/Release.gpg... ok
Get Packages and Sources files and other miscellany.
dists/intrepid/partner/binary-i386/Packages.gz needs fetch
[ 84%] Getting: dists/intrepid/partner/binary-i386/Packages.gz... ok
Parse Packages and Sources files and add to the file list everything therein.
Cleanup mirror.
deleting project/trace/geochelone-jaunty
Download all files that we need to get (1 MiB).
Downloaded 1 MiB in 30s at 1.4 kiB/s
Everything OK. Moving meta files.
All done.
macrochelodina@macrochelodina-repo:~/Desktop$

Is this a bug? Why Jaunty try to fetch Packages.bz2 instead of Packages.gz? I check the repo, there is indeed no bz2, only gz.

I failed to update Canonical repo in Jaunty because of this. In Intrepid, Canonical is updated good.

If this is a bug, somebody with Launchpad account may want to highlight this.

This is from Ubuntu Jaunty Beta.