The Backports Developer Cheat Sheet

[jdong]

Not finished yet, but it's coming together

Purpose:
To provide a quick how-to for making backports, along with a common set of policies and rules governing Ubuntu backports.

Part 1: The Request

A user has requested backports on the forum. Should it be accepted?
Good Backport Candidate:
Anything in Universe
Desktop Applications
Compelling reason for backporting

Bad Candidate
Servers / daemons in Main -- they would benefit from security updates
Anything introducing new library versions -- other packages have to be recompiled against it, or breakage may occur.
Big, major things, like X11 or GNOME
Kernels/drivers, hardware stuff that may cause additional bugs for some users.

The Acceptance Process
If it's determined that the package is fit for backporting, a Developer should "call" the backports, or assign it to another developer. This way, we don't have two developers doing the same work, and valuable time is wasted! A developer from Each architecture must respond.

Part 2: Building the Package

Let's walk through GAIM.

You should have a copy of the Subversion repository checked out. If not, secretly do so now! LOL.

$ cd ~/backports-arena
$ svn co http://backports.ubuntuforums.org/backports .

You should have a development branch's sources in your sources.list:

deb-src http://archive.ubuntu.com/ubuntu hoary main universe multiverse restricted

Make a temporary directory:

$mkdir ~/tmp/gaim-backport
$cd ~/tmp/gaim-backport

If the source is in your sources.list (i.e. backporting from Hoary with the deb-src line)

$apt-get source gaim
$cd gaim-1.1.1-blah/

Else, you need to get the sources yourself:

$wget http://archive.ubuntu.com/ubuntu/pool/main/g/gaim/gaim_1.1.1.orig.tar.gz
$tar xzvf gaim_1.1.1.orig.tar.gz
$wget http://archive.ubuntu.com/ubuntu/pool/main/g/gaim/gaim_1.1.1-2ubuntu6.diff.gz
$cd gaim_1.1.1/
$zcat gaim_1.1.1-2ubuntu6.diff.gz | patch -p1

Update the version. Open up debian/changelog with your favorite editor. In the version field, add the UBP trailer, such as ~4.10ubp1. Put the target Ubuntu release, then ubp, then the build #. The first revision would be 1, the next bugfix would be 2, etc etc.

In this case, we're gonna use "~4.10ubp1". Why the ~ instead of a dash like with Warty backports? I've been told that "~" is a future less than operator for versions. Unsure whether or not this is in Hoary, but I certainly hope that by the next release, it's gonna be implemented.

Try to satisfy build dependencies. To get a list of what you're missing, run from the source folder:

$fakeroot dpkg-buildpackage -rfakeroot
TODO: Sample output

If you don't have the proper dependencies, the command will bomb out and list what you need. Use apt-get to install as many as possible.

It's perfectly possible that you might not be able to satisfy some of the versions it's looking for. In this case, install the stable version in Ubuntu.

Once you've gotten as many dependencies as you could, or all of them, run:

$fakeroot dpkg-buildpackage -rfakeroot -d

This skips the dependency check. If you skipped any dependencies, remember them -- you need that info for the backports announcement! Also, make extra sure to test the package in that case!

After this command finishes, go to your temp directory:

$cd ..
$ls *.deb

You should see one or more .deb's. Congratulations, your packages are built! Now, we'll commit them to the backports tree.

Part 3: Committing to the backports tree

I'm assuming you're in the temp directory containing the .debs. If not, GET THERE!

First, copy the debs to the appropriate section in the tree copy you checked out of Subversion:

$cp -v *.deb ~/backports-tree/dists/warty-backports-staging/main/binary-i386/

Make sure you get the proper distribution (warty-backports-STAGING), the proper section (GAIM is in main), and the proper architecture (I'm building for i386).

Now, you need to tell Subversion that there's new packages, or it'll ignore them.

$cd ~/backports-tree/dists/warty-backports-staging/main/binary-i386/
$svn status
?      gaim-data_1.1.1-2ubuntu6-4.10ubp1_all.deb
?      gaim-dev_1.1.1-2ubuntu6-4.10ubp1_i386.deb
?      gaim_1.1.1-2ubuntu6-4.10ubp1_i386.deb

Subversion tells you what packages need to be added. Items with a ? mark means that they aren't in the tree, but are in the folder. We need to put these in the repository.

$svn add (name of all files to be added)
A   File1
A   File2
...

Now, subversion's marked that on the next commit, these packages need to be uploaded.

Next step, we need to update the Packages file. This is an index of all the packages -- apt-get update downloads these files. I have written an automatic script to handle this:

$cd ~/backports-tree/
$./update-lists.sh

WARNING: Don't be lazy. You must be in the same directory as update-lists.sh for the script to work!!!
This command may take a while, as it indexes all the packages in the tree. On my system, it takes a whopping 10 seconds to finish indexing the final Warty backports tree (which has 270 packages)

Now, you're ready to commit:

$cd ~/backports/tree
$svn commit

Subversion will pop up your CLI text editor, where you're prompted to fill in a changelog. First, make sure the changes it lists are correct -- if not, exit and correct them! ALWAYS Write a changelog entry -- others need to know what modifications you made to the tree. I may DELETE commits that don't have a changelog entry for security reasons. ONE package per commit, please. That way, it's simple to roll back the tree if somehow it gets corrupted!! If multiple debs came from a single source package, that's fine. For this commit, I added the text "GAIM 1.1.1 testing backport".

Save & quit the editor. Subversion will upload your changes. Now, it's time to announce your testing backport to the public. Don't close your commandline yet -- you'll need that last revision info!

FYI, the Subversion versioning system is fully atomic. That means either the whole commit finishes or nothing shows up -- this way, the downloads aren't interrupted while you're uploading -- perfect for what we need!

Part 4: Beta tester party

Now, the package is in the staging section of the APT repository. It's time to test. First off, let's write an announcement in this forum. Always follow the following format:

Title: Backport: Gaim 1.1.1-2ubuntu6

Synopsis
As a tutorial for the Hoary backports system, I created a GAIM backport.

Backporting Process
The source archive directly came from Hoary. I was not able to meet the following dependencies, although the package compiled cleanly:

libgnutls11-dev (>= 1.0.16-5) libebook1.2-dev libedata-book1.2-dev libxss-dev libcamel1.2-dev

Packaging Status:
i386 -- Staging at revision 2, Stable at revision 4.
amd64 -- Not packaged
ppc -- Not packaged



Beta tester notes:
Make sure everything works.

There. Now, you have some beta testers on your side.

Part 5: Promotion

Promotion refers to the process of packages being marked stable. Remember that packages promoted should be of no less than the superceded package's quality! General rules:

NO package may directly enter the stable tree, without being in the staging tree for some time.

Packages should remain in staging for at least 7 days before being promoted. Exceptions include security updates, which may move out of staging as soon as it's verified to work.

Maintainers should be the ONLY people making promotions. Developers may promote packages only if it's a pressing security fix, or if a maintainer is on vacation...

The Process
Promoting is simply moving a package from the -staging area to the corresponding non-staging area. Make sure you use svn mv, not just mv. The standard UNIX mv will not instruct the repository to remove the old instance and add the new one.

In keeping to the GAIM example:

$cd backports-tree/
$svn mv dists/warty-backports-staging/main/binary-i386/gaim_1.1.1-2ubuntu6-4.10ubp1_i386.deb dists/warty-backports/main/binary-i386/
$svn mv dists/warty-backports-staging/main/binary-i386/gaim-data_1.1.1-2ubuntu6-4.10ubp1_all.deb dists/warty-backports/main/binary-i386/
$svn mv dists/warty-backports-staging/main/binary-i386/gaim-dev_1.1.1-2ubuntu6-4.10ubp1_i386.deb dists/warty-backports/main/binary-i386/
$./update_list.sh
$svn commit

IMPORTANT: Make sure you run ./update_list.sh -- otherwise, the backport won't show up.

A commit will follow at the end -- fill in the changelog! Promote one set of packages at a time. Set is defined the same way as above, in the backporting packages section -- With GAIM, the gaim source package creates the "gaim", "gaim-data", and "gaim-dev" binary packages. Notice how you don't need to re-upload all the data -- Subversion can do REAL moves, as opposed to my old rsync system, which can't!

Then, go back and edit the backports announcement, updating the "Packaging Status" field appropriately.
===================================
Congratulations, you've completed the tutorial! You should be all trained to handle backports.

Appendix: Tips and such

• Run the svn update command on your checked-out copy of the repository regularly. It make sure you have others' changes to the repository, and the latest changelog.
• Want to see the changelog? Run svn log | less
• Install "fakeroot" to get the Fakeroot command, "subversion" to get the svn command.
• Conflicts may occur with Packages.gz. I haven't made sure of that yet, but I anticipate it. Please report such issues to me. In the meantime, refer to a Subversion tutorial/handbook for information on resolving conflicts. In our case, if you run "svn update", then ./update_list.sh, you may safely replace the current packages.gz with your copy. Take a look specifically at the "svn resolved" command.
• Messed up real badly? Run svn revert to undo all changes since last commit. You may even roll back farther than that, but do an svn update before continuing, and examine the changelog before blatantly reverting others' changes!
• If Subversion gives you timeout errors when committing huge packages, you need to set the global http-timeout value to 0 in ~/.subversion/servers.