During the Karmic cycle I'm planning to develop a "Testing Weather Report" web application (inspired by its counterpart developed for the release team), which will display various bits of information on the state of the development branch.
The information I plan it will aggregate would involve, but is not limited to:
- Held back packages
- Critical bugs in core packages (kernel, gcc, glibc, etc.) that may have high impact on a broad range of testers
- Ongoing library transitions
- Archive integrity
- Daily image build status
- Daily image health check (oversized or not)
- ISO package diff (how up-to-date are the current ISO images with the archive?)
- Upgrade situation ( possibly with automatic upgrade test results)
- Current milestone, release notes / technical overview, next milestone name, how many days are left
- Freeze status (freezes we are on and upcoming ones)
- New packages in the archive
- Bugs targeted for the next milestone
- Calls for testing of specific packages (possibly not yet pushed to the development branch) by developers
- Significant updates in components related to release goals
- Archive mirror status
- Build daemon status
It's worth emphasizing that all of this information is already available to testers, but is scattered over various resources, and can be hard to evaluate altogether at a single time point. The goals are to:
- come up with a list of most relevant metrics and other data that will help testers
- display the data in a manner in which the most relevant elements can be evaluated at a single glance, on a single page, and the slightly less relevant can still be easy to reach
- present it all on an attractive web page that will hopefully present a small bit of extra incentive for people to do testing for Ubuntu.
The feedback I'm hoping to get would be on:
- whether you'd find this useful at all and why
- what kind of information you'd want aggregated by it, in addition to what I've listed above
- what particularly useful ways of displaying the information would be (don't worry, I do have some ideas).
I'll be holding a session tomorrow on the QA track at UDS-Karmic to lay down the initial plan and get preliminary feedback. Those interested can tune in to room #9 at 14:15 UTC tomorrow to follow up, and participate live via IRC (#ubuntu-devel-summit on Freenode) and Gobby (gobby.ubuntu.com - document name will be "karmic-qa-testing-weather-report"). I'll also keep you posted once I have some mockups and rudimentary code.