Page 2 of 2 FirstFirst 12
Results 11 to 14 of 14

Thread: So called "cadence testing" is a failure

  1. #11
    Join Date
    Apr 2008
    Beans
    10,841

    Re: So called "cadence testing" is an utter failure!

    Quote Originally Posted by Noskcaj10 View Post
    While i don't like the idea of candence testing either, you've went too far.
    first, the bug list you had shows all bugs found in raring reported on the iso-tracker. for all *buntu's
    second, the freeze bug has been found and confirmed (making it no longer a problem caused by bad testing), but is very difficult to fix, if it's "our fault", please, show us how we could have fixed it.
    third, the network manager bug is private because it shows private info of some form.
    third, i'm shocked that you overlooked the areas even worse, like testdrive, the PPC recursive bug, and the lack of testing for netboot and ppc.
    I recently used testdrive w/virtualbox and noticed no problems different than those I'd encountered on bare metal. I have no hardware to test PPC. I did just run a test on the latest netboot mini.iso that failed:

    https://bugs.launchpad.net/ubuntu/+s...r/+bug/1161898

    I'm doing my job as a tester. I report bugs and I follow up on my bugs. And I don't create a new forum user ID to flame someone that tries to point out an obvious problem with the current testing scenario.

  2. #12
    Join Date
    Sep 2010
    Location
    Beta Testing in Canada
    Beans
    6,038
    Distro
    Ubuntu Development Release

    Re: So called "cadence testing" is an utter failure!

    Quote Originally Posted by kansasnoob View Post
    I recently used testdrive w/virtualbox and noticed no problems different than those I'd encountered on bare metal. I have no hardware to test PPC. I did just run a test on the latest netboot mini.iso that failed:

    https://bugs.launchpad.net/ubuntu/+s...r/+bug/1161898

    I'm doing my job as a tester. I report bugs and I follow up on my bugs. And I don't create a new forum user ID to flame someone that tries to point out an obvious problem with the current testing scenario.

    Most of us know you are one of the best beta-testers @ ubuntu-forums. When we point things out about programs that are buggy or a process that just does not work, we are going to raise hair up on devlopers backs. Thats part of being a good beta-tester.

    regards,
    ventrical
    This is Rolling Release
    Warnings for New Beta Testers& Helpful Terminal Commands:
    Running Trusty /devel/@ 5.120GHz32bit/ Please put [ prefix] on New Threads!

  3. #13
    Join Date
    Apr 2008
    Beans
    10,841

    Re: So called "cadence testing" is a failure

    Quote Originally Posted by guitara View Post
    It's a busy busy week as you know, but I'll address a couple of these points. We should discuss this more later when we all have a bit more time (post-cycle?).

    On the dates being incorrect on the calendar, there was an original note of april 4th for the actual release date for the beta, but it wasn't showing up. I think it was a separate line item even. Anyways, I made sure the note shows up now. That said, the cadence calendar is showing the saturday start dates, not actual event dates, whereas the release schedule is showing the exact important dates. The Final Beta release happens the week of March 30th from our cadence perspective, hence it's listed on that weeks' line -- I hope this makes sense. (EDIT: I should note here that the calendar needs to be easy to understand, so please do give more feedback on tweaks that should happen to make it so. We had an integrated calendar (it was part of the release schedule calendar) for testing last cycle, and it was very confusing to follow because so many events and timelines where trying to be convened on one page; I hope this is an improvement, but let's keep iterating!)

    In general we as testers actually like bugs.. Bug counts are funny. A low bug count during the cycle doesn't mean a stable release (it could mean we're not testing!), and a high bug count doesn't necessarily mean the release is horrible (maybe we're testing really well). Now, the caveat with bugs is while we like them, we don't like them in the stable release -- in development though, if we find a large amount of bugs, I would consider our job as successful.

    I feel you on the private and long-standing bugs, but we as a testing community have done our part on those by finding and documenting the bug. For the private bug, given the severity and longevity it would likely be worth asking someone to manually prune sensitive info from the report and open it, or ask the original reporter to merely publicize his info willingly if he is able to do so.

    All that said, what can we do about high bug counts and longstanding bugs or regressions? If we've reported them and documented them well, we've done our part in helping get the bug fixed. From the development side, we need those folks to also address and work the bugs found. I did a post last year about bug shepherding, and it's important. I know many of you practice this;

    you report the bug in detail, making sure it can be reproduced by others
    you follow-up on the bug to make sure it's properly filed and can be seen by those who can fix it
    you help test the fix if necessary

    Beyond that, the actual bug fix is in the hands of the developers. We can and should do everything we can to help make sure the bug is fixed, but the fixes themselves rely on a developer.
    You're completely missing the point. There's no better proof of that than this:

    http://ubuntuforums.org/showthread.php?t=2129906

    It's too late to recruit new testers in the middle of final Beta testing, and it's a waste of time to try and teach old testers new tricks!

    The problem is that our so-called cadence testing in no way syncs with the various freezes during the dev cycle.

  4. #14
    Join Date
    Apr 2008
    Beans
    10,841

    Re: So called "cadence testing" is a failure

    I wish I could change that Lubuntu flag to "all variants".

    At the time I posted this I was testing Lubuntu images per the "cadence testing" schedule (week 9).

    What's really important ATM is that we hammer down on serious bugs in this final Beta!

Page 2 of 2 FirstFirst 12

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •