Page 2 of 3 FirstFirst 123 LastLast
Results 11 to 20 of 21

Thread: Confused about cadence testing cycles

  1. #11
    Join Date
    Apr 2008
    Beans
    10,345

    Re: Confused about cadence testing cycles

    Quote Originally Posted by Elfy View Post
    Code:
    * elfy still doesn't understand all that cadence stuff 
    <balloons> :-( I'm sorry elfy
    <elfy> good job I'll be testing xubuntu 
    <elfy> :)
    <balloons> Well, what are you still missing?
    <elfy> oh d'oh just realised I'll be doing arm as well ... lol
    * cwayne (~cwayne@pool-98-118-37-160.bstnma.fios.verizon.net) has joined #ubuntu-quality
    <elfy> balloons: I suspect what I am missing is the more normal term people would use :)
    <balloons> elfy, sure sure
    <balloons> basically we're going to have test-a-thons if you will every couple weeks
    <balloons> each time we'll pick somethng else to focus on
    <balloons> we'll pull the tests from the tracker and make a milestone for them and have people report on them
    <elfy> ok - that makes sense :)
    <balloons> so for next week I've got to pull the tests and make the milestone and announce i
    <elfy> we'll not be doing that
    <balloons> it's that simple
    <balloons> everyone will have saturday - saturday to test and report
    <elfy> well kind of we will - when we have something we need specifically testing then we'll be shouting 
    <elfy> right - and then the 'other' week is not cadence testing, just normal dailies I assume 
    * elfy likes the idea of that with arm 
    <elfy> ty balloons :)
    <balloons> the other week is an off week
    <balloons> I'll be doing things like maintaining and updating tests, playing with bugs, etc, etc
    <elfy> but dailies will be updated daily still?
    <balloons> writing new tests
    <balloons> yes dailies will update, and no one is stopping your from running them if you wish :-)
    <elfy> okey doke
    <balloons> our focus is just much wider than the iso's now
    <elfy> yea - as it perhaps should be 
    <balloons> <3
    * alourie_lunch has quit (Remote host closed the connection)
    <elfy> so it in effect gives people longer than a day to test - have the whole of a week to do so in 
    <elfy> is that correct?
    <balloons> yes, exactly
    <balloons> flexibility
    <balloons> we're spread out and it's so much easier for you to plan to help out at some point in 7 days, rather than 7 hours
    <elfy> k - I will post that on the forum, bit of confusion I think 
    * balloons hasn't been to the forum
    <balloons> I've been a bit busy
    <balloons> but I wanted to hop over there today
    <elfy> balloons: has missed this then Confused about cadence testing cycles 
    <balloons> gotcha..
    <balloons> well let's try and clear everything up then
    <balloons> I think it should be really understandable after our first (I hope)
    <elfy> I'll just post this over there :)
    I was a bit confused too - hope this helps a bit, bear in mind when I'm talking of we I mean Xubuntu

    As far as I can see - it means you have a week to test in rather than a day.
    I'll be testing Lubuntu here, not because I hate Unity, but I maintain over 20 PC's with P4M800 graphics that simply won't run either Compiz or Mutter in an acceptable manner

    But I've been iso/upgrade testing since Hardy or Intrepid and I really don't get this change

    When I started we'd typically have 3 or 4 days (generally about Monday thru Thursday) to test before each point-release and you could depend on the devs to fix any critical bugs, or at least set a target to have the bug fixed.

    And the alphas were typically about 4 weeks apart, while the betas were closer together, and we'd generally even have an RC about a week before final. IMHO things have continued to go downhill ever since they dropped the RC.

    Anyway it now sounds to me like no images will be frozen and they want testers to test each test case every day, seven days straight, every other week

    Before I retired I worked in manufacturing engineering and I worked closely with the QA team to achieve ISO 9000 certification and I very strongly feel that this newest plan is an excellent example of letting the inmates run the asylum.

    Or maybe Canonical wants to move all testing to "internal automated VM" which will mean even less testing on actual hardware ........... I don't know

    I hope I'm wrong, maybe this will work out great

  2. #12
    Join Date
    Apr 2008
    Beans
    10,345

    Re: Confused about cadence testing cycles

    OK, this notification clears up a lot of my confusion:

    Ok, so we're a bit delayed in starting this because, well, there was a Holiday in North America called Thanksgiving. Forgive me. :-p

    So, this week (through Saturday) is our first cadence week. What that means is we will be focusing testing specific packages (or isos) during the week. What you can do to help is to run through a testcase (or a few ) at some point during the next week and report your results. The goal here is NOT to make all the tests green everyday. Instead, we want to test on a deeper level and really look for bugs in the software (or even the testcases ) so we can be confident in the health of the package(s) we are testing. To that end, there is no daily target -- rather we want to test through Saturday (whenever that is in your localtime ). Think quality, not quantity (I know, it's a bad pun.. quality quality...)

    What this means practically is during the next week, try to execute some testcases for our week 1 targets (which are libreoffice, unity autopilot tests and the daily raring iso). You can find the wiki page detailing this here:

    https://wiki.ubuntu.com/QATeam/Cadence/Raring/Week1

    The master schedule can be found here:

    https://wiki.ubuntu.com/QATeam/Cadence/Raring

    Again, I would suggest finding a day and time that works for you and adding your results. Feel free to use IRC (#ubuntu-quality on freenode) or this mailing list to discuss any bugs you may find. At the end of the week, I'll recap what we did (bugs, tests ran, etc), and we can evaluate the results to see if further followup and focus is needed or not. Now, our testing focus's each week will change in order to target packages that need more QA work, or new packages that recently landed in QA.

    Now, this week I will also be posting the next bits in our autopilot series -- intending to teach you about running and writing autopilot testcases. Our cadence testing this cycle will include running both automated and manual tests -- we need both You'll note the autopilot unity testsuite is included this week. I hope our subsequent testing weeks can feature some new automated autopilot tests written by some of you If writing automated tests aren't up your alley, don't worry we need manual testcases too!

    Happy Testing everyone,

    Nicholas

  3. #13
    Join Date
    Sep 2010
    Location
    Beta Testing in Canada
    Beans
    5,423
    Distro
    Ubuntu Development Release

    Re: Confused about cadence testing cycles

    I got time to check into that this week. Not sure why Nick did not post that here.
    This is Rolling Release
    Warnings for New Beta Testers& Helpful Terminal Commands:
    Running Trusty /devel/@ 5.120GHz32bit/ Please put [ prefix] on New Threads!

  4. #14
    Join Date
    Jun 2010
    Location
    London, England
    Beans
    6,389
    Distro
    Ubuntu Development Release

    Re: Confused about cadence testing cycles

    I am trying to get in on this.

    Just downloaded the 20121126 ISO image and during the second purple screen I do not get all the usual disk activity as the hardware is being detected. I then get a black screen with the error message "unable to find medium containing a live file system."

    As regards those Autopilot tests beware. There are a lot of them. Note what is said by Nick on 20th November on this page:

    http://qa.ubuntu.com/

    As of this writing, I get 461 tests returned. Feel free to try and run them. Pick one from the list and see what happens. For example,

    autopilot run unity.tests.test_dash.DashRevealTests.test_alt_f4_ close_dash

    Just make sure you run them in a guest session -- I don't want anyone's default desktop to get hammered by the tests!

    If you are feeling adventurous, you can actually run all the unity testcases like this (this will take a LONG TIME!).
    These tests are also supposed to be run on 12.10 or 13.04 and not 12.04.

    Regards.
    It is a machine. It is more stupid than we are. It will not stop us from doing stupid things.
    Ubuntu user #33,200. Linux user #530,530


  5. #15
    Join Date
    Sep 2010
    Location
    Beta Testing in Canada
    Beans
    5,423
    Distro
    Ubuntu Development Release

    Re: Confused about cadence testing cycles

    Quote Originally Posted by grahammechanical View Post
    I am trying to get in on this.

    Just downloaded the 20121126 ISO image and during the second purple screen I do not get all the usual disk activity as the hardware is being detected. I then get a black screen with the error message "unable to find medium containing a live file system."

    As regards those Autopilot tests beware. There are a lot of them. Note what is said by Nick on 20th November on this page:

    http://qa.ubuntu.com/



    These tests are also supposed to be run on 12.10 or 13.04 and not 12.04.

    Regards.
    Ahhhhhhhhrrggg.... I just logged on and ran the Unity AutoPilot .. and it just keeps on going .. also on one of my main *installs*! I was just going to start a new thread about 'no heads up" but .. well .. here it is !

    So far , so good. Nice work on the Unity auto Pilot test. Behaves like a remote access.

    I don't understand the code comming from terminal though .. ssssss...FF..ssss EE FE .. etc... so I will just let the test run it's course. It it foobars my desktop, well.. then thats just the way it is.

    Umm.. thanks for the *heads up* Graham.
    Last edited by ventrical; November 26th, 2012 at 05:07 PM.
    This is Rolling Release
    Warnings for New Beta Testers& Helpful Terminal Commands:
    Running Trusty /devel/@ 5.120GHz32bit/ Please put [ prefix] on New Threads!

  6. #16
    Join Date
    Apr 2008
    Beans
    10,345

    Re: Confused about cadence testing cycles

    My "normal" test box won't run Unity since they dropped Metacity/Unity-2D in favor of "llvmpipe+compiz" so I guess my testing is simple

    All I need to do is test the installation of the available Lubuntu images and report pass or fail. Of course I'll have to file a bug report to mark a test failed but we'll see how this goes.

    I should have some time to test an Lubuntu i386 iso this evening.

  7. #17
    Join Date
    Apr 2006
    Location
    USA
    Beans
    402

    Re: Confused about cadence testing cycles

    Glad you guys are enjoying things! I hope all is clear now -- it's not gloom and doom at all! I hope the autopilot tests are the first of many..

    If you run all of the unity autopilot tests, yes it will take a long time. I think I'm going to split up the tests and have people run them a bit differently with some heads-up. This is a learning process on this end too -- need to figure out the best way to run and report (these are a bit different than the manual tests), but we'll figure it out. Enjoy the testing guys!

  8. #18
    Join Date
    Sep 2010
    Location
    Beta Testing in Canada
    Beans
    5,423
    Distro
    Ubuntu Development Release

    Re: Confused about cadence testing cycles

    Quote Originally Posted by guitara View Post
    This is a learning process on this end too -- need to figure out the best way to run and report (these are a bit different than the manual tests), but we'll figure it out. Enjoy the testing guys!

    Whew..!. Good to know that we are not alone !
    This is Rolling Release
    Warnings for New Beta Testers& Helpful Terminal Commands:
    Running Trusty /devel/@ 5.120GHz32bit/ Please put [ prefix] on New Threads!

  9. #19
    Join Date
    May 2007
    Location
    The New Forest
    Beans
    Hidden!
    Distro
    Xubuntu Development Release

    Re: Confused about cadence testing cycles

    Be good to look at how other flavours can adapt these tests to suit their needs.

    No unity here
    Forum Social IRC Channel
    Xubuntu IRC Support
    Xubuntu Support

    Please do not PM me about Registration issues without having been asked to. I will tell you to post here

  10. #20
    Join Date
    Sep 2010
    Location
    Beta Testing in Canada
    Beans
    5,423
    Distro
    Ubuntu Development Release

    Re: Confused about cadence testing cycles

    Quote Originally Posted by Elfy View Post
    Be good to look at how other flavours can adapt these tests to suit their needs.

    No unity here
    There is lots of testing needed here also : http://iso.qa.ubuntu.com/qatracker/m...nes/243/builds

    ..all flavours.. looks like a gazzilion things to keep us busy

    I guess it would have to be up to the devs of any particular flavour whether or not they are going to write an auto-pilot test for those DEs.
    This is Rolling Release
    Warnings for New Beta Testers& Helpful Terminal Commands:
    Running Trusty /devel/@ 5.120GHz32bit/ Please put [ prefix] on New Threads!

Page 2 of 3 FirstFirst 123 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •