At the Ubucon at Southern California Linux Expo on Friday, February 21st I’ll be doing a presentation on 5 ways to get involved with Ubuntu today. This post is part of a series where I’ll be outlining these ways that regular users can get involved with while only having minimal user-level experience with Ubuntu.
Interested in having a polished release but not able to contribute in a very technical way? Testing pre-releases is a great way to get started, even Mark Shuttleworth is getting in on the testing fun!
In this post, I’ll walk you through doing an ISO test, but there are also package and hardware/laptop tests that can be done, full details here: https://wiki.ubuntu.com/Testing/QATracker
Testing an ISO
Log on the the testing tracker
You will want to go to http://iso.qa.ubuntu.com/. The page can be a bit overwhelming at first, but there are two sections you’ll want to focus on, the log in button and the list of builds available for testing.
To log in you’ll need an account with https://login.ubuntu.com/, clicking on the “Log in” button will take you to a page where you can set up one or use your existing one.
Select a build to test
Most days the only build that is currently being tested is the “Daily” image – so in the screenshot above that is “Trusty Daily” and you’ll want to click on that link. The “Trusty Alpha2″ and “Trusty Alpha1″ images have already been released, so ISO testing on those is no longer necessary.
Select something to test
This screen can be a bit overwhelming too since it lists all the possible builds in the ISO tracker, which is a lot! I highly recommend using the Filters on the left hand side of the screen to select only the builds you’re interested in. In this screenshot I selected only Ubuntu and Xubuntu to make the list short.
Then you can look to see what you want to test. Do you have a new computer? You can test the 64-bit image isos, I circled where you want to click in the screenshot if you want to test the Xubuntu 64-bit ISO (I do!).
Select what test you want to do
At this next screen you will be presented with a series of tests that you can do. The easiest is “Live Session” since it doesn’t require you to install anything, it’s just testing a live session. You then also have various options for Installation-based testing.
But let’s say you have a virtual machine (Virtual Box is free and pretty easy to use for this) or a spare computer you want to do tests on, so for the purpose of this walkthrough we’ll select “Install (entire disk)” test case.
Download and prepare the ISO
Once you’re on the screen for the test case, there will be a link to downloading the ISO. There are many options for downloading, including just clicking it to download via http, downloading via rsync, zsync or torrent; you can read more about all of these options once you learn more about testing. For now, downloading it through http is fine.
While you’re waiting for it to download you can click on “Testcase” in the grey box below that to read through what you’ll be doing in the test case.
Once downloaded, either use the image directly in something like VirtualBox, or put it on a USB stick or burn a DVD.
Begin the test case
Scrolling down on the same page, you will see this:
This is where you will report the results of your test. The “Bugs to look for” are a list of bugs that others have reported that you may encounter, so you might want to look at some of those and include those bug numbers in your test if you encounter them too.
A quick rundown of the meaning of each field is as follows:
Result: Whether or not you were able to get through to the end of the test case with no fatal errors
Critical bugs: Bugs that prevented you from finishing the test case (would generally go along with “failed” above)
Bugs: Bugs that exist, but you were able to work around them and finish the test case (it can be marked as “passed” and still have bugs)
Note: Don’t stress too much over whether you believe a test is really passed or failed and whether bugs are critical or not, there is some judgement involved in here and results are reviewed by release managers who decide whether the ISO is ready for releasing. Just do your best!
Hardware profile: This is an optional field that can give the team an idea of what your hardware is. Using a virtual machine? Actual hardware? How much RAM and what type of graphics card? Put as much information as you can online somewhere and paste your link here. For example, here’s the testing profile I use for my Lenovo G575 and another for when I test in a 1.5G RAM virtual box instance. You can also choose to use the “hardinfo” command to generate information about your hardware and put it online somewhere.
Comment: You can add any additional comments you may have about doing the test case
If you run into any bugs while doing your test, you will need to submit those bugs for them to be recorded. For this you will need an account on launchpad.net, if you don’t have one, get one by clicking here. Once you submit a bug you’ll want to add that bug number to your list of Bugs (or Critical Bugs). Learn more about reporting bugs here.
Note: Reporting bugs can be hard, particularly determining what package to file them against, even I still struggle with this! My recommendation is to do your best, make sure you add your bug to the tracker so people notice it and ask for help on the ubuntu-quality mailing list if you’re really unsure.
Click “Submit Result” and you’ll be finished reporting a test case! The Ubuntu community thanks you :)
Learn more about testing
A more thorough walkthrough with more screenshots can be found here: https://wiki.ubuntu.com/Testing/ISO/Walkthrough
You can sign up for and email the ubuntu-quality mailing list to introduce yourself and ask any questions you may have, they’re a friendly bunch.
Visit the Quality Assurance Team wiki for more about other kinds of testing.