Talking testing, agility and automation... and anything else.



06
Feb 15

How To Write A Good Bug

A reader gave mefusca9 flak for writing about automation way more than about testing. Okay. Guilty. So towards making amends for this fact, I give you, Brine’s guide to writing a good bug.

A good bug includes four things:

  • Title
  • Steps
  • Results
  • Impact

How to write a good bug:

1. Write the title last

A title is the most important part of a bug. A good title should convey all (or as much of) the information about the issue… to the point where reading the rest of the bug is unnecessary. And it should do so in as few words as possible.

I used to try to nail the title before writing anything else, but I found it just takes too much time. And I would usually end up rewriting it before submitting anyway. Write it last! I’ve found titles are much easier to write after you’ve written the bug itself.

2. Write concise steps to reproduce the issue

Steps should have just enough detail. Too little and it won’t be reproducible, too much and it can get confusing. If you find yourself writing a bug that has 10+ steps, you either have a very complex bug, or overly complex steps. This can also depend greatly on the bug’s audience and the preferences of the team. Still, keep it concise.

3. Describe the results and the expected results

This is the easiest part. After following your steps, what happened, and what did you expect to happen?

4. Define the impact of the issue:

You might think the impact of your bug is “high” (and what QA wouldn’t), but that term isn’t very useful, and certainly wouldn’t rate a whole section in a bug! No. The Impact section is where you try to PROVE the impact of a bug. This helps the writer because attempting to prove the impact of a bug requires proper regression. It also shows the reader what you’ve tried, and provides insight into the severity of the issue, and areas to verify upon being fixed.

For example:

  • Is there data loss?
  • Is there a workaround?
  • Does it occur on multiple environments?
  • Does it occur in Production
  • Does it occur on multiple browsers/apps/OSs?
  • Did it occur in the last build?

These questions show impact. They tell the reader just how bad, how far reaching, and/or how recoverable, the issue is.

Oh, and there’s one more thing…

4.5 Attach a screenshot

If the issue is cosmetic, it had better have a screenshot attached! Screenshots and/or movies can be worth the entirety of that classic cliché, a thousand words

fin


06
Jan 13

Verify Sorting With Sahi (or any tool really)

4800819674_3cf963deaa_bI recently found a bug when sorting table columns, while running IE9. Ignoring the obvious fix, I wrote up the bug and then as I’m a fan of doing, I wrote a failing automated test to test it (Defect Driven Development!).

Testing sort proved a bit tricky… I thought I would share the results to perhaps save others same pain. I would also not be surprised to find a more elegant solution out there. If you have one, do feel free to share!

My example is in Sahi but it should be easy to transfer to your tool of choice. The gist is:

  1. Sort your column
  2. Collect all the elements in the column in an array
  3. Copy the array and sort the copy using javascript’s sort()
  4. Compare the two arrays

And here’s the Sahi code…

First off, thanks to JavascriptKit.com for providing an example for my example!

The script starts by navigating to JavascriptKit.com and clicking on the Name table header to get our initial sort of that column.

Then we collect each element in the Name column, in table0 and store them in an array, $appSortedValues. To iterate through our loop, we get the number of rows in table0 by counting the number of rows and subtract 1 for each table header.

Now we need a copy of our $appSortedValues array but in Javascript, you can’t just set a new array from our existing array like so: $jsSortedValues = $appSortedValues; . This will actually create a reference to our original array; not what we want. Instead we use the slice() method to select elements 0 through the end of the array and put them all in our new array, thus copying it.

Finally, we use javascript’s sort() method to sort the copy of our array, $jsSortedValues. But the sort() method has a little wrinkle; by default, it sorts alphabetically and is case sensitive. This is unlikely to be how your application’s sort works… but luckily, you can roll your own sort filter by passing a function as an argument to the sort() method. In our case, we want it to be case-insensitive, hence our function caseInsensitiveSort.

Now we have two arrays and can simply use Sahi’s _assertEqual method to verify both arrays are sorted in the same order.

That’s it!


06
Dec 12

Should You Just Stop Tracking Bugs?

With apologies to Ian Betteridge for the hyperbolic headline, I wanted to share this 6 minute lightning talk in which Jon Tørresdal argues for “The simplest solution to bug tracking: don’t!”.

To paraphrase Jon’s list of 5 “crazy ideas”:

  1. Don’t track bugs; just fix them
  2. Delete all bugs in your backlog that you can’t fix immediately
  3. All newly reported bugs are either rejected or fixed immediately
  4. Automated tests are created for each new bug
  5. Set WIP limit for bugs (eg. 20 total)

Ten years ago I would have scoffed at the idea of not tracking bugs and deleting the bug backlog, but today, I can see this as a realistic possibility. In fact, this isn’t far from how my current team operates today. So what’s stopping me from going all in? Guts?


14
Dec 11

Zero Known Bugs!

My current project, internally known as Bug Remediation, had the lofty goal of fixing every single bug for one of my company’s mature products. Today we accomplished just that. One of my apps now has zero known issues. No really… zero!

Now how long that will last is another story…