Feb 6, 2015
A reader gave me flak for writing about automation way more than about testing. Okay. Guilty. So towards making amends for this fact, I give you, Brine's guide to writing a good bug.
A good bug includes four things:
How to write a good bug:
1. Write the title last
A title is the most important part of a bug. A good title should convey all (or as much of) the information about the issue... to the point where reading the rest of the bug is unnecessary. And it should do so in as few words as possible.
I used to try to nail the title before writing anything else, but I found it just takes too much time. And I would usually end up rewriting it before submitting anyway. Write it last! I've found titles are much easier to write after you've written the bug itself.
2. Write concise steps to reproduce the issue
Steps should have just enough detail. Too little and it won't be reproducible, too much and it can get confusing. If you find yourself writing a bug that has 10+ steps, you either have a very complex bug, or overly complex steps. This can also depend greatly on the bug's audience and the preferences of the team. Still, keep it concise.
3. Describe the results and the expected results
This is the easiest part. After following your steps, what happened, and what did you expect to happen?
4. Define the impact of the issue:
You might think the impact of your bug is "high" (and what QA wouldn't), but that term isn't very useful, and certainly wouldn't rate a whole section in a bug! No. The Impact section is where you try to PROVE the impact of a bug. This helps the writer because attempting to prove the impact of a bug requires proper regression. It also shows the reader what you've tried, and provides insight into the severity of the issue, and areas to verify upon being fixed.
- Is there data loss?
- Is there a workaround?
- Does it occur on multiple environments?
- Does it occur in Production
- Does it occur on multiple browsers/apps/OSs?
- Did it occur in the last build?
These questions show impact. They tell the reader just how bad, how far reaching, and/or how recoverable, the issue is.
Oh, and there's one more thing...
4.5 Attach a screenshot
If the issue is cosmetic, it had better have a screenshot attached! Screenshots and/or movies can be worth the entirety of that classic cliché, a thousand words...
Feb 3, 2015
What/how much should I automate in the UI?
Everyone who writes UI automation is faced with this question. At first, it's often thrown to the side, while the daunting task of getting ANY automation up and running is the initial goal. I can subscribe to this reasonable omission, but eventually, automation's nemesis will rear it's nasty little head. Maintenance.
If maintaining ui automation doesn't eventually become a pain in the ass, you're either doing it wrong, or smarter than I am (at which point, please leave me the link to your blog where you explain it to us lesser mortals! No... really... do).
And this assumes that you're a very clever boy/girl, and have already deduced that time is also an issue... and are running your tests in parallel, or in the cloud, or on a grid, or in another dimension (blog link please).
Anyway... there's no escaping maintenance. So after you've refactored, added page objects, and any anything else you could think of to mitigate maintenance, you're now really ready to start talking about how much you should automate. Sadly, there's not a lot of conversations out there with real meat on the subject, and/or aren't generalized into non-usefulness. And probably for the very fact that it takes a lot of doing to even get to this point. Plus, context matters.
How much to automate is a tough question
That said, here's the answer: SPOILER ALERT: The answer typically ranges somewhere between "it depends" and "everything".
Yeah... I know... lame. So let me trim that down and say this: in my experience, even in the most robust development environments, and using the best tools available today, automating "everything" in the UI is the wrong answer.
This leaves us with the answer "it depends"... and while this is totally true, I want to try and flesh this out a bit. But I'm out of time for now...
Look for Part 2, coming soon!
Nov 16, 2014
I've been playing around with Protractor, a great, new(ish) testing framework from our friends at Google. I've shared my example code on GitHub for those that might be interested in such things.
Examples in the code include:
- Using page objects
- Running on TravisCI
- Running tests on Sauce Labs and Browserstack
- Running multiple browsers at once
I've also made a quick screencast that walks you through downloading and running the example code.
Cut to the chase, in the video we:
- Install Node (okay, I don't show this but you need do to it!)
- Download the example code from GitHub
npm install to install the project dependancies
- Briefly discuss a config file
protractor conf.js to run the tests
Sep 24, 2014
A QA Engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 999999999 beers. Orders a lizard. Orders -1 beers. Orders a sfdeljkn.
Aug 22, 2014
Because UI tests are inherently slow, running them in parallel--that is, running multiple tests, in multiple browsers concurrently--is all but a necessity.
Luckily, it's hilariously easy to run Geb tests in parallel, using Maven... all you need do is add the following code to you Maven pom.xml:
And then run your tests via Maven. Eg.
mvn test -Dgeb.env=chrome . You can try it for yourself by running my geb-example on GitHub.
This code will spread your tests between up to 4 threads/forks and run tests at the method (test) level. You can also swap out "methods" for "classes" if you're prefer to run at the spec (class) level. Either way, you can dramatically speed up your automated tests with a few lines of code...
You did think about running tests in parallel when writing your tests... didn't you?
Jul 9, 2014
Dave Thomas (PragDave) has an interesting post regarding the use of the word "Agile". In it, he argues against the use of "agile" as a noun, preferring using "agility"... fair enough... but like the context driven folks, who would have us use the phrase "automated checks" instead of "automated tests", I don't really care. The argument itself may be interesting, but semantic change is a bitch; good luck!
But he also talks about "How to do something in an agile fashion"... which I found most intriguing:
What to do: • Find out where you are • Take a small step towards your goal • Adjust your understanding based on what you learned • Repeat
How to do it: When faced with two of more alternatives that deliver roughly the same value, take the path that makes future change easier.
I love the simplicity of this. Additionally, he also states: "...anyone who comes up with something bigger or more complex is just trying to sell you something."
Nice work, sir... and for that, I changed my tagline to aid your cause :)
Jun 19, 2014
When I'm finally in charge, two of my first mandates will be: If you're driving in the left lane, BE passing. But only slightly less important will be: "Please read Mythical Man Month".
And you don't even have to read the book if you don't want. Simply read the Wiki page about the book and believe it. In fact, you can skip the book and the Wiki page if you'll repeat the following three times, out loud:
"Adding manpower to a late software project makes it later".
You may now become a manager...