Talking testing, automation... and anything else.

Nov 12

Test Automation Roadmap: The 5 Ts

Where are you headed with your test automation efforts? Like all journeys into the unknown, a map can prove to be especially handy.

I think of test automation as being broken down into 5 goal groupings, that themselves have 1 ├╝ber goal. Thus, I present my test automation roadmap or what I call The 5 Ts


Don’t find it… make it!
Developing test automation takes time (I just rolled my eyes at myself as I typed that). Of course this is painfully obvious to you and absolutely everyone else… but still, time is always an issue when developing test automation. you will always struggle to find time to write new tests, design better tests, maintain existing tests and refactor tests when things change.

But you can’t find time for automation… you have to make time for it. Factor it right into your testing estimates and/or block off specific time for automation. The amount of time, or a lack thereof, allocated to automation is a great indicator to how committed your team/stakeholders are to automation and will be directly related to it’s success.


The right tool for the right job…
There is a veritable plethora of test automation tools available today and there are even more opinions on which is best. Picking the best tools depends heavily on the context of your project and the skills of your team. Choose wisely and try before you buy! Give your tool candidates a spin for an iteration (week or two) and see how they perform in the field.


Not just tests… great tests!
Start writing tests! Write them iteratively, refactor regularly and fix failing tests quickly (keep ’em green!). Value working tests like you would working software. Start small (eg. smoke tests) and expand to improve coverage. Consider continuously adding to your framework rather than striving for the perfect framework up front. Aim for beautiful tests that are concise, easy to read and will be easy to maintain.


Show your cards…
Get your automated tests in front of your team/stakeholders/management and solicit their input. Testing should be a group activity… get the group involved! Share your testplans with your team; set up regular test-code reviews; pair-program. Celebrate your milestones and accomplishments! Schedule tests to run often and post results for all to see. Fast, consistant feedback will improve your tests, help manage expectations and show a return on your automation investment.


Trust me…
The ultimate goal of test automation, and the destination on this roadmap, is trust. Without trust, test automation has no value. As the oracle for your SUT (system under test), you, your stakeholders, and your team, must be able to trust the answers it gives. Such trust takes considerable time and effort to build and is derived by successes with the previous 4 Ts. With enough time, the right tools, great tests and a transparent effort, trust in your automation will grow. Trust me!

May 12

Don’t Drive Angry…

My last couple of iterations have been challenging, to say the least. It’s early on in a new project with some big changes and we took on some stories that were, quite frankly, too large. And while they probably could/should have been broken down into smaller chunks, the team decided to take them on. Not surprisingly (in glorious hindsight), we all had a turn dealing with their frustrating vastness, and passed on the displeasure with each handoff. Luckily (sarcasm), with QA partaking in each facet of the process, and ultimately being the caboose, I got to enjoy the punishment of each facet–ba, dev and qa–end to end, as we each took a turn driving the angry train. Good times…

The last couple of retrospectives have been–shall we also say–less than enjoyable. There’s a reason finger pointing is NOT an olympic sport.

But the worst part was that I also got upset with myself. During said turmoil, and due to the number of changes (or so I thought), a number of my scripts started to fail… but because of the pressure to get these titanic stories done, I let them continue failing while I got caught up. Of course ONE of these scripts was failing because of a bug… but I wouldn’t find this out until an iteration later. Sadness…

The moral of the story isn’t really don’t drive angry, though this too isn’t bad advice… it’s to NOT let your automation fail… no matter what! We write automation to find bugs faster and to police change. Letting tests fail defeats both of these tasks and diminishes faith in the tests themselves. This is just foolishness… the sky may be falling but that’s not a reason to leave the skylight open! Bad QA; no biscuit.

Mar 12

It Takes a Troop of Monkeys…

I doubt they think it takes a whole troop!In the recent uTest blog post, Gerald Weinberg hit an old nerve of mine…

“To me, the biggest weakness [in the way companies test software] is not considering software testing anything but a (barely) necessary evil. Testing is seen as something that could be done by a troop of monkeys, so serious testers are treated like third-class individuals. The lack of means of acquiring testing skills arises from this attitude, as do most of the other poor practices in the testing business. You treat people as if they are stupid, then they will wind up acting stupid.”

This truth is something that I’ve dealt with my entire career… and has been on my mind lately.

Mar 12

Can Not Reproduce

This is my favorite xkcd comic and one (arguably) appropriate for this site…

Feb 12

Testing James Bach

I’m watching a talk by James Bach (who can impressively talk over heavy nose-breathers) in which he claims he has a google alert for blogs with the terms exploratory testing. I am now testing this claim.