Improving Testing Time By 288,000%

Sep 18, 2012

Before I explain the hyperbolic title of this post, a short interlude...

Summer is over. Long live Fall! Along with all the things that keep normal folks busy in the summer, I also play music and on that front, I've been more busy than usual. There were a lot of gigs and a debut album release, not to mention the practice time required for the aforementioned endeavors. All of this and a long-ish project at work, conspired against writing about QA. But because of said long-ish project, I have lots of fodder for the blog.

Anyway, I thought I would ease my way back to the blog with this post...

At work we have a basically-sunset product. And like many basically-sunset products, there is no team associated with it. And when you have a basically-sunset product with no associated team... over time, you end up with teams without a deep knowledge of the product. But also like most basically-sunset products, it does require a modicum of testing when things around it change.

This is where our hero comes in. Not me, no... my boss, actually! Because instead of making me test this basically-sunset product, he volunteered (admittedly after a bit of complaining by me) to test it himself. This actually worked out pretty well because he's one of the last people with the company that knows the product and it's history well. The downside is time. Because he's extremely busy with his own work, it takes him 4+ days to find time to actually do the testing. Still, this is how it went for over a year...

Long story only slightly longer, this past iteration planning meeting, it was time to once again strap on the test-bag (not sure that metaphor works but let's go with it anyway) and test the basically-sunset product. But because I always feel bad seeing my boss struggle to find time to test something that he was nice enough to not make me test, and since I had a few open days in the iteration, we decided to have me spend the time automating his tests. So working from my boss' testplan, along with input from the rest of the team, I was able to create a smoke test comprised of 20-odd (and counting) automated scripts that performed the same amount of testing in under two minutes (special thanks to Sahi's ability to thread tests).

Thus, what used to take 4+ days to test now takes under 2 minutes... which by my count (Google my math) comes roughly to a 288,000% improvement :)

Why did we wait a year to do this again?