Talking testing, automation... and anything else.

Jun 14

Mythical Man Month or: How I Learned to Stop Worrying and Love Management


When I’m finally in charge, two of my first mandates will be: If you’re driving in the left lane, BE passing. But only slightly less important will be: “Please read Mythical Man Month”.

And you don’t even have to read the book if you don’t want. Simply read the Wiki page about the book and believe it. In fact, you can skip the book and the Wiki page if you’ll repeat the following three times, out loud:

“Adding manpower to a late software project makes it later”.

You may now become a manager…

Feb 13

Amazon EC2 Ubuntu HowTos

ec2Having a server in the cloud is a nicety. That Amazon gives you one free for a year to tinker with… that’s being downright friendly!

The following are some notes/howtos for setting up an Ubuntu Amazon EC2 instance. I post them to help me remember them but they might also be useful to others…

Allow SSH Access

If you’d prefer to just ssh into your instance instead of using key/pair…

1. Login using the key/pair you created when setting up your EC2 instance:

2. edit your sshd_config file and uncomment and/or set PasswordAuthentication yes:

3. Reload SSH:

4. Create a new user and set the user’s password:

Install LAMP

1. Update Ubuntu’s package database, install and run Taskel. Select the LAMP server (don’t deselect anything that is also checked) and let it install:

2. Install PHPMyAdmin; select Apache2, enter a root password and say No to config the database later:

3. Verify by going to: http://myIPaddress/phpmyadmin and login as root with the password you entered. Of course you’ll probably want to secure/move this!

Install XRDP on Ubuntu 12.10

Installing X11 on your instance–should you be so inclined–may prove tricky… I tried MANY ways of doing it (OpenBox/FluxBox, VNC, etc…) but each attempt ended in failure. I found victory with XRDP!

1. Found from the instructions here. First install Gnome (will take a while)…

2. Ubuntu 12.10 no longer includes gnome-session-2d, so install fallback and edit .xession to use it:

3. Edit Xwrapper file and set allowed_users=anybody:

4. Create a new Security Group Rule for RDP(i.e. open port 590x (where x is the vncserver id))

5. Install RDP client on your local machine…

Install Chromium brownser on Ubuntu 12.10

FireFox is in need of Unity by default and it appears it doesn’t exist on 12.10. I wanted Chrome on there anyway but it didn’t work either. Chromium does:

Feb 13

Headed To GTAC New York

I’m looking forward to attending this year’s Google Test Automation Conference (GTAC) in New York!

Jan 13

Verify Sorting With Sahi (or any tool really)

4800819674_3cf963deaa_bI recently found a bug when sorting table columns, while running IE9. Ignoring the obvious fix, I wrote up the bug and then as I’m a fan of doing, I wrote a failing automated test to test it (Defect Driven Development!).

Testing sort proved a bit tricky… I thought I would share the results to perhaps save others same pain. I would also not be surprised to find a more elegant solution out there. If you have one, do feel free to share!

My example is in Sahi but it should be easy to transfer to your tool of choice. The gist is:

  1. Sort your column
  2. Collect all the elements in the column in an array
  3. Copy the array and sort the copy using javascript’s sort()
  4. Compare the two arrays

And here’s the Sahi code…

First off, thanks to for providing an example for my example!

The script starts by navigating to and clicking on the Name table header to get our initial sort of that column.

Then we collect each element in the Name column, in table0 and store them in an array, $appSortedValues. To iterate through our loop, we get the number of rows in table0 by counting the number of rows and subtract 1 for each table header.

Now we need a copy of our $appSortedValues array but in Javascript, you can’t just set a new array from our existing array like so: $jsSortedValues = $appSortedValues; . This will actually create a reference to our original array; not what we want. Instead we use the slice() method to select elements 0 through the end of the array and put them all in our new array, thus copying it.

Finally, we use javascript’s sort() method to sort the copy of our array, $jsSortedValues. But the sort() method has a little wrinkle; by default, it sorts alphabetically and is case sensitive. This is unlikely to be how your application’s sort works… but luckily, you can roll your own sort filter by passing a function as an argument to the sort() method. In our case, we want it to be case-insensitive, hence our function caseInsensitiveSort.

Now we have two arrays and can simply use Sahi’s _assertEqual method to verify both arrays are sorted in the same order.

That’s it!

Nov 12

Test Automation Roadmap: The 5 Ts

Where are you headed with your test automation efforts? Like all journeys into the unknown, a map can prove to be especially handy.

I think of test automation as being broken down into 5 goal groupings, that themselves have 1 ├╝ber goal. Thus, I present my test automation roadmap or what I call The 5 Ts


Don’t find it… make it!
Developing test automation takes time (I just rolled my eyes at myself as I typed that). Of course this is painfully obvious to you and absolutely everyone else… but still, time is always an issue when developing test automation. you will always struggle to find time to write new tests, design better tests, maintain existing tests and refactor tests when things change.

But you can’t find time for automation… you have to make time for it. Factor it right into your testing estimates and/or block off specific time for automation. The amount of time, or a lack thereof, allocated to automation is a great indicator to how committed your team/stakeholders are to automation and will be directly related to it’s success.


The right tool for the right job…
There is a veritable plethora of test automation tools available today and there are even more opinions on which is best. Picking the best tools depends heavily on the context of your project and the skills of your team. Choose wisely and try before you buy! Give your tool candidates a spin for an iteration (week or two) and see how they perform in the field.


Not just tests… great tests!
Start writing tests! Write them iteratively, refactor regularly and fix failing tests quickly (keep ’em green!). Value working tests like you would working software. Start small (eg. smoke tests) and expand to improve coverage. Consider continuously adding to your framework rather than striving for the perfect framework up front. Aim for beautiful tests that are concise, easy to read and will be easy to maintain.


Show your cards…
Get your automated tests in front of your team/stakeholders/management and solicit their input. Testing should be a group activity… get the group involved! Share your testplans with your team; set up regular test-code reviews; pair-program. Celebrate your milestones and accomplishments! Schedule tests to run often and post results for all to see. Fast, consistant feedback will improve your tests, help manage expectations and show a return on your automation investment.


Trust me…
The ultimate goal of test automation, and the destination on this roadmap, is trust. Without trust, test automation has no value. As the oracle for your SUT (system under test), you, your stakeholders, and your team, must be able to trust the answers it gives. Such trust takes considerable time and effort to build and is derived by successes with the previous 4 Ts. With enough time, the right tools, great tests and a transparent effort, trust in your automation will grow. Trust me!