Talking testing, automation... and anything else.



21
Dec 12

Test Automation By Example With Sahi

Matrix-CodeI created the following example test script to illustrate some practices I’ve found useful when writing test automation. This example is written in Sahi but these practices are tool-agnostic and could be transfered to any tool (though Sahi is a great one!). For our example, we’ll be using MyTrashMail.com–a fine testing resource–as the SUT (system under test). Our goal is simply to confirm that an email can be deleted from the system.

The Test

Size Matters!

The first thing you may notice is the size of the test… it’s quite small and size does matter! Barring end-to-end or scenario tests, small tests are best. They should be as singular and short as possible. 20 lines of code or less is a good rule-of-thumb, as is a single assert in each test. Long tests and multiple asserts are signs that you might be testing more than one thing. Short, concise tests are easier to read and maintain, and lend themselves to be run in parallel (multiple tests running at once).

Filenames as documentation

You may have glanced right over another practice I find invaluable… the script filename (which I’ve placed in the header comment): “0001.delete.random.email.from.indbox.and.confirm.deletion.sah”. I’m a big fan of using code for documentation and that definitely includes filenames. Test script filenames (or class names), like bug titles, help to concisely communicate what is being tested. In this way, a list of filenames from your test suite could double as an impromptu test plan/test script.

Having an ID in the filename is also very helpful. This ID could be the story card ID, bug ID or just an incremental ID. This will allow you to group like files together, aid in searches, and provide a cross-reference from the test to associated story/bug should you need it (and you probably will).

Script Walkthrough

Now let’s walk through the script a bit… The first two lines just include the script’s associated functions (more on this in a minute) and sets the MyTrashMail account name we’ll be using.

The next line uses Sahi’s built-in javascript method _navigateTo to navigate to our system under test, MyTrashMail.com. Sometimes there’s no need to reinvent the wheel as Sahi’s built-in methods are quite good.

Abstraction Via Functions

Next, instead of using Sahi code to enter our email account name, we break it out into a function for a couple of reasons. First, the function name checkEmailAccount() is more descriptive than the Sahi code and aids readability, and second, it can help maintainability. For example, should we use this function in multiple scripts, and should the application code change, we need only update the function to fix multiple tests. Both of these improvements could likely see further enhancement if we also incorporated page objects… but that’s a topic for another day.

Now we’re into the heart of the test… we need to click into an existing email and here again we’ll break this into a function which will greatly help to streamline the code. This could be handled a number of ways but strategically adding a little randomness into tests can expand coverage, and enhance the possibility of the script finding a bug. Thus, we create clickRandomEmailLink() to do the dirty work. As an added bonus, the function also provides some error handling and returns the random email’s url for use later in the script!

Functions

As you can see, clickRandomEmailLink() gets the number of emails in the account by counting the number of little mail icons (messagestatus0.gif (thanks MyTrashMail devs!)). If there is at least 1 email, we then pick a random number between 0 and the $numDisplayedEmails and use that to select the Nth link under the Subject column (_cell("Subject")), in the Table1 table (_table("Table1")).

We also return the browser’s current URL… more on that in a bit.

If there are no emails in the account (and the test currently expects at least one to exist), the script fails and logs the failure via Sahi’s method _logExceptionAsFailure().

With me?

Found Data

It’s important to note that we’re not creating an email in this script; we don’t know (and don’t need to know) anything about the email. We’re boldly jumping in and finding what’s there in the system, and testing it.

In this example, we’re using a popular account name “trash” that seems to have a continuous number of emails coming in. In other “normal” instances, we would have another script to test email creation elsewhere (and likely more than one). These tests would then feed off of each other… one creating and one deleting.

Final Stretch

Finally, our work pays off… we delete the email. We could have just left the Sahi code in the script but deleteEmail() reads a bit better…

And now that we’ve successfully deleted the email, we use the $emailURL that was returned by clickRandomEmailLink() to return us to the scene of the crime… so to speak. We navigate back to the email we deleted to assert that it’s actually been deleted. I use this practice a lot when jumping back and forth in the app. It ensures that we’re testing the right element. Eg. you could also have returned the email subject but there could be multiple emails with the same subject. Plus, in the log, we’ll have a direct link to the email should we need to investigate.

And lastly, we come to the assertion itself. As I mentioned, I’m a fan of one assertion per test. In addition, I like to put them on their own line separated by white space to aid in scanning for them. In our case we simply assert that the delete message is displayed and in doing so, make use of Shahi’s ability to use regular expressions (an absolutely fantastic feature).

Summary

Even with such a simple test, there’s a lot of good insights to observe. Here again are some of the key points…

  • Short tests are best. Keep them singular and simple
  • Use naming conventions that aid readability and can double as documentation
  • Abstract large blocks of code into functions
  • Use strategic randomness to expand coverage
  • Make use of the data available in the system
  • One assertion per test

You can copy and paste the code examples above or download them here.


25
May 12

Javascript Page Object Pattern for Sahi

I’ve been reading about the virtues of the Page Object design pattern and am intrigued by the benefits it could offer. I found examples of it’s use in many of the popular automation tools like Selenium, Watir and WatiN, but very little information about a Javascript implementation, or it being used in Sahi (my current automation tool of choice). Thus I was forced to have a go at it myself.

First off, a Page Object is a design pattern intended to create a model of a web page in code via a object or class. If you’ve written even a bit of automation code, you’ll recognize the amount of chaff created to accommodate your ultimate test goal; clicking through ui elements, repetitive steps, etc…. a Page Object attempts to abstract those clicks and steps and allow for more elegant and maintainable test code. Therefore, for each page in your web application, you’d have a corresponding object that would provide the services offered by that page to your test.

An example

Here is my take at a Page Object pattern for Sahi, written in Javascript (though Sahi does use a “$” in front of it’s vars to differentiate them from page Javascript vars). In my example, I’ve created three objects; a home page, a search results page and a shared, common object page (that might exist on multiple pages). You can copy this code and run it as is in Sahi Pro (the open source version currently has a bug that will make this fail. Email me and I’ll explain a workaround).

As you can see, in addition to the cleaner testing code, Page Objects also offer a bit of encapsulation, not to mention the ability to define a domain language for your tests.

I decided to go with object literals for the page objects as they take care of instantiation (no need to use new to create an object) and they are easy to use and read. One downside is they can’t currently take advantage of inheritance but they seem to do the trick nonetheless.

Any items shared across multiple pages (nav bar, sidebar, etc…) should be broken out into their own objects and instantiated in the various page objects that make use of them (like SeachFilter in my example).

As I’m new to this concept, please do feel free to email me and let me know how this code might be improved. I’d be happy to have the input.

UPDATE: I’ve since added an updated and expanded example with working code:


21
Apr 12

Accessing Browser Objects Using Sahi

Sahi is a great automation tool for testing web applications. While Similar to Selenium, Sahi offers many notable advantages over Selenium, including some that were deal breakers in my current situation (IE support for instance).

Sahi also offers some ways to access page objects that are just flat out awesome. I’ll discuss a few here, specifically strings, indexes and it’s built-in Browser Accessor API _under.

Consider the following HTML table…

Last Name First Name
Deng Luol
Rose Derrick
Gibson Taj

Suppose you had a story that called for you to click a link from the table above. Simple enough… in Sahi you could write something like this:

Yup. This code does indeed open Lu’s Wikipedia page as expected. It’s using the link text to access the page’s object (not an xpath). Handy for sure but what if you had other sites to test that contained different data in this same table (hopefully not any Miami Heat Players)? Or maybe these links are commonly deleted by other scripts you’re running? The code above would fail.

No problem. Sahi indexes all page objects which allows you to access the “nth something” on the page. In this example, we could just click the first link. And in case there are other links elsewhere above, we’ll be more specific and make use of one of Sahi’s fantastic relational accessors, _under. So…

Lu’s Wikipedia page again appears! What’s happening here is we’re clicking the first link (indexes are zero-based a thus start with zero) under the table header “Last Name”. Thanks to the indexing of the links, we could also iterate through all of the links via a simple for loop.

But now maybe we’d like to take this up a notch and inject some randomness that will help broaden our testing scope and perhaps improve our chances of finding a bug (the purpose of testing after all). And maybe we’ll also make it a function to help keep our automation more readable. So perhaps something like…

This function uses Sahi’s _count method to count the number of cells that have an id of lastname. (Note: because the results of _count are NOT zero-based (starts with 1) it returns “3” so we just subtract 1. We then use Sahi’s method _random to get a random number between 0 and $numPlayers (2). Now we call the same code as in our previous example to click one of the player’s links at random, under the table header “Last Name”.

And now any one of my Bulls player’s Wikipedia pages is displayed when running the script! These are just a few of the ways Sahi lets you get at page objects… there’s a bunch more including _leftOf and _rightOf, new to Sahi Pro 4.2.