Talking testing, automation... and anything else.

Jun 19

E2E Testing with Testcafe

I started a new job at the beginning of the year which has contributed to my lack of blogging, but is also the impetus for exploring some new automation frameworks. I’d settled into using Protractor at my last job, but was interested to take a peek at any new tools that had popped up in the last few years… which is how I found Testcafe, a great, open source, E2E test framework.

Take a peek at my github account, you’ll see that I often port a handful of dorky, example tests over to various test frameworks to get a sense for them. And now you’ll find my port to Testcafe there as well. Porting these tests gives me a real sense of what makes a framework tick; the good and the bad.

Testcafe has a number of interesting features but the one that immediately caught my eye was implicit waits (ie. the framework handles waiting for page/element loading). For anyone who has written their own explicit waits (in which waiting for things is your problem), implicit wait would likely be very compelling! I feel like at some point, the industry decided that implicit waits were bad… I disagreed with that then and I disagree with it now. Implicit waits save a TON of time and assuming the framework offers a reasonable way to handle negative cases (eg. that an element does not exist), we’re all good. YMMV…

Another interesting feature is it does not use Webdriver. Probably like most people, I have a love-hate relationship with Webdriver: love what it can do; hate the bugs/inconsistencies in the various browser implementations. Testcafe (similar to Sahi) uses a proxy to inject test code into the browser. Personally, I don’t care how a tool makes the sausage… I just care that it does work and [SPOILER:], it works!

Of course Testcafe also hits on a number of goodies:

  • It’s open source
  • Tests are written in Javascript (es6)
  • Parallel test runs
  • Support for all the major cloud browser services
  • Page object support

But it was when I was porting my example testcases to Tescafe, that I found the best feature of all… the community. Simply Googling for testcafe [shplah] almost every time returned pertinent results for the questions I had. They have great documentation, an active community forum, and a fine showing on stackoverflow.

So in my new position, I spiked out a few small projects in Testcafe and another leading Webdriver framework and posed it to my team to choose. Testcafe won out!

Nov 14

Screencast: Protractor Test Automation Framework Example Code

I’ve been playing around with Protractor, a great, new(ish) testing framework from our friends at Google. I’ve shared my example code on GitHub for those that might be interested in such things.

Examples in the code include:

  • Using page objects
  • Running on TravisCI
  • Running tests on Sauce Labs and Browserstack
  • Running multiple browsers at once

I’ve also made a quick screencast that walks you through downloading and running the example code.

Cut to the chase, in the video we:

  • Install Node (okay, I don’t show this but you need do to it!)
  • Download the example code from GitHub
  • run npm install to install the project dependancies
  • Briefly discuss a config file
  • run protractor conf.js to run the tests

Jan 14

Spock: Test Framework With Creamy BDD Center

spock[1]Two things I’m willing to admit:

  1. I think it’s a bad idea to dismiss tools prematurely
  2. I think most BDD tools are crap

Delicious hyperbolic contradiction aside, it would probably be more accurate (and PC) to say I’ve yet to work in a situation where using tools like Cucumber or JBehave would make sense. I.e. these tools just don’t compliment my, or my team’s, workflow.

Enter Spock. Spock is a testing framework for Java/Groovy, that is very similar to JUnit, but also adds BDD-friendly specification, baked right in. With Spock, there’s no need to maintain a separate feature or spec file; no weird broken-English language to learn. Your code is the spec and the end result is clean, readable and maintainable.

Case in point: the following is a Geb test, written in Groovy and using Spock. This is working code from my geb-example project on GitHub.

    def 'searches with few results offer nearby results'() {
        at CLMainPage

        when: 'select category from dropdown and perform search'
        dropdown = category

        then: 'nearby results message is displayed'
        at CLSearchResultsPage

This test, from my CLSearchTest class, which tests searching on Craigslist (had to use something :) ). With that in mind, the code above is probably very readable to just about anyone. I’ll break down what’s going on a bit…

We start by defining our test: 'searches with few results offer nearby results', which is also our method name. Method names are string literals (can have spaces).

Next we have Spock’s built-in code blocks, labeled given:, when: and then:. These three blocks represent the different phases of a test. Each block can also include an optional description to help with readability. A bit about each block:

  1. given: (optional): Think of this block as a setup. It gives you the optional ability to specify state.
  2. when: (required): This is where the meat of your test code will go. You can have multiple Whens and Thens but they must be used in pairs.
  3. then: (required): This block is for asserts. All code in this section must return true or your test will fail. This gives your reader an easy way to see what the test is actually testing!

This just scratches the surface of Spock’s offerings. You of course get a full boat of annotations (including my favorite, @IgnoreRest) and fixture methods, additional blocks, etc… Plus it runs anywhere JUnit runs.

Don’t get me wrong, the principals of BDD are sound but I want tools that work the way I do, not the other way around. When using Spock along with Geb’s page object pattern and selectors, and Groovy’s fabulous syntax, the resulting code is both elegant and readable. YMMV…

Feb 13

Amazon EC2 Ubuntu HowTos

ec2Having a server in the cloud is a nicety. That Amazon gives you one free for a year to tinker with… that’s being downright friendly!

The following are some notes/howtos for setting up an Ubuntu Amazon EC2 instance. I post them to help me remember them but they might also be useful to others…

Allow SSH Access

If you’d prefer to just ssh into your instance instead of using key/pair…

1. Login using the key/pair you created when setting up your EC2 instance:

ssh -i myprivatekey.pem [email protected]

2. edit your sshd_config file and uncomment and/or set PasswordAuthentication yes:

sudo pico /etc/ssh/sshd_config

3. Reload SSH:

sudo reload ssh

4. Create a new user and set the user’s password:

useradd NAME
passwd // enter password twice...

Install LAMP

1. Update Ubuntu’s package database, install and run Taskel. Select the LAMP server (don’t deselect anything that is also checked) and let it install:

sudo apt-get update
sudo tasksel

2. Install PHPMyAdmin; select Apache2, enter a root password and say No to config the database later:

sudo apt-get install phpmyadmin

3. Verify by going to: http://myIPaddress/phpmyadmin and login as root with the password you entered. Of course you’ll probably want to secure/move this!

Install XRDP on Ubuntu 12.10

Installing X11 on your instance–should you be so inclined–may prove tricky… I tried MANY ways of doing it (OpenBox/FluxBox, VNC, etc…) but each attempt ended in failure. I found victory with XRDP!

1. Found from the instructions here. First install Gnome (will take a while)…

export DEBIAN_FRONTEND=noninteractive
sudo apt-get update
sudo apt-get install -y ubuntu-desktop

2. Ubuntu 12.10 no longer includes gnome-session-2d, so install fallback and edit .xession to use it:

sudo apt-get install gnome-session-fallback
pico .xsession
gnome-session --session=gnome-fallback // add/change this line in .xession 
/etc/init.d/xrdp restart

3. Edit Xwrapper file and set allowed_users=anybody:

pico /etc/X11/Xwrapper.config

4. Create a new Security Group Rule for RDP(i.e. open port 590x (where x is the vncserver id))

5. Install RDP client on your local machine…

Install Chromium brownser on Ubuntu 12.10

FireFox is in need of Unity by default and it appears it doesn’t exist on 12.10. I wanted Chrome on there anyway but it didn’t work either. Chromium does:

sudo apt-get install chromium-browser

Feb 13

Adding Copy Filename and Copy Path to Sublime Text’s Sidebar

Sublime_Text_Logo[1]My text editor of choice is Sublime Text but right out of the box, it does not include copy filename or copy file path in the sidebar (both of which I use constantly). Seems that whenever I install it on a new machine (which I just did) I forget how to install the Package Control package, which offers such goodies. Therefore, I thought I would park the instructions here for your–and more importantly, my–convenience.

It’s pretty easy…

  1. In Sublime Text, type ctrl+` to access the console
  2. Copy the code below, paste it in the console and hit return
  3. Restart Sublime Text
  4. import urllib2,os; pf='Package Control.sublime-package'; ipp=sublime.installed_packages_path(); os.makedirs(ipp) if not os.path.exists(ipp) else None; urllib2.install_opener(urllib2.build_opener(urllib2.ProxyHandler())); open(os.path.join(ipp,pf),'wb').write(urllib2.urlopen(''+pf.replace(' ','%20')).read()); print('Please restart Sublime Text to finish installation')

Now, when right/ctrl-clicking a file or files in Sublime Text’s sidebar, you should see a wealth of options including copy path/name.

If for some reason this fails to work, you can download and install manually with the instructions here…

Now that you have Package Control installed, use it by typing shift-ctrl-p. You can then, as you might imagine, control Sublime packages. Eg. install, remove, etc…. My two must-haves are: SideBarEnhancements and All Complete. To install them:

  1. Click shift-ctrl-p
  2. Start typing “install” and select Package Control: Install Package
  3. A package prompt appears; start typing the package name, eg. All Auto Complete

That’s it! Just repeat these steps for each package you’d like to install.