Talking testing, agility and automation... and anything else.



18
Dec 16

Re-Run Flakey Tests with Protractor-Flake

If you’ve worked with E2E tests for any amount of time, you will have experienced tests that, for whatever reason, randomly fail; aka flakey tests. Such failures can be caused by any number of things: a network glitch, browser barf, app hiccup, act of God, etc….

Test flake can seriously undermine the trust in automation, and drive your automation engineer to drink (or perhaps just drink more). A wise test engineer will embrace that flake happens, and account for it in their automation strategy.

One popular strategy is to re-run failed tests. This way, if a test fails, it gets re-run ‘n‘ number of times, and if it passes, life is good. Enter Protractor-Flake.

Protractor-Flake parses your Protractor test output, looking for failures, and re-runs any failing tests (note: at the spec file level) at the end of your test run. It works really well and is simple to setup. Here’s how…

First install it as a dependency:

Then you can run it directly by using it in place of Protractor (note: the ‘–‘ denotes the end of args passed to protractor-flake, and the beginning of args passed to Protractor):

Or you can use it programmatically. I like to create a simple node script, which allows me to add logging and reporting (future blog post). Here’s an example:

Saving the above script as a file, 'flake', you could then run your tests thusly:

Now obviously, you should be killing off flakey tests to the best of your ability, but Protractor-Flake can save you from non-test-related flakiness, and keep your tests green! You can see a working example in my protractor-example repo on GitHub.


06
Feb 15

How To Write A Good Bug

A reader gave mefusca9 flak for writing about automation way more than about testing. Okay. Guilty. So towards making amends for this fact, I give you, Brine’s guide to writing a good bug.

A good bug includes four things:

  • Title
  • Steps
  • Results
  • Impact

How to write a good bug:

1. Write the title last

A title is the most important part of a bug. A good title should convey all (or as much of) the information about the issue… to the point where reading the rest of the bug is unnecessary. And it should do so in as few words as possible.

I used to try to nail the title before writing anything else, but I found it just takes too much time. And I would usually end up rewriting it before submitting anyway. Write it last! I’ve found titles are much easier to write after you’ve written the bug itself.

2. Write concise steps to reproduce the issue

Steps should have just enough detail. Too little and it won’t be reproducible, too much and it can get confusing. If you find yourself writing a bug that has 10+ steps, you either have a very complex bug, or overly complex steps. This can also depend greatly on the bug’s audience and the preferences of the team. Still, keep it concise.

3. Describe the results and the expected results

This is the easiest part. After following your steps, what happened, and what did you expect to happen?

4. Define the impact of the issue:

You might think the impact of your bug is “high” (and what QA wouldn’t), but that term isn’t very useful, and certainly wouldn’t rate a whole section in a bug! No. The Impact section is where you try to PROVE the impact of a bug. This helps the writer because attempting to prove the impact of a bug requires proper regression. It also shows the reader what you’ve tried, and provides insight into the severity of the issue, and areas to verify upon being fixed.

For example:

  • Is there data loss?
  • Is there a workaround?
  • Does it occur on multiple environments?
  • Does it occur in Production
  • Does it occur on multiple browsers/apps/OSs?
  • Did it occur in the last build?

These questions show impact. They tell the reader just how bad, how far reaching, and/or how recoverable, the issue is.

Oh, and there’s one more thing…

4.5 Attach a screenshot

If the issue is cosmetic, it had better have a screenshot attached! Screenshots and/or movies can be worth the entirety of that classic cliché, a thousand words

fin


24
Sep 14

It’s Funny ‘Cause It’s True

n-HELLO-MR-large570A QA Engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 999999999 beers. Orders a lizard. Orders -1 beers. Orders a sfdeljkn.


19
Jun 14

Mythical Man Month or: How I Learned to Stop Worrying and Love Management

Dilbert-add-people-to-project

When I’m finally in charge, two of my first mandates will be: If you’re driving in the left lane, BE passing. But only slightly less important will be: “Please read Mythical Man Month”.

And you don’t even have to read the book if you don’t want. Simply read the Wiki page about the book and believe it. In fact, you can skip the book and the Wiki page if you’ll repeat the following three times, out loud:

“Adding manpower to a late software project makes it later”.

You may now become a manager…


24
Feb 13

Amazon EC2 Ubuntu HowTos

ec2Having a server in the cloud is a nicety. That Amazon gives you one free for a year to tinker with… that’s being downright friendly!

The following are some notes/howtos for setting up an Ubuntu Amazon EC2 instance. I post them to help me remember them but they might also be useful to others…

Allow SSH Access

If you’d prefer to just ssh into your instance instead of using key/pair…

1. Login using the key/pair you created when setting up your EC2 instance:

2. edit your sshd_config file and uncomment and/or set PasswordAuthentication yes:

3. Reload SSH:

4. Create a new user and set the user’s password:

Install LAMP

1. Update Ubuntu’s package database, install and run Taskel. Select the LAMP server (don’t deselect anything that is also checked) and let it install:

2. Install PHPMyAdmin; select Apache2, enter a root password and say No to config the database later:

3. Verify by going to: http://myIPaddress/phpmyadmin and login as root with the password you entered. Of course you’ll probably want to secure/move this!

Install XRDP on Ubuntu 12.10

Installing X11 on your instance–should you be so inclined–may prove tricky… I tried MANY ways of doing it (OpenBox/FluxBox, VNC, etc…) but each attempt ended in failure. I found victory with XRDP!

1. Found from the instructions here. First install Gnome (will take a while)…

2. Ubuntu 12.10 no longer includes gnome-session-2d, so install fallback and edit .xession to use it:

3. Edit Xwrapper file and set allowed_users=anybody:

4. Create a new Security Group Rule for RDP(i.e. open port 590x (where x is the vncserver id))

5. Install RDP client on your local machine…

Install Chromium brownser on Ubuntu 12.10

FireFox is in need of Unity by default and it appears it doesn’t exist on 12.10. I wanted Chrome on there anyway but it didn’t work either. Chromium does: