This article was published in Sogeti’s QANews on September 30, 2011. See their article here
Acceptance tests are a great way to improve collaboration between testers, developers, and customers. When all three parties agree ahead of time how to test the feature, it makes it more likely the developer will deliver working code.
Automating those acceptance tests is even better, because now the tester can easily execute all previous acceptance tests written and ensure that new features are not breaking previously existing code.
Brief introduction to ATDD
Acceptance Test Driven Development (ATDD) is a powerful way to write acceptance criteria for your agile project’s user stories.
Instead of just writing a list of requirements, you write acceptance criteria in a specific format. Your acceptance tests are written in simple formats, but they can be turned into executable code.
These acceptance tests are written before development is done, and so when the tests are executed the first time, they will all fail. This is a good thing!
The developer now knows exactly what they need to do to write the feature. They keep developing code until the acceptance tests pass, and then they are done.
Writing your “requirements” as executable code that the developer can use will improve communication between the customer, tester, and developer. Ideally all three parties will sit down together to write the acceptance tests, and then the developer can take it from there to implement the code.
Manual or Automated ATDD?
You could write your acceptance tests with the developer, and then have testers manually test them at the end of the project. In a loose sense this is still acceptance test driven development.
But you will get much more value out of it if you write the tests so they can be automated. Then testers can quickly execute all the acceptance tests written for the project (even beyond the current features being developed), and they will have confidence that the new development is not breaking old features.
Cucumber is a common framework for automating acceptance tests, and the tips in this article assume you are using it. Cucumber is implemented in Ruby, but you can use it to test virtually any system, regardless of whether your system is written in Ruby, Java, .NET, etc.
Cucumber also has multilingual support. So you can write your tests in English, Spanish, or many other languages.
Tip #1: Adopt the Given/When/Then format
The Cucumber testing framework uses a language called Gherkin for writing the acceptance tests. These tests follow this basic pattern:
Given [some precondition or starting state]
When [the user does something]
Then [something happens]
Using this format helps to make the test clear, and encourages you to properly describe the preconditions for the test, what initiates the test, and what the expected outcome is. This simple pattern makes it easy to visually scan a test and see what it does.
Tip #2: Group scenarios based on common backgrounds
In many of your tests, the “Given” clauses will be very similar. Gherkin allows you to save time by putting these common pre conditions in a “Background” section of the test file so that you are not writing duplicate code.
This also helps you determine when you can group scenarios together in the feature test. If a scenario does not fit the background pre conditions, then it should go somewhere else.
As an example, you might initially consider grouping together tests that deal with adding an item to a shopping cart. But on further consideration, you notice there are really two different set of features to consider: adding items to the cart when you are a logged in shopper, and when a new customer adds items to their cart. These could be considered two distinct sets of features since they have different pre-conditions.
Tip #3: Mock external dependencies
Automated tests need to be fast, otherwise you will not run them often enough. Teams may delay starting the automated tests if they take hours to run. The tester says “We’ll start it just as soon as a couple more developers check in their code…”
But this means you are delaying the rework of any bugs found, and that can slow down the whole project.
If your automated tests take too long, it’s may be due to an external dependency. Is the test making a call to another system? Can you “mock” that other system and replace it with fake code so that the test runs faster?
Frameworks like JMock and nMock make this possible, so that you can focus on testing your code instead of worrying about the external systems it depends on.
Tip #4: Example scenarios to build common terminology
The Cucumber framework is great, but it relies on “ubiquitous terminology.” You don’t want to write different pieces of test code for all of these statements:
Given that a user has logged in
Given that the user logs in
Given that the user is logged in
When you read these sentences as a human, it’s obvious they all mean the same thing. But the automated testing framework doesn’t understand that, and it expects you to implement three different pieces of test code for them.
Using common terminology across your team is important, so consider posting examples of common test steps on the team room wall or on a wiki. Then you can re-use other test code more easily.
Automating ATDD improves communication
Many teams want to speed up testing, increase automation, and improve communication between developers, testers, and customers. Automating your acceptance tests with Cucumber can definitely achieve all of these goals, and is worth the investment.
Following these tips will help you make the most of your automated ATDD adoption.
About the author
Arin Sime is owner of AgilityFeat, where he provides agile coaching and development services. Arin is a regular speaker at agile conferences and user groups, including the recent XP2011 conference in Madrid. Arin holds a Masters degree in Management of I.T. from the University of Virginia. You can reach him at Arin@AgilityFeat.com or on twitter @ArinSime.
