How not to prepare test data in JBehave and Cucumber
Preparing test data is hard, avoiding duplication and unnecessary setup time are common issues in all test automation. But it is especially important when we automate acceptance criteria using BDD tools such as JBehave and Cucumber.
Duplicating setup logic in each scenario can lead to cluttered and hard-to-read scenarios. And having to inject the same test data for each test can slow down the test suite. In the rest of this article, we look at different strategies you can use to set up your test data. And learn why you should avoid many of the more common ones.
The case of the duplicated frequent flyer
Suppose we are testing a Frequent Flyer application. We might need to set up a customer with a frequent flyer account for many different test scenario.
We could describe how a frequent flyer registers in a scenario like this one:
Scenario: Joe registers as a standard member
Given Joe is a frequent flyer
When Joe provides his personal details
Then he should be registered as a Bronze Frequent Flyer
And he should receive a welcome package via email
We might also have another scenario if Joe is transferring his status from a competing airline
Scenario: Joe registers as a gold member
Given Joe has 10000 points with a competing airline
When Joe provides his personal details
And Joe transfers his points to our Frequent Flyer programme
Then he should be registered as a Gold Frequent Flyer
And he should receive a welcome package via email
Now suppose we have a scenario describing how Joe can use his gold status. How do we make sure that Joe the gold-card frequent flyer member exists before the scenario starts?
We could duplicate the steps from the previous scenario:
Scenario: A Gold-card member can always access the lounge
Given Joe has 10000 points with a competing airline
And Joe provides his personal details
And Joe transfers his points to our Frequent Flyer programme
And Joe is flying economy
When Joe trys to enter the lounge
Then he should be granted access to the lounge
But this is noisy and muddled. The setup steps make it hard to see what we are trying to demonstrate. And we would need to repeat these steps for every scenario involving a gold card member.
How can we avoid duplication like this?
Using GivenStories
JBehave's solution is to propose the GivenStories keyword, which lets you run an entire story before the steps of your scenario. For example, the scenario shown above might look like this:
Scenario: A Gold-card member can always access the lounge
GivenStories: create_gold_user.story
Given Joe is flying economy
When Joe trys to enter the lounge
Then he should be granted access to the lounge
Don't write stories like this.
Seriously, don't.
Using a story as a precondition or as a reusable piece of logic is a big anti-pattern, and leads to unclear and poorly written scenarios. In programming terms, you might say it is a violation of the "Single Responsibility Principle" - a component should do one thing and one thing only. It is also a cumbersome tool; you can pass parameters to the stories, but the syntax is clunky and hard to read.
The problem is, Stories and Scenarios are not really suited for reuse. JBehave takes the "hammer-and-nail" approach to reuse: when all you have got to work with are story files, the only way to avoid duplication is to reuse stories. But it works very poorly; Gherkin is not a scripting language and shouldn't be used as one. And there are much better options available.
Relying on scenario order
Some testers rely on the fact that the scenarios in each story are executed in their order of appearance, and have one initial scenario to set up the test data, and then subsequent scenarios that use this data. For example:
Feature: Checkout
Scenario: Create a registered frequent flyer
Scenario: The frequent flyer earns some points
Scenario: The frequent flyer books a flight using his points
This approach is not ideal either. While it avoids duplication, it means that you can only ever execute all of the scenarios as a set. For example, you can't run "The frequent flyer books a flight using his points" in isolation, which makes debugging slower. It also means that if one scenario fails, the following scenarios are likely to fail as well, which can be misleading: just because there is an issue with the "earn points from flight" feat does not mean that the "book flight with points" feature is broken.
Background Steps
In Cucumber, you can use background steps like this:
Feature: Gold card members
Background: Joe is a gold-card frequent flyer
Given Joe has 10000 points with a competing airline
And Joe provides his personal details
And Joe transfers his points to our Frequent Flyer programme
Scenario: A Gold-card member can always access the lounge
Given Joe is flying economy
When Joe trys to enter the lounge
Then he should be granted access to the lounge
Scenario: A Gold-card member can use the priority access lane when boarding
...
This reads more smoothly. However each scenario will run the background steps, so it will slow down the test suite if setting up the test data takes time. The three background steps will also appear in each scenario report, so it tends to clutter up the living documentation.
Reusable Components
A far better solution than any of the above is to avoid including setup logic in the Gherkin scenarios at all. Instead of describing the steps you need to perform, the Gherkin scenario should focus exclusively on describing the initial state.
Scenario: A Gold-card member can always access the lounge
Given Joe is a gold-card frequent flyer
And Joe is flying economy
When Joe trys to enter the lounge
Then he should be granted access to the lounge
The first line ("Given Joe is a gold-card frequent flyer") describes the initial state we want. We don't describe a particular solution for getting to that state; we leave that to the step definition code.
There are many ways to implement setup code. For example, using classic Serenity BDD, you could create a FrequentFlyers class to encapsulate the logic of injecting test data:
@Steps
private FrequentFlyers frequentFlyers;
@Given("(.*) is a (.*)-card frequent flyer")
public void givenAFrequentFlyerMember(String name, String level) {
frequentFlyers.ensureThatMemberExistsWithNameAndLevel(name, level);
}
Another option would be to use the Screenplay Pattern and a builder strategy, to allow more flexibility about the test data being injected.
Actor daveTheDataManager = new Actor("Dave");
@Given("(.*) is a (.*)-card frequent flyer")
public void givenAFrequentFlyerMember(String name, String level) {
daveTheDataManager.attemptsTo(
RegisterAFrequentFlyerMember.called(name)
.withStatusLevel(level)
)
}
These programmatic approaches are much more flexible than the Gherkin-based ones we saw previously. We are not reliant on the way a particular story or scenario does things. For example, the "Register frequent flyer" scenario might demonstrate how a user registers online, and be implemented as a web test, whereas our setup component might inject data directly into the database. Or we might check whether the test data has already been injected, to avoid doing it twice.
Conclusion
Setting up test data, particularly if there is conditional logic involved, is better done at the coding level; the steps in JBehave and Cucumber should simply describe the state they need the system to be in. And with this approach, you will never need to use constructs like GivenStories.
Related Events and Workshops
Want to learn more about BDD and advanced test automation skills, at your own pace? Take a look at our new Serenity Dojo online training programme.
Not your typical online course, the Serenity Dojo programme combines training videos, tutorials, exercises, and a great online community where you can interact with other students and Serenity experts, and learn about the range of skills you need to master the art of agile test automation.
Advanced BDD Test Automation
Deliver world-class test automation to your team. Write more robust, higher quality automated acceptance tests using state of the art test automation practices. And learn how to get the most out of BDD and test automation using libraries like Cucumber, Serenity BDD, Selenium WebDriver and Rest Assured.
