Don’t let automation sabotage your BDD adoption
Test Automation can be a powerful aid for teams adopting Behaviour Driven Development. But do it too early, or in the wrong way, and your efforts can have the opposite effect.
We all want to deliver great products. We all want to delight our customers. But sometimes, reality gets in the way, and delivering features that truly deliver value isn’t as easy as it seems. No matter how hard we try, defects appear, deliveries are delayed, and customers are less-than-satisfied. Worse still, sometimes our best efforts to ensure we deliver better, higher quality software can backfire in our faces and actively hamper our efforts.
Test automation and tooling can be a prime example of this, especially when it is used in the context of BDD adoption.
“It’s about the collaboration, stupid!”
Software development is a collaborative game. Communication is key. Indeed, some studies find that up to 70% of software issues can be traced back to misunderstood or misinterpreted requirements.
And Behaviour Driven Development is designed to help reduce these misunderstood requirements. Behaviour Driven Development, or BDD, is a collaborative practice where teams use conversation about examples and rules to build a deep shared understanding of customer needs, with the goal of delivering software that matters. The principles of BDD can be applied at a product design and planning level, where they help product owners make more educated choices about what features to build or to try out on the market, right down to the unit testing level, where they help developers design easier-to-use and better documented APIs. But it is most commonly associated with writing better acceptance criteria, or “executable specifications”, like the one shown here:
Given Jill is a Premier customer
And Jill has £1000 in her bank account
When she withdraws £100 from a third party ATM
Then she should be charged £1 transaction fee
And she should have a balance of £899 in her account
When done well, BDD does work. I have seen may teams adopt BDD in various forms, and reap the benefits. Teams that embrace BDD well:
- collaborate more effectively
- are more engaged,
- introduce fewer defects,
- and deliver features that delight the customer sooner
But, like many techniques that seem easy on the surface, BDD too can be tricky to master.
When BDD adoption goes wrong
BDD adoption can also go horribly wrong. Done poorly or inappropriately, BDD can confuse stakeholders, discourage developers, and make life harder for testers, and put teams off BDD forever.
In this article, I want to look at a few ways that I have seen this happen, and what you can do to avoid falling into the same traps.
Conversation establishes. Formalisation validates
Starting out with BDD is as simple as asking a product owner “can you give me an example of that requirement”. The heart of BDD is the conversation between the product owner and the development team, discussing concrete examples of flows and business rules. These conversations helps us establish an accurate understanding of what the business needs and how we can help.
These conversations correspond to the Discover and Define phases in the diagram below:
In a team practicing BDD, the conversations typically involve a business representative (a product owner or sometimes a business analyst acting as a proxy), a technical representative (developer) and a testing representative (a tester). In practice, there may be more people involved, but these three roles are a minimum. We often refer to these workshops as the “3-amigos” workshops.
But conversation alone is not enough. We need to know that we have not misinterpreted the requirements, or that we don’t forget the finer points later on in the sprint. So oftentimes we record the examples in a more formal format, one that we can execute as part of our acceptance tests, but one that the product owner can still read and understand. This happen in the Formalise phase of the diagram shown above. These unambiguous “executable specifications” help us double-check and confirm the shared understanding that we established in the first phase.
In other words conversation establishes the shared understanding, and then formalisation validates it:
And once the shared understanding is established, transforming the examples into executable specifications can easily done by one or two team members well versed in automation skills.
Cucumber indigestion
But for many teams, the natural place to start with BDD seems to be to install Cucumber, write some feature files and try to automate them. In some teams, business analysts write acceptance criteria in the Given-When-Then format when they create new stories, well before they show them to other team members. In other teams, the requirements discovery workshops or “3-amigo” sessions focus on producing Cucumber-formatted features and scenarios.
These approaches actually make BDD adoption harder.
Despite its deceptively business-friendly vocabulary and readable format, the Gherkin syntax used by Cucumber is a formal notation. It is designed for computers to analyse as well as for business folk to understand. Indeed, this is it's special power. Other executable specification tools such as FitNesse and Concordion are similar - executable specifications in Concordion are well-crafted documents designed to be both business-readable and computer friendly.
But, despite what you may have read on the interwebs, formal notations are rarely a good way for product owners or BAs to express requirements. Business should be able to read executable specifications easily, but asking them, or even business analysts, to write in this format will just slow things down, potentially compromise team creativity and engagement, and even put people off BDD for good. As we will see, there are much more efficient ways for product owners and business analysts to express their understanding of the requirements.
Formal notations like those used by Cucumber and Concordion can be easy to read (if they are well written), but they take time and care to write well. If we do this as we are discovering the requirements, we are less attentive to what really matters: flushing out assumptions risk and uncertainty, identifying edge cases, and understanding why the business is asking for a particular feature. This deep, collaboratively-build shared understanding is the real goal of a BDD requirements workshop: feature files are merely a way of documenting this understanding.
How BDD can work for you
So it seems, there is more to BDD than meets the eye. How can you make a success of your BDD adoption? Every team is different, but here are a few pointers.
Mindset over mechanics
Behaviour Driven Development is above all a collaboration technique. It is about making the right conversations, with the right people, happen. Get this right, and formalising your acceptance criteria is smooth sailing. Get the formalisation right, and good automation becomes a lot easier. But the order is important.
This is why experienced BDD teams often find it more efficient to use faster facilitated techniques such as Example Mapping and Feature Mapping to get their ideas in order, to flush out uncertainty, and to clarify ambiguity, before they commit to a more formal notation like Cucumber or Concordion. And they do not ask BAs or product owners to write requirements in Gherkin.
Breadth before depth
When we start a project, or when we first learn about, or think of, a feature, there is a lot we don’t know. There are a lot of unchecked assumptions and cognitive biases that can lead us down the wrong path. Some of our initial ideas might turn out to be dead ends, and others might pivot in unexpected directions. During these stages of Discovery and Definition, we want quantity over quality. We don’t want to be slowed down by the need to write formal detailed executable specifications - a few words or a short phrase on a post-it card it just fine.
As we become more confident in what we know, we can start to remove the dead ends, identify uncertainty and decide how to deal with it, and focus on the high value features. At this point, we can start to explore the requirements in more detail.
Embrace the art of automation
Automating acceptance criteria can seem easy at first. After all, it is just "scripting", and "scripting" is meant to be quick and easy. However, teams that take this approach soon run into trouble. The test scripts become unreliable and hard to maintain. Updating test scripts in one place inadvertently breaks other tests. And testers end up spending more and more time maintaining the growing test suites.
Treat your automation code like your production code. Test automation is a development activity, requiring good software engineering skills. It pays to take the time to design quality into your test suite.
Conclusion
So don’t get me wrong. Libraries like Cucumber, SpecFlow, Concordion and the like are awesome, ground-breaking, time-saving tools when used well. They are excellent communication and collaboration tools. But starting your BDD efforts by diving into automation can get you into trouble. The trick is knowing when to automate, what to automate, and then, how to automate.
A special thanks to Jan Molak for reviewing and helping to wordsmith this article.
Additional Reading:
- A day (or a sprint) in the life of a BDD team
- In the “three amigos” band, the BA sets the tune. But it shouldn’t always be that way
- Broad brushes and narrow brushes: there’s more to BDD than Given/When/Then
- Step Away From The Tools
Related Events and Workshops
Low Tech High Impact Sprint Planning
Set your team on the fast track to delivering high quality software that delights the customer with these simple, easy-to-learn sprint planning techniques!
BDD in Action: Mastering Agile Requirements
Focus on the features your customers value the most by building a shared understanding of the real business needs, and empower your teams to deliver more optimal and more reliable solutions sooner!
