Running Serenity BDD tests with Tags

Tags are a powerful feature of Serenity, making it much easier to read, understand and navigate through the test reports. But tags also allow you to be more selective about what tests you run. This article shows you how.

Tags are a powerful feature of Serenity BDD reporting

There are many times when it comes in handy to run just a subset of our tests. . For instance, you might want to run different sets of tests in parallel, or only run some of the tests to get faster feedback on a particular functional area. In this article, you will learn how to do this with Serenity BDD, using either Cucumber, JUnit or a combination of both.

Serenity tags in Cucumber

If you are only running Cucumber tests, the simplest approach is to use Cucumber's powerful tag options (https://github.com/cucumber/cucumber/wiki/Tags). Cucumber tags can be placed either at the feature or at the scenario level, and use a simple notation. Suppose we want to mark a Feature with two tags, "fast" and "frontend". In Cucumber, we could write the following.

@fast @frontend
Feature: Add new todos

In Serenity, tags (usually) have types. This means it easier to use them for aggregation and sorting in the reports. So, rather than saying @red, we would write @color=red, as shown here:

@color=red
Feature: Add new todos

  @color=blue
  Scenario: Adding a blue and red item to an empty list

  @color=blue @color=green
  Scenario: Adding a red, blue and green item to a list with other items

  Scenario: Adding a red item to a list with other items

Running tagged scenarios

You tell Serenity and Cucumber what tagged tests to run with the cucumber.options system parameter, and the "--tags" option. To run all of the red tests, we could do the following:

  $ mvn clean verify -Dcucumber.options="--tags @color=red"

Alternatively, to run only the green test, we could do:

  $ mvn clean verify -Dcucumber.options="--tags @color=green"

You could even run the tests that are both green and blue by providing the --tags option twice:

mvn clean verify -Dcucumber.options="--tags @color=green --tags @color=blue"

Or to run the tests that are blue but not green, you can use the "~" prefix to indicate what tags you do not want included:

mvn clean verify -Dcucumber.options="--tags ~@color=green --tags @color=blue"

Serenity tagging using JUnit

The cucumber.options option will filter Cucumber tests, but has no effect on non-Cucumber (e.g. JUnit) tests. For JUnit tests, we need to use the -Dtags option. Suppose we have the following test case:

@RunWith(SerenityRunner.CLASS)
@WithTags({
        @WithTag("color:red")
})
public class CompleteAnOrangeTodo {
    @Test
    @WithTag("color:yellow")
    public void should_be_able_to_complete_a_todo() {}

    @Test
    @WithTag("color:blue")
    public void should_see_the_number_of_todos_decrease_when_an_item_is_completed() {}
}

All of these tests have the tag "color:red". So if you wanted to run all of the red tests, you would use:

$ mvn clean verify -Dtags="color:red"

The third test contains the tag "color:blue", at a method level. If you wanted to run only the blue test, you would do this:

$ mvn clean verify -Dtags="color:blue"

Serenity tagging using JUnit and Cucumber

Note that neither of these commands would run any of the Cucumber scenarios we saw earlier, as Cucumber does not understand the -Dtags option. So to get the red tests in both JUnit and Cucumber, we need to provide both the tags parameter and the cucumber.options parameter:

$ mvn clean verify -Dtags="color:red"  -Dcucumber.options="--tags @color=red"

You may have noticed how we use a colon in the tags parameter (-Dtags="color:red"), but an equals sign for the Cucumber options. Cucumber doesn't allow the colon notation normally used in Serenity for tags, so we use the alternative "tag=value" notation using the equals sign. This notation will also work with JUnit if you prefer to be consistent.

These features will work with version 1.4.0 of serenity-core and 1.1.34 of serenity-cucumber.

RELATED EVENTS AND WORKSHOPS

Advanced BDD Test Automation Workshop

Set your team on the fast track to delivering high quality software that delights the customer with these simple, easy-to-learn sprint planning techniques! Learn how to:

• Write more automated tests faster
• Write higher quality automated tests, making them faster, more reliable and easier to maintain
• Increase confidence in your automated tests, and
• Reduce the cost of maintaining your automated test suites

LEARN MORE