Serenity BDD Tip: fine-tuning screenshots in your living documentation

One of the distinguishing features of Serenity BDD is its powerful reporting capabilities. If you organise and structure your tests well, Serenity can help you turn your tests into a sort of light-weight functional documentation, describing both what your application does in high level terms, and how users use the application to achieve specific goals.

As an example, consider the following simple Serenity Screenplay test:

@RunWith(SerenityRunner.class)
public class WhenPlanningATrip {

    @Managed
    WebDriver webDriver;

    Actor tracy;

    @Before
    public void setTheStage() {
        tracy = Actor.named("Tracy");
        tracy.can(BrowseTheWeb.with(webDriver));
    }

    @Test
    @WithTag("Priority:High")
    public void booking_a_simple_trip() {

        givenThat(tracy).has(ChosenTo.bookATicket());

        // WHEN
        when(tracy).attemptsTo(
                ViewTheAvailableTickets.from("London Paddington").to("Oxford")
        );

        // THEN
        then(tracy).should(
                seeThat("the departure station", TheOutboundJourneySummary.origin(), is("London Paddington")),
                seeThat("the destination station", TheOutboundJourneySummary.destination(), is("Oxford"))
        );
    }
}

This test will produce a test report like the following:

This kind of detailed report is a great example of living documentation. Testers can use the screenshots to verify that the scenario plays out as they would expect, and product owners get a step-by-step rundown of how a user would interact with the system, including both what they are trying to do ("Tracy chooses to book a ticket", "Tracey views the available tickets from London to Paddington") and how they go about it (clicking on buttons, entering field values and so on).

However the blow-by-blow detail of the screenshots does come at a cost. Each screenshot request causes WebDriver to pause while it takes the screenshot, and when there are a lot of screenshots, this time can add up and slow down the build pipeline quite significantly. And web-based acceptance tests are naturally slow as it is, so we don't need them being slowed down further at the wrong time.

Fortunately, screenshots in Serenity BDD are highly configurable. In Serenity 1.1.38, you can configure screenshots according to the type of activity you are doing. For example, you can distinguish how you take screenshots for high level business tasks (represented by the Task interface in Screenplay, direct interactions with the UI (represented by the Interaction interface), and Questions (assertions) that you ask about the state of the system. For example, the following configuration (placed in the serenity.properties file) would let you record a single screenshot for each task and question, and only record screenshots for interactions if an error occurs:

serenity.take.screenshots.for.tasks=after_each_step
serenity.take.screenshots.for.interactions=for_failure
serenity.take.screenshots.for.questions=after_each_step

This sort of fine-tuning can lighten up the reports considerably without compromising the "living documentation" aspect too much, as illustrated here:

The following figures are by no means scientific, but will give a rough idea of  the sort of numbers we are talking about:

Many teams using Serenity BDD on larger projects use this approach to configure their builds according to their purpose. For example, standard builds will either limit the screenshots to tasks and questions, or set the screenshot capture to FOR_FAILURE across the board, while a separate "living documentation" build (often run nightly) produces the full set of screenshots and acts as a reference description of the current feature set for product owners, BAs and testers.

To learn more about Serenity and Serenity Screenplay, be sure to check out the video tutorials that you can find here.