Cucumber Tip: Dry Run

Working with Cucumber, you run the scenarios all the time. Typically, you care about the result of a set of scenarios, so you want Cucumber to execute each matched scenario step-by-step and report results of this execution.

However, there are a few tasks that come up quite regularly where actually executing the step definitions is not necessary. One example: you'd like to find all scenarios with a particular tag. Second example: you'd like to find location of a step definition that matches a particular step. Yet another example: you'd like to verify that a step definition's regular expression matches the text in the scenario you're working on.

When working on such tasks, you can save a lot of time by using the dryRun option in the Cucumber runner. It instructs Cucumber to find the scenarios to run by matching the given tags, just like it does during normal run, but then skip each step, and pass this step info to formatters, again, just like in a normal run.

Using Cucumber-JVM, you can get Cucumber to do a dry run by setting the dryRun parameter in CucumberOptions annotation, like this:

import org.junit.runner.RunWith;

import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

/**
 * Executes only a dry run of released scenarios!
 *
 * @author Sanjin Tulac
 */

@RunWith(Cucumber.class)
@CucumberOptions(
    dryRun = true,
    tags = { "@released" }, 
    monochrome = true, 
    format = { "pretty" },
    glue = { "com.tulac.stepDefs" },
    features = { "src/main/resources/com/tulac/features" })

public class DryRunOfReleasedScenarios {}

In this example, I'm using "pretty" formatter to list all the matched scenarios and their step definitions, including their location. This can be very useful in case your IDE does not support finding matching step defs on its own.

To give you sense of the output without overly complicating with too many details, here's a partially shortened example of a dry run output for a simple secondary scenario:

Feature: ... (the rest of feature name is listed here)
  
  This feature ... (the rest of feature description is listed here)

  Background:                                 # campaignApis.feature:6
    Given subscription with ReST APIs enabled # FeatureStepDefs.subscription_with_ReST_APIs_enabled()

  @released ... (the rest of scenario tags are listed here)
  Scenario: asking a trigger campaign to process more than 100 leads results in a 1003 # campaignApis.feature:27
    Given a brand new global static list "101 leads"                                   # ListStepDefs.a_brand_new_global_static_list(String)
    And 101 leads in the list "101 leads"                                              # ImportLeadStepDefs.leads_are_imported_into_list(Integer,String)
    Then deleting leads in list "101 leads" via a trigger campaign fails with "1003"   # CampaignStepDefs.deleting_leads_in_list_via_a_trigger_campaign_fails_with(String,String)

1 Scenarios (1 skipped)
4 Steps (4 skipped)
0m0.000s

Notice that both background and scenario steps have comments (starting with '#') in which the file and function name of a matching step def is listed. Similarly, feature file that contains the matching scenario and the line number in which it begins are listed as comment for each scenario and associated background.

In conclusion, using dryRun option when running Cucumber scenarios can save you a lot of time by skipping all matched step defs, while still producing specified reports, so remember to use it when appropriate!