Gherkin BDD: Python edition

There has been a trend in test automation to move towards behavorial driven development (bdd for short) for front-end ui automation. This post is a tutorial to bootstrapping a bdd ui automation suite.

We will be using the following technology stack:

• Behave - Gherkin BDD framework
• Splinter - Selenium wrapper
• Python 2.7+

First we will grab our example repository and install all the requirements:

Note: It is a good idea to use virtualenv to keep your python project package dependencies isolated. (by good idea I mean you should do it!)

git clone https://github.com/StephenDavidson/python-bdd-behave.git
cd python-bdd-behave/
pip install -r requirements.txt

Let’s run a test to make sure everything is in working order:

behave features/search.feature # to run a specific feature
behave features/ # to run all features

Now that we successfully ran our first test let’s discuss the project structure:

Project_name
├── config # contains all hardcoded strings and settings
|   ├── defaults.yaml # default configuration values to be overriden by the environment
|   ├── production.yaml # production configuration values
|   └── __init__.py # config loader
├── features
|   ├── steps # contains all test steps
|   |   ├── common.py # common steps between features
|   |   └── search.py # feature specific steps
|   ├── environment.py # Setup/Teardown environment. Includes all hooks.
|   └── search.feature # Feature file that contains test scenarios for the search feature
├── lib # contains services, handlers, and other scripts
├── unittests # tests for all lib code
├── .gitignore # list of files to ignore when pushing to github
├── README.md # readme
└── requirements.txt # list of project package requirements

We have four primary directories: config, features, lib, and unittests.

The config folder contains all of the project’s hardcoded strings. These files typically include urls, users, and api keys. I prefer to keep my config files separated by environment because tests are usually isolated by the environment they run on. This in turn cuts down the noise when troubleshooting issues with tests.

The features folder contains all of our front-end ui automation tests. This folder also contains our environment file that helps to setup and teardown tests. The features folder also has the steps directory. The steps directory contains all of the steps that we will use in our feature files.

The lib folder contains all of our locators, pages, services, and handlers for the project. The reason pages and locators end up in the lib folder is because they are independent from the features and do not depend on bdd in any way.

The unittests folder contains all of the tests for our lib files. For this project I use nose to run unittests. Yes, we have unit tests. Developers are not the only ones who should be testing their code!

Since the focus of this post is on BDD we will focus in on the features folder.

The feature folder has three significant components: feature files, environment, and steps.

We will start with environment.py file that is the foundation of all of our tests. This area is where we create and define our environment, which includes our browser, services, logging, and error handling. It is also where we can bring in global packages that will be available in every file. To understand the scope of the environment, it is best to think of it as a global file. Anything that will need to happen outside of tests at a global level should normally happen in the environment file.

Another aspect of the environment file that is extremely useful are hooks. Hooks dictate behavior that is before and after different parts of our test. It allows us to create a browser before each scenario or before each feature for example. It also allows us to tear down test components in after each scenario and after each feature functions. It is important to perform actions in hooks that are independent from test cases. If you need to perform some environment actions for a subset of test cases, you can combine the hooks with tags. You can read more about tags in the behave api docs.

The next layer to look at is the feature. Features follow the behavior driven development style using gherkin syntax.

Feature: Search

  Background: User visits the home page
    Given I visit the home page


  Scenario: User would like to search for a topic
    When I search for test
    Then I should be on the search results page
    And the first search result should be visible

When you view the search.feature file, you will notice that the steps are pretty general and the language is very project managery. I write in the this style for two reasons. In my experience, feature files in an agile environment are most heavily tied to user stories, so writing the feature files in a way project managers are familiar with ensures that the scenarios cover the acceptance criteria written by project managers. The collaboration between quality assurance testers and project managers is essential, since typically you are both the subject matter experts of the product and understand user behavior the best. The reason for the general language is because you need the scenarios to be flexible. As the frontend changes, so do steps to execute tests. This style of general steps prevents us from having to go back and update as many feature files, it also prevents our feature files from becoming to lengthy, and lastly it makes our features more readable.

Now let’s break down the feature file components:

• The Feature is a description of the general area we are testing. In functional testing it can also be considered our test scenario. This is what we are testing.
• The Background is steps performed before each Scenario. The background will setup are test state in many cases.
• The Scenario is a general description of what the user will do. It is equivalent to a test case from functional testing. It is how the user will interact with the product.

• The Steps are our smallest layer. They dictate the steps of our test case. They are separated into Given, When, and Then.

  • Given steps provide test state. Where are we as a user?
  • When steps describe user actions. What does the user do?
  • Then steps describe the user’s expectations. What should the user see? What should happen after the user performs the When actions?

Note: There are more advanced flows such as scenario outlines and data tables that can condense our feature files. To read more visit the behave documentation.

As we saw in the feature file, we use steps to describe user actions, expectations, and state. These steps are defined in step files within the steps folder. There are many ways to organize your steps. My preferred method is to have common steps that perform general actions not linked to a feature and to have feature specific step files that are linked to features in a one to one relationship.

Let’s take a closer look at our common.py file.

For this project we use regex step matchers to define steps. Each step has a python decorator that tells us which step type it is (given, when, then, or step). Notice how the steps in common are very general. I visit X, or I click on the X. This generality allows to use the steps in other more advanced steps, and to have a quick set of modular steps for feature files. You will also notice a context object in many of the steps. This context is global, and is reset between scenarios. The context is primarily used to hold states of pages and users. For example, the I click on step, calls an attribute from the context.page that has been set by the I visit step. We also make use of a page factory. The page factory will generate our page objects for us from the lib/pages folder. We also add helper functions into the common.py file. The helper functions are functions meant to be used in step functions, but are not related to the pages or features.

Thanks to the common.py file, the search.py steps file is pretty bare. We just have one function that makes use of our steps from the common.py file.

That covers the major components of the project structure in relation to BDD. I encourage any readers to clone the project and play around. Also I love feedback so feel free to reach out.