Phillip Trelford's Array

POKE 36879,255

Cucumber style BDD Step Definitions with F#

This article describes how to define and execute BDD Step Definitions using F# in a similar, perhaps more concise, manner to using Ruby with Cucumber.

Behaviour Driven Development (BDD)

BDD is a relatively new Agile software development approach (circa 2003) that focuses on communication; encouraging collaboration between developers, QA and business participants; to help bridge the Business-IT alignment gap. Through discussion with stakeholders, scenarios in which the software will actually be used are written to build up a clear understanding of the desired behaviour.

Scenarios are specified in plain-text in a structured form composed of steps:

  • Given some context
  • When some event occurs
  • Then I expect some outcome

Developers and Testers use these scenarios as acceptance criteria to drive development. A feature is considered done when the tests pass for all of its scenarios.

Using a BDD framework, scenario specifications can be automated as acceptance tests, thereby creating living documentation that is in sync with the code. These days there is a plethora of Open Source BDD frameworks covering just about every programming language and platform.

Cucumber

Cucumber is probably the best known BDD framework at the moment. It executes plain-text files written in a business-readable domain-specific language (known as Gherkin) as automated tests. Each file describes a feature and its scenarios. During execution steps of a scenario are matched against Step Definitions composed of a regular expression and a function to execute. Cucumber is written in Ruby, and has benefited from Ruby’s light syntax and expressiveness to minimize the ceremony while defining step definitions:

Given /a customer buys a black jumper/ do
end

.Net BDD Frameworks

There are a number of Open Source BDD frameworks for.Net that can parse Gherkin based text specification files including:

Reflection

Typically .Net BDD frameworks take a reflection based approach, marking functions with attributes containing the regular expression to match against:

[Given(@"a customer buys a black jumper")]
public void GivenACustomerBuysABlackJumper()
{
}

 

StorEvil supports Reflection-based pattern matching representing spaces in C# method names as underscores:

public void Given_a_customer_buys_a_black_jumper()   
{ 
}

 

While TickSpec supports F# back-tick method names that can contain white space.

let [<Given>] ``a customer buys a black jumper`` () = ()

 

Note: A reflection based approach requires a compile and run cycle to test a feature. For iterative development in larger applications, Ruby developers may have the advantage, with Ruby being a dynamic language no compilation step is required. Matters can be improved for .Net developers by running individual tests in isolation using tools like Test Driven .Net or ReSharper, however this still does not mitigate the build step.

F#

F# is a powerful .Net language that ships with Visual Studio 2010. Although F# is a statically typed language, using type inference, developers can still benefit from a light and expressive syntax. F# can also execute scripts interactively. While as a first-class .Net programming language F# code interoperates easily with C# and VB.Net.

Using the TickSpec BDD framework (also written in F#) to parse the feature and scenarios; it is possible to specify step definitions concisely in a similar way to Cucumber for Ruby, i.e. without reflection:

Given "a customer buys a black jumper" <| fun []> ()

 

This approach allows step definitions to be executed in F# interactive

F# Interactive

Fragments of F# code can be executed as scripts using an F# interactive session which can be run from:

F# Interactive makes it easier for developers to take a more explorative approach to coding.

Example

 

Lets take the example from the front page of the Cucumber website and convert the code portions to F#:


Feature text file:

Feature 


Ruby step definition:
Calculator Steps 

F# step definition:

Given "I have entered (.*) into the calculator" <| fun [Int n] ->         
    calculator <- Calculator()
    calculator.Push n  

Ruby Calculator class:

Feature

F# Calculator class:

type Calculator () =
    let mutable values = []
    member this.Push(n) = values <- n :: values

Ruby Console output:

pending_thumb2

F# Interactive output:

image_thumb[1]

 

How it works

 

Given, When and Then are defined as F# functions with 2 parameters:

  1. s: a regular expression pattern string
  2. f: a handler function that takes a list of the string values matched

/// Registers specified pattern's handler function
let Given s f = RegisterStep givens s f
/// Registers specified pattern's handler function
let When s f = RegisterStep whens s f
/// Registers specified pattern's handler function
let Then s f = RegisterStep thens s f

 

Function RegisterStep records the pattern and handler in a dictionary:

let private givens, whens, thens = 
    Dictionary<_,_>(), Dictionary<_,_>(), Dictionary<_,_>()

let private RegisterStep 
        (stepDefinitions:Dictionary<_,_>) 
        (pattern:string) 
        (handler:string list -> unit) = 
    stepDefinitions.[pattern] <- handler

 

Feature text files are parsed using the TickSpec parser and scenario steps are tried against the registered pattern and function handlers:

/// Executes specified feature lines writing output to console
let Execute (lines:string[]) =  
    ...
    let feature = TickSpec.FeatureParser.parseFeature lines
    Console.WriteLine feature.Name
    feature.Scenarios 
    |> Seq.iter (fun scenario -> 
        Console.WriteLine scenario.Name
        scenario.Steps |> Seq.iter tryStep
    )

 

Inside the step definitions Active patterns can be used to parse strings:

let (|Int|) = function s -> System.Int32.Parse(s)

 

Allowing easy type conversion of parameters:

Then "the result should be (.*) on the screen" <| fun [Int n] ->
    pending()

 

All the above code is available in the Interactive project example of TickSpec on CodePlex.

Conclusions

 

Businesses are reporting real business value from adopting BDD; in Gojko Adzic’s forthcoming Specification by Example book he examines over 50 case studies of this style of technique. BDD is gaining real traction in the development community with plenty of talks at conferences and user groups on the subject, along with blog articles and books now available.

Hopefully this article has helped demonstrate the applicability of F# for developing BDD Step Definitions on the .Net platform. From experience, I believe that this can be a good way to gain value from introducing F# to an organization.

Regardless of your language preference, given that your business specifications could outlive your current platform and programming language, I’d recommend choosing a framework that is based on the well supported Gherkin business language, so that you are not locked in to a specific technology in the future.

References

 

BDD References:

F# Interactive References:

BDD with ``TickSpec`` talk

The podcast for “BDD with TickSpec and Concurrency with Agents” from the F#unctional Londoners Meetup is now available, many thanks to Skills Matter: http://skillsmatter.com/podcast/scala/tickspec-bdd-for-c-sharp-f-sharp

Podcast content:

  • a quick look at F# in MonoDevelop
  • BDD with TickSpec with examples in Visual Studio using NUnit, MbUnit & Silverlight
  • Tomas Petricek on Concurrency with Agents including Chat and Twitter samples

This talk coincided with the November 2010 stable release of TickSpec, which supports both .Net 2.0+ and Silverlight 4. Plus examples for all major .Net testing frameworks.

All TickSpec examples presented are on CodePlex: http://tickspec.codeplex.com

Thanks to everyone who made it a really fun evening!

FSharpMeet

Picture taken at the Slaughtered Lamb pub courtesy of Ankur Gurha

Tic-tac-toe

Thanks to a recent article by Cameron Taggart it is now possible to Create F# Silverlight Apps from Visual Studio 2010 Shell. This means that you now don’t need to use a C# project to build the XAP file; which previously outside of VS2010 Pro meant resorting to building the XAP with Visual Web Developer 2010 Express and F# libraries in the VS2010 shell. The only issue I’ve found with this solution is that it is not possible to directly add new files to the F# application project, but you can easily add existing files.

From his example I've created a small reference Silverlight solution with a Tic-tac-toe theme:

  • TicTacToe: the Silverlight application
  • Game: a Silverlight library with some game logic
  • TicTacToe.Test: a Silverlight Unit Test project for testing game logic

If you have Silverlight 4 installed you should see a Tic-tac-toe board that you can mark:

The Silverlight Unit Test project contains a small Behavioural Driven Development (BDD) example to test winning positions using TickSpec:

Feature: Winning positions

Scenario: Winning positions
    Given a board layout:
        | 1   | 2   | 3   |
        | <O> | <O> | <X> |
        | <O> |     |     |
        | <X> |     | <X> |
    When a player marks <X> at <row> <col>
    Then <X> wins
    
Examples:
    | row    | col    | 
    | middle | right  |
    | middle | middle |
    | bottom | middle |

Examples:
    | X | O |
    | X | O |
    | O | X |

 

The example above shows that TickSpec will execute all combinations for multiple examples blocks. This makes it possible to also run the tests with X’s swapped for O’s:

image

The steps in the scenario are mapped to attributed F# tick methods:

let [<Given>] ``a board layout:`` (table:Table) =
    table.Rows |> Seq.iteri (fun y row -> 
        row |> Seq.iteri (fun x value -> board.[x,y] <- parseMark value)
    )
   
let [<When>] ``a player marks (X|O) at (top|middle|bottom) (left|middle|right)``
        (mark:string,Row row,Col col) =
    board.[col,row] <- parseMark mark

let [<Then>] ``(X|O) wins`` (mark:string) =
    Game.mark <- parseMark mark |> Option.get
    let line = winningLine()
    Assert.IsTrue(line.IsSome)

With the column and row positions parsed with F# Active Patterns:

let (|Col|) = function 
    | "left" -> 0 | "middle" -> 1 | "right" -> 2
    | s -> invalidCast s

let (|Row|) = function 
    | "top" -> 0 | "middle" -> 1 | "bottom" -> 2 
    | s -> invalidCast s

 

If you are interested in learning more about BDD & TickSpec there is a free F#unctional Londoners Meetup Group evening event on Wed 24th Nov 2010 at Skills Matter:

Teaser for Tomas Petricek’s talk on his Agent’s:

References:

Source code: TicTacToe.zip (334.65 kb)