Phillip Trelford's Array

POKE 36879,255

TickSpec: Production Ready

The TickSpec F# Behavioural Driven Development (BDD) project started on CodePlex just over a month ago, and is now in use in production for at least one commercial application. TickSpec executes plain text specifications written in the Gherkin business language against a .Net implementation using reflection and regular expressions (like Cucumber does for Ruby). All English language Gherkin keywords are supported including Background and Examples, along with Bullet Points which are TickSpec specific.

The project now includes support and samples for 3 popular .Net Unit Testing Frameworks:

Screen shot of the MbUnit’s Icarus GUI Test Runner:

TickSpecMbUnit

TickSpec gives the ability to step through plain text feature files in Visual Studio and see the current value of template placeholders of the executing Example:

WinningPositions

Debug support is achieved using Reflection.Emit which executes an IL code representation of the source files at runtime. This allows it to be used from any .Net language, and avoids the need to install a Visual Studio plug-in. Plain text specification files can easily be added to a project as an Embedded Resource.

See TickSpec in action in the following video (includes sound):

 

Yet despite all these features, the TickSpec distribution is still lightweight (<100K) comprising a single standalone assembly (TickSpec.dll). And inside there is support for both C# and F# step definitions using either instance or static methods. So that given the following specification:

Given a bullet list of:

  • 1
  • 2
  • 3

Then the following F# step definition will be passed an array of integer values:

let [<Given>] ``a bullet list of`` (xs:int[]) = ()

 

And also for C# given the regular expression is specified as an argument of the attribute:

[Given(@"a bullet list of")]
public void GivenABulletListOfNumbers(int[] xs)
{
}

 

Get started with the binary, source and examples at: http://tickspec.com and for more information on BDD check out the following free fun e-book: http://www.cuke4ninja.com/

TickSpec: An F# BDD Framework

TickSpec is a lightweight Behaviour Driven Development (BDD) framework on CodePlex:

  • “Tick” because it supports F# ``Tick`` methods (*) (as well as C# annotated methods)
  • “Spec” because it parses plain text Specifications
  • “Lightweight” as it is currently implemented in a single F# file with <200 LOC
  • “BDD” as you can describe the software’s behaviour using a subset of the Gherkin language
  • “Framework” as it is simply a library
  • “CodePlex” so that it can be used by the community

(*) ``F# methods`` delimited with double backtick characters

Why BDD?

 

BDD, originally named by Dan North, is an Agile technique that can help deliver real customer value for LOB applications.

Lest we forget, the Agile Manifesto recommends valuing:

Customer collaboration over contract negotiation

By defining executable scenarios with natural language, BDD encourages collaboration between Developers, QA and Business Participants. This differs from TDD, where unit tests are limited to being readable only by developers. More info:

Agile software development also favours:

Working software over comprehensive documentation

By focusing on writing executable scenarios in a natural language, BDD helps build good enough living documentation.

 

Why F#?

 

For the framework, F# features such as pattern matching, particularly active patterns, combined with regular expression support make writing a text parser and state machine pretty easy.

Consumers of the framework use step definitions as methods that are mapped to lines in the specification text file. In C# typically methods are annotated with attributes describing a regular expression to match the line, with the method name repeating the annotation. By escaping methods in F# with double ticks, spaces and regular expression characters can be used directly, avoiding the duplication.

In his F# Fundamentals article for MSDN Magazine, Luke Hoban claims:

F# is in many ways a higher-level language than C#

it means F# developers can often solve problems and think about their programs at a higher level, closer to the domain of the problem at hand.

A higher level language may be more appropriate for writing acceptance tests. Ben Hall writes in his article Automate Acceptance Tests with IronRuby again for MSDN magazine:

I think you can justify the context switching (between IronRuby and C#) in order to take advantage of the readability and the more natural way of writing the verifications and scenarios

In the end, integrating acceptance testing into the development process can be a hugely positive step for a development organization.

I would assert that F# is similar to IronRuby in its expressiveness, but has an advantage in being a first class language within Visual Studio 2010. I would recommend looking at acceptance tests in F# as a way to introduce the language inside an organization. See Zach Bray’s talk on Automating Acceptance Testing with F# at Skills Matter to learn more.

 

Gherkin feature example

 
Feature: Refunded or replaced items should be returned to stock

Scenario 1: Refunded items should be returned to stock
	Given a customer buys a black jumper
	And I have 3 black jumpers left in stock 
	When he returns the jumper for a refund 
	Then I should have 4 black jumpers in stock 

 

F# Step Definitions

 

let [<Given>] ``a customer buys a black jumper`` () = ()
      
let [<Given>] ``I have (.*) black jumpers left in stock`` (n:int) =  
    stockItem <- { stockItem with Count = n }
      
let [<When>] ``he returns the jumper for a refund`` () =  
    stockItem <- { stockItem with Count = stockItem.Count + 1 }
      
let [<Then>] ``I should have (.*) black jumpers in stock`` (n:int) =     
    let passed = (stockItem.Count = n)
    Debug.Assert(passed)

 

Why TickSpec?

 

Right now TickSpec is intended as a lightweight framework to get you started with BDD using F#. It is standards based, supporting a subset of the Gherkin language, so should be easy to change to another Gherkin based framework like Cucumber, SpecFlow or StorEvil.

 

Finally

 

The Agile Manifesto values:

Individuals and interactions over processes and tools

A BDD framework helps automate the execution of scenarios. But don’t forget, the real value in BDD comes from greater customer collaboration.

"In The Brain of Gojko Adzic: What is FitNesse and should I use it?"

Thanks to Gojko Adzic for an excellent introductory talk on Fitnesse: In The Brain of Gojko Adzic: What is FitNesse and should I use it? The slides of the talk are available here: http://gojko.net/resources/whatisfitnesse.pdf.

The value of Fitnesse appears to be in providing a way to quickly and easily generate requirements documentation with examples that can be shared between customers and developers. Documentation is written as Wiki pages that describe requirements with tables of example inputs and outputs. The example tables are parsed to create automatic tests with the tests expecting some support classes called fixtures.

Fitnesse is not suitable for Unit Testing; equally Unit Tests are not typically suitable for reading by customers. In his talk Gojko humorously compared asking customers to read unit test code akin to asking them to read Klingon. 

Opening quotes on Fitnesse:

  • Fitnesse home: “Welcome to FitNesse! The fully integrated standalone wiki, and acceptance testing framework.”
  • Fitnesse from Wikipedia: “FitNesse is a web server, a wiki, and a software testing tool. It is based on Ward Cunningham's Framework for Integrated Test. FitNesse is designed to support acceptance testing rather than unit testing in that it facilitates detailed readable description of system function. 
    FitNesse allows users of a developed system to enter specially formatted input (its format is accessible to non-programmers). This input is interpreted and tests are created automatically. These tests are then executed by the system and output is returned back to the user. The advantage of this approach is very fast feedback from users. The developer of the system to be tested needs to provide some support (classes named "fixtures", conforming to certain conventions).
  • Framework for Integrated Tests (FIT) from Wikipedia: Framework for Integrated Test, or "Fit", is an open-source tool for automated customer tests. It integrates the work of customers, analysts, testers, and developers. 
    Customers provide examples of how their software should work. Those examples are then connected to the software with programmer-written test fixtures and automatically checked for correctness. The customers' examples are formatted in tables and saved as HTML using ordinary business tools such as Microsoft Excel. When Fit checks the document, it creates a copy and colors the tables green, red, and yellow according to whether the software behaved as expected. 

Example of a Fitnesse example: 

Division
numerator denominator quotient?
10 2 5.0
12.6 3 4.2
22 7 ~=3.14
9 3 <5
11 2 4<_<6
100 4 33

 

 Fitnesse examples should declare the What not the How:

  • What: the inputs and outputs
  • How: the control flow 

Or to put it another way Fitnesse examples should be Declarative not Imperative. 

Final quote: FitNesse is written in Java (by Micah Martin, Robert C. Martin and Michael Feathers) but versions for several other languages had been added over time (C++, Python, Ruby, Delphi, C#, etc)”.