Phillip Trelford's Array

Since the 1980s the 8086 architecture has dominated micro-processors and so too has the relational model. The x86 series has papered over the cracks with larger and larger chips adding huge caches and requiring smarter compilers, with the relational model seeing ever larger RDBMSs systems and ORMs.

Even with an ORM like Hibernate in place, to create a working data driven solution is cumbersome. We must define a database schema, along the way explicitly defining the bits and bytes of parent/child relationships, then an XML mapping file and finally plain old objects. As new features are added all of these definitions must be kept in synch.

For say a basic web store we may only require a few tables, say products, categories, orders and customers. But what if you wanted to extend the web store to have features like the online retailer Amazon, e.g. multiple sellers, recommendations, etc.?

Answer: serious table and relationship proliferation.

Enter an alternative model: the Associative model of data, a dynamic model where data is defined simply as items and links:

/// Associative data value    
type Value =
    /// Item value
    | Item of string
    /// Link of source value, verb and target value
    | Link of Value * string * Value

The following is a minimal implementation of an Associative repository using F#:

/// Naive Associative Model implementation
type Repository () =    
    let mutable items = []
    let mutable links = []  
    let invalidOp s = new System.InvalidOperationException(s) |> raise                         
    let obtainItem value =
        let valueOf = function Item v -> v | Link _ -> invalidOp ""
        match items |> List.tryFind (valueOf >> (=) value) with
        | Some item -> item
        | None ->
            let item = Item(value)
            items <- item :: items
            item 
    let createLink (source,verb,target) =
        let link = Link(source,verb,target)
        links <- link :: links
        link    
    let matchLink f = function 
        | Link(s,v,t) as link -> f (s,v,t)
        | Item _ -> invalidOp ""         
    let filterLinks f = links |> List.filter (matchLink f)                            
    let chooseLinks f = links |> List.choose (matchLink f)
    let pickLink f = links |> List.pick (matchLink f)         
    let rec toString = function   
        | Item value -> value
        | Link (s,v,t) -> toString s + " " + v + " " + toString t
    let rec createEntity (source:Value) =        
        { new IEntity with
            member this.Add (verb,target) = 
                createEntity(createLink(source,verb,obtainItem target))            
            member this.Value verb =
                fun (s,v,t) -> if s = source && v = verb then Some(t) else None                
                |> pickLink |> createEntity           
            member this.Links verb =
                filterLinks (fun (s,v,t) -> s = source && v = verb) 
                |> Seq.map createEntity        
            member this.Values' verb = 
                fun (s,v,t) -> if t = source && v = verb then Some(s) else None
                |> chooseLinks |> Seq.map createEntity                                            
            member this.ToString() = toString source            
        }                    
    /// Gets or creates item
    member this.ObtainItem (value:string) = 
        createEntity(obtainItem value)               
/// Encapsulates associative data entity                                 
and IEntity =
    /// Adds link with specified verb and target
    abstract Add : string * string -> IEntity
    /// Returns all links from this entity matching the specified verb
    abstract Links : string -> IEntity seq
    /// Returns first value matching the specified verb
    abstract Value : string -> IEntity    
    /// Returns all values to this entity matching the specified verb
    abstract Values' : string -> IEntity seq    
    /// Returns a string that represents this instance
    abstract ToString : unit -> string

Add some operator overloads to help prettify the code:

// Dynamic lookup operator oveload
let (?) (source:IEntity) (verb:string) = source.Value(verb)
//  Addition operator overload
let (+) (source:IEntity) (verb:string,target:string) = source.Add(verb, target)

Now we can build the flight example from Wikipedia:

let r = Repository()
let flight = r.ObtainItem("Flight BA111")
let trip = 
    flight +
    ("arrives at", "London Heathrow") + 
    ("on","Dec 12") + 
    ("at","10:25")
do  System.Diagnostics.Debug.WriteLine trip

Or a fragment of a web store:

open System.Diagnostics   
   
do  let category1 = "F# Books"
    let product1 = "Functional Programming with examples in F# and C#"
    let item1 = r.ObtainItem(product1)
    item1 + ("author","Tomas Petricek") |> ignore
    item1 + ("sold by","Amazon") + ("price","27.99") |> ignore    
    item1 + ("sold by","Paperback World") + ("price", "25.99") |> ignore
    item1 + ("category", category1) |> ignore
    let product2 = "Expert F#"
    let item2 = r.ObtainItem(product2)
    item2 + ("author", "Don Syme") |> ignore
    item2 + ("sold by","Amazon") + ("price","27.99") |> ignore
    item2 + ("sold by","Hardback World") + ("price","27.99") |> ignore
    item2 + ("category", category1) |> ignore
    let user1 = r.ObtainItem("Phil")
    user1 + ("viewed", product1) |> ignore 
    user1 + ("viewed", product2) |> ignore
        
    let ShowItemInfo (item:IEntity) =
        item.Links("sold by") |> Seq.iter (fun seller ->
            Debug.WriteLine seller
            Debug.WriteLine seller?price
        ) 
    ShowItemInfo item1
    ShowItemInfo item2     
    
    let amazon = r.ObtainItem("Amazon")
    amazon.Values'("sold by") |> Seq.iter Debug.WriteLine  

To serialize the data to XML simply add the following members to the repository:

/// Writes data to specified XmlWriter instance
member this.WriteTo (writer:XmlWriter) =
    let rec traverse = function
        | Item value as item -> 
            writer.WriteStartElement("Item")
            writer.WriteAttributeString("Value", value)                                
            filterLinks (fun (s,_,_) -> s = item) 
            |> Seq.iter traverse 
            writer.WriteEndElement()
        | Link(source,verb,target) as link ->
            writer.WriteStartElement("Link")
            writer.WriteAttributeString("Verb",verb)
            writer.WriteAttributeString("Target",toString target)                
            filterLinks (fun (s,_,_) -> s = link) 
            |> Seq.iter traverse
            writer.WriteEndElement()                                      
    writer.WriteStartElement("Repository")        
    items |> Seq.iter traverse
    writer.WriteEndElement()        
/// Reads data from specified XmlReader instance
member this.ReadFrom (reader:XmlReader) =         
    reader.ReadStartElement("Repository")
    let mutable xs = []
    while reader.Read() do
        match reader.NodeType, reader.Name with
        | XmlNodeType.Element, "Item" ->
            let value = reader.GetAttribute("Value")
            let item = obtainItem(value)
            xs <- item :: xs
        | XmlNodeType.Element, "Link" ->
            let source = xs.Head
            let verb = reader.GetAttribute("Verb")
            let target = reader.GetAttribute("Target")   
            let link = createLink(source,verb,obtainItem target)
            xs <- link :: xs
        | XmlNodeType.EndElement, "Item" 
        | XmlNodeType.EndElement, "Link" ->
            xs <- xs.Tail
        | _ -> ()
    done  

The implementation presented is purely for interest; there are many improvements and optimizations that could be made for a production system.

Finally, a Java implementation of the Associative model exists called Sentences, and is free.

The Edge UG, based in London, is the “Uber Usergroup for all developers and IT Pros working on the Microsoft technology stacks.” This statement doesn’t seem too wide of the mark either; when put to a show of hands most attendees responded yes to experience of both C++ and C#, with plenty also playing with WPF and Silverlight!

I presented a 1 hour talk introducing F# with 4 demos (attached to this post):

• Full-screen WPF POS Checkout sample with Barcode scanner integration (in 100 lines)
• Twitter WPF client script (200 lines)
• Matermind board game in WPF (300 lines)
• Lunar Lander XNA game (>400 lines)

Expect a video of the talk should be on the Edge UG site soon.

Some tweet feedback from the event:

ebrucucen:F# session started with @ptrelford at #edgeuk

johanbarnard: Watching really interesting F# presentation by @ptrelford at #EdgeUG.

ptrelford:Talking about F# again in front of a lot of people

ColinMackay: F# is more fun than I expected. Demos use XNA to create games in F#. #edgeug

raybooysen: Lunar lander in XNA and F#! #edgeug

johnno31: @ptrelford when can we have pizza??

My tweet was delivered from the F# Twiiter client script during the talk!

F_ Introduction_EdgeUG.ppsx (660.16 kb)

FSharpDemos.zip (53.40 kb)

P.S. To see more F# related talks please join the new F# London User Group

F# lets you define your own operators, and like a man with a new hammer hunting for nails :) I’ve found an application of F# custom operators for sorting multiple columns:

let inline (|?) a b = if a = 0 then b else a

Lets start by trying to sort the table position of the current top 5 of English Football’s Premier League (compiled on 2010-01-21):

Lets say we represent the table as an array of tuples:

let table = [|  
        ("Arsenal", 22, 34, 48) 
        ("Chelsea", 21, 34, 48)          
        ("Spurs",   22, 18, 38) 
        ("Man City",21, 12, 38)
        ("Man Utd", 22, 30, 47)
    |] 

If we could just sort on points, and wanted to sort the array in place, we could write:

do  Array.Sort(table, fun (_,_,_,a) (_,_,_,b) -> b - a)   

But as we can see from the table, some of the teams have the same points, so we need to order on points and goal difference. In the case of Arsenal and Chelsea at the top of the table, they not only have the same points, they have the same goal difference. In this situation the custom is to sort alphabetically (lucky for you if your team starts with the letter ‘A’).

The sort function must now apply secondary sorting criteria if the first expression gives zero:

do  Array.Sort(table, fun a b -> 
        let nameA, _, gdA, pointsA = a
        let nameB, _, gdB, pointsB = b
        let points = pointsB - pointsA
        let gd = gdB - gdA
        let names = String.Compare(nameA, nameB)
        if points <> 0 then points
        else if gd <> 0 then gd
        else names
    )  

) 

Now the custom operator introduced at the start of the article really captures it:

do  Array.Sort(table, fun a b -> 
        let nameA, _, gdA, pointsA = a
        let nameB, _, gdB, pointsB = b
        pointsB - pointsA |? gdB - gdA |? String.Compare(nameA, nameB)                
    )  

The beauty of using the custom operator here, say over a function, is operators can be placed between expressions so can be used to compose the sort.

Finally, there are however a couple of other ways to hammer this nail:

• sort by mapping the table entry tuple
• using LINQ OrderBy and ThenBy

Mapping the table entry is easy on the eye but it does require the generation of temporary tuples and a new array:

do  table |> Array.sortBy (fun (name,_,gd,points) -> -points,-gd,name) 

Using LINQ is also pretty easy on the eye but requires the generation of a new sequence:

do  table.OrderByDescending(fun (_,_,_,p) -> p)
        .ThenByDescending(fun (_,_,gd,_) -> gd)
        .ThenBy(fun (name,_,_,_) -> name)

PS If you are looking for a similar pattern in C#, Jon Skeet shows a similar (but more verbose) way of composing sort expressions using the ?? operator in his C# in Depth book.

PPS Tomas Petricek has a nice article with some sample use of custom operators in F#. Coincidentally he has just authored a book with Mr Skeet on Functional Programming.

PPPS For clarity I should point out that I am a Man Utd supporter, so the timing for this article could be seen as somewhat unfortunate.