Random, Source Code

Tips & Tricks to Improve Your F# Library’s Public API

This post is part of the 2016 F# Advent Calendar.

With the addition of F# in 2010, the .NET run-time gained a terrific, production-grade, functional-first programming language. This post is aimed at anyone who uses that terrific language to develop .NET libraries.

Because, you see, there’s a bit of a problem we need to discuss.

One of the popularized strengths of F# is its ability to both consume and be consumed by other .NET languages. And yet, if not done carefully, consuming F# code from, for instance, Visual Basic can be an absolute nightmare. Fortunately, over the years, I’ve settled upon a few different techniques that can mitigate, and in some cases even obliterate, any unpleasantness. Of course, this article assumes you, as a library author, actually want other developers to have a pleasant experience consuming your work (regardless of the .NET language they employ). If that’s not how you feel, no worries — but you can stop reading now. The rest of this post really won’t appeal to you.

What follows are 12 guidelines, listed in descending order of priority, which each address one, or more, potentially awkward points of integration between F# and other .NET languages. It’s important to note: this advice is intended for the public API of your libraries. By all means, do whatever awesome, clever things you’d like internally. This list is just to help give it a pretty face.

1. Do limit the use of advanced generic constraints

F# supports a wider range of possible generic type constraints than either C# or VB. Not matter how useful, or cool, a constraint might seem (sorry, SRTP fans), it’s meaningless if a consumer can’t possibly comply with it. To that end, public APIs should only leverage the following types of constraints:

Constraint Syntax
Subtype type-parameter :> type
Constructor type-parameter : ( new : unit -> 'a )
Value Type type-parameter : struct
Reference Type type-parameter : not struct

2. Do expose TPL rather than Async workflows

Asynchronous programming in F#, via Async workflows, is a simply unintelligible mess in other languages. Fortunately, it’s quite easy to integrate System.Threading.Tasks.Task into a workflow. Tasks received as input can be sequenced via Async.AwaitTask:

let runAll tasks = 
  let rec loop agg work = async {
    match work with 
    | [  ] -> return List.rev agg
    | h::t -> let! v = Async.AwaitTask h
              let  r = v * 2
              return! loop (r::agg) t } 
  tasks |> loop []

Meanwhile, if you need to return an async workflow to a non-F# caller, you can leverage Async.StartAsTask:

let getUserPrefs conn uid =  
  async { use db = new DbUsers() 
          let! prefs = db.ExecuteAsync(conn,uid) 
          return marshal prefs }
  |> Async.StartAsTask

3. Prefer BCL collection types

F# ships with a small number of persistent functional collections. These are the bread and butter of functional programming. But they’re cumbersome and confusing in other languages. So, for input parameters and return values, consider converting to or from common collection types. For example, when working with List or Set:

(* NOT RECOMMENDED *)
let transform (value :int list) =
  // do stuff with values...

(* DO THIS INSTEAD *)
let transform (values :int seq) =
  let values' = Seq.toList values
  // do stuff with values' ...

Similarly, when working with Map:

(* NOT RECOMMENDED *)
let transform (table :Map<string,int>) =
  // do stuff with table ...

(* DO THIS INSTEAD *)
let transform (table :IDictionary<string,int>) =
  let table' = 
    table 
    |> Seq.map (function KeyValue (k,v) -> (k,v)) 
    |> Map
  // do stuff with table' ...

Note, in the previous samples, type signatures are for demonstration purposes only. Also, note that similar conversions (e.g. Set.toSeq, et cetera) can, and should, be used for return values.

4. Do provide conversions from standard delegates to F# functions

Owning to a number of very good technical and historical reasons, F# uses a different first-class function mechanism than other languages. And, while the F# compiler makes it pretty easy to turn (fun x -> x * 2) into a Func, the inverse is not so easy. So, it becomes important to provide some means of supporting the standard BCL delegates Func and Action (which is what C# and VB use for their first-class functions). This can take several different formats. For instance, we can give the caller the ability to handle converting from a common delegate to an F# function. If we define a utility like:

[<Qualified>]
type Fun =
  static member Of(act :Action<'T>) = (fun t -> act.Invoke t)

Then a VB consumer might use:

Dim opt = BizLogic.ImportantCalc(42)
If Option.IsSome(opt) Then
  Option.Iterate(Fun.Of(PrintOption), opt)

However, often I will provide an extension method which handles the conversion internally, saving the consumer a bit of work. For example, a method like:

  [<Extension>]
  static member IfSome(option, act :Action<'T>) =
    option |> Option.iter (Fun.Of withSome)

Would turn the previous consumer example into something a bit simpler:

Dim opt = BizLogic.ImportantCalc(42)
opt.IfSome(PrintOption)

5. Do emulate matching with higher-order functions

While C# and VB do not support the rich pattern matching enjoyed in F#, we can still leverage higher-order functions to approximate an expression-oriented API. This technique is especially effective with discriminated unions, as seen here:

[<Extension>]
static member Match(option, withSome :Func<'T,'R>, withNone :Func<'R>) =
  match option with
  | Some value  -> (Fun.Of withSome) value
  | None        -> (Fun.Of withNone) ()

Given the above definition, consuming C# code might look like this:

return ValidationSvc.Validate(input).Match(
  withSome: v  => v.ToString(),
  withNone: () => "<NOT SET>"
);

6. Prefer overloaded methods to optional parameters

One of the recurring themes of this post is: F# does things differently. And optional parameters are no exception. In F# they are based on the Option type. However, in other languages they are handled via dedicated syntax. Due to this mismatch, F# “optional” arguments are, in fact, required in a call from C# or VB. In fairness, you can achieve the non-F# behavior in F# by careful use of attributes. However, I generally find it easier to explicitly define overloads, which vary in the number of parameters, and thus give callers the effect of having optional arguments.

ASIDE: a more in-depth look at this topic, by Mauricio Scheffer, may be found here.

7. Do remember to properly export extension methods

I can’t stress this one enough. In order to properly comply with published specifications — and to support Visual Basic.NET even recognizing an extension method — F# code should decorate each method being provided with [<Extension>]. Additionally, any class which contains any such decorated methods should also be decorated with [<Extension>]. Finally, somewhere — anywhere — in a library which provides extension methods, you need to add an assembly-level attribute of the form [<Extension>].

ASIDE: for a more detailed explanation of this mechanism please see this blog post, by Lincoln Atkinson.

8. Do use classes to provide extension methods

F# actually offers two different ways to define extension methods. The first approach is to decorate a module and one, or more, functions inside that module.

[<Extension>]
module Socket =
  [<Extension>]
  let Send (socket :Socket) (frame :byte[]) =
    // ...

  [<Extension>]
  let SendAll (socket :Socket) (frame :byte[][]) =
    // ...

But, as an alternative, you can decorate a class and one, or more, static methods defined on that class. This second approach, besides more closely mirroring the consumer’s mental model, offers a slight advantage: it greatly simplifies the process of providing method overloads. You simply list the separate methods, and implement them as normal.

[<Extension>]
type Socket =
  [<Extension>]
  static member Send(socket :Socket, frame :byte[]) =
    // ...

  [<Extension>]
  static member Send(socket :Socket, frames :byte[][]) =
    // ...

With the first approach, because each function needs a unique name within the module, you must leverage the CompiledNameAttribute to “fake out” method overloads (note: see the next tip for more details).

9. Do make judicious use of CompiledNameAttribute

The CompiledNameAttribute, like much of F#, is a double-edged sword. It can be used to great effect, especially when trying to support C# or VB. But it can also lead to ruin, increasing complexity and confusion for no real benefit. Use with caution. The concern, you see, is that by using this attribute, you cause a construct to have two different names. One will be used by/visible to F#. While the other will be used by/visible to other languages and reflective meta-programming. However, this is sometimes exactly what’s needed. For example, while it often makes sense to collect all of your “language interop” extension methods into a single, dedicated class. For very simple functions, requiring no additional manipulation, it may make sense to avoid the extra layer of indirection. For example, this code:

[<Extension; CompiledName "Send">]
let sendAll (socket :Socket) (msg :byte[][]) =
  // ...

Would be consumed from F# as:

msg |> Socket.sendAll pub

And equally from VB as:

pub.Send(msg)

Another time where CompiledNameAttribute can be helpful in sorting out naming conflicts is when types and modules need to have similar names:

type Result<'T,'Error>

[<Qualified; Module(Suffix)>]
module Result =
  // ...

[<Extension; CompiledName "Result")>]
type ResultExtensions =
  // ...

As this example demonstrates, we can partition the functionality around an abstract data type. We can put all the F#-facing components into a module. Then provide the C#-facing equivalents in a class for of static methods. Adding CompiledName to the mix ensures a clean, per-language experince.

In F#:

// invoking a function on the Result module
Result.tryCatch (fun () -> getUserFromDb conn) 

And in C#:

// invoking a static method on the ResultExtensions class
Result.TryCatch(() => { return getUserFromDb(conn); });

10. Prefer records or unions over tuples or primitives

Folks might shy away from exposing F#-centric types like records or discriminated unions to other languages. However, especially if heading the previous guidelines, there’s no reason not to share these powerful constructs with C# and VB. In particular, using types like Option (rather than “null checking” or using a Nullable) can greatly improve the overall robustness of an API. Consider this example:

let postRequest body :bool =
  // ...

It can only tell a consumer “Yay!” or “Nay!”. Meanwhile, this example gives calls a much richer experience:

let postRequest body :Result<bool,exn> =
  // ...

ASIDE: for more details about this approach to error handling, please see the excellent work presented here, by Scott Wlaschin.

11. Do expose a LInQ provider (if appropriate)

While not appropriate for all domains, exposing F# functionality via Language-Integrated Query (hereafter, LInQ) can be an excellent bit of “sugar” for C# and VB. Consider this example (leveraging techniques discussed earlier), which tries to combine multiple Result instances:

var vsn = major.Match(
    m => minor.Match(
        n => revision.Match(
            r => string.Format($"{m}.{n}.{r}"),
            e => e.Message),
        e => e.Message),
    e => e.Message);

WriteLine(vsn);

Now look at the improvements LInQ offers:

var vsn = from m in major
          from n in minor
          from r in revision
          select string.Format($"{m}.{n}.{r}");

vsn.IfOK(WriteLine); 

It’s true that providing LInQ support will mean defining 3, or more, extension methods against a type. But it’s usually worth the effort.

12. Do NOT rely on type aliases

Type aliases can dramatically improve the intensionality of your F# code. Unfortunately, they are fully erased at compile time (i.e. stored in special metadata only read by other F# libraries). So a C# or VB consumer won’t see your well-intended, self-documenting type alias… just the raw type underneath.

Advertisements
Community, Source Code

A Mixed-Paradigm Recipe for Exposing Native Code

(Note: this post assumes some familiarity with either .NET or Mono… it’s also going to help if you’ve worked with C#, VB, or F# before.)


F# is frequently called a “functional first” programming language. Don Syme, creator of the language, has explained it thus:

Functional-first programming uses functional programming as the initial paradigm for most purposes, but employs other techniques such as objects and state as necessary.

However, the simplicity of this statement belies the tremendous power and flexibility of the language. This is seldom more apparent than when trying to wrap unmanaged libraries in F# code. In fact, we may combine two different approaches — one common to OO languages and the other popularized by pure functional programming — into a sort of recipe for wrapping native functionality in F#. Specifically, we’ll bring together deterministic resource management[1][2] with the notion of abstract data types[3][4]. As a case study for exploring this, we’ll look at the fszmq project.


Sidebar: What is fszmq?

fszmq is an MPLv2-licensed CLR (e.g. Mono, .NET) binding to the ZeroMQ distributed computing library. ZeroMQ, meanwhile, provides a complete library of building blocks for developing high-performance, message-passing systems.

fszmq is primarily concerned with Sockets which pass stateless Messages to one another. These messages are comprised of 1 or more frames of 0 or more bytes. fszmq makes no demands on the actual representation of message data. Sockets exchange messages in well-defined patterns which provide proven semantics on which to build distributed systems. Additionally, sockets provide (inaccessible to application code) inbound and outbound in-memory message queues. This makes centralization optional rather than mandatory. Sockets also provide a uniform abstraction over various transport protocols, the most popular of which are In-Process (i.e. threads), IPC, TPC, and PGM. Finally, a Context groups together a collection of sockets into a logically distinct “node”. There is typically one context instance per OS-level process.

A simple example of a server, which receives updates from a client, and then replies with an acknowledgement might look as follows:

// create, configure Context, Socket instances
use context = new Context ()
use server  = router context
Socket.bind server "tcp://eth0:5555"

while not hook.IsCancellationRequested do
  let msg    = Socket.recvAll server
  let sender = Array.get msg 0
  // actual work would go here
  [| sender; 0x00uy |] |> Socket.sendAll server 

For more information on getting started with fszmq and ZeroMQ please visit:

And now, back to the main feature…


F# code is subject to garbage collection, just like any other CLR language. This poses particular issues when working with unmanaged resources, which — by definition — are outside the purview of a garbage collector. However, we can take two explicit steps to help manage this. First, we define a type whose (ideally non-public) constructor initializes a handle to unmanaged memory:

type Socket internal (context,socketType) =
  let mutable disposed  = false // used for clean-up
  let mutable handle    = C.zmq_socket (context,socketType)
  //NOTE: in fszmq, unmanaged function calls are prefixed with 'C.'
  do if handle = 0n then ZMQ.error ()

Then, we both override object finalization (inherited from System.Object) and we implement the IDisposable interface, which allows us to control when clean-up happens:

  override __.Finalize () =
    if not disposed then
      disposed <- true // ensure clean-up only happens once
      let okay = C.zmq_close handle
      handle <- 0n
      assert (okay = 0)

  interface IDisposable with

    member self.Dispose () =
      self.Finalize ()
      GC.SuppressFinalize self // ensure clean-up only happens once

With our creation and destruction in place, we’ve made a (useless, but quite safe) managed type, which serves as an opaque proxy to the unmanaged code with which we’d like to work. However, as we’ve defined no public properties or methods, there’s no way to interact with instances of this type.

And now abstract data types enter into the scene.

Ignoring the bits which pertain to unmanaged memory, our opaque proxy sounds an awful lot like this passage about abstract data types:

[An ADT] is defined as an opaque type along with a corresponding set of operations… [we have] functions that work on the type, but we are not allowed to see “inside” the type itself.

This would exactly describe our situation… if only we had some functions which could manipulate our proxy. Let’s make some!

For the sake of navigability, we group the functions into a module with the same name as the type they manipulate. And the implementations themselves mostly invoke unmanaged functions passing the internal state of our opaque proxy.

module Socket =

  let trySend (socket:Socket) (frame:byte[]) flags =
    match C.zmq_send(socket.Handle,frame,unativeint frame.Length,flags) with
    | Message.Okay -> true
    | Message.Busy -> false
    | Message.Fail -> ZMQ.error()

  let send socket frame = 
    Message.waitForOkay (fun () -> trySend socket frame ZMQ.WAIT)

  let sendMore socket frame : Socket =
    Message.waitForOkay (fun () -> trySend socket frame (ZMQ.WAIT ||| ZMQ.SNDMORE))
    socket

  //NOTE: additional functions elided, though they follow the same pattern

And that’s primarily all there is to this little “recipe”. We can see from the following simple example how our opaque proxy instances are a sort of token which provides scope as it is passed through various functions calls.

// create our opaque Socket instance
use client = dealer context 
//NOTE: the `use` keyword ensures `.Dispose()` is called automatically

// configure opaque proxy
Socket.connect client "tcp://eth0:5555"

// ... elsewhere ...
// send a message
date.Stamp () |> Socket.send client

// recv (and log) a message
client 
|> Socket.tryPollInput 500<ms> // timeout
|> Option.iter logAcknowledgement

Now, we could stop here. However, this clean and useful F# code will feel a bit clumsy when used from C#. Specifically, in C# one tends to invoke methods on objects. Also, the tendency is for PascalCase when naming public methods. Fortunately — as an added bonus — we can accommodate C# with only minor decoration to our earlier code. We’ll first add an ExtensionAttribute to our module. This tells various parts of the CLR to find extension methods inside this module.

[<Extension>]
module Socket =

And then we add two attributes to each public function. The ExtensionAttribute allows our function to appear as a method on the opaque proxy (when used from C#). Meanwhile, the CompiledNameAttribute ensures that C# developers will be presented with the naming pattern they expect. Calling the code from F# remains unaltered.

  [<Extension;CompiledName("SendMore")>]
  let sendMore socket frame : Socket =
    Message.waitForOkay (fun () -> trySend socket frame (ZMQ.WAIT ||| ZMQ.SNDMORE))
    socket

  //NOTE: additional functions elided, though they follow the same pattern

Now C# developers will find it quite straight-forward to use the code… and we’ve maintained all the benefits of both deterministic resource management and abstract data types.

// create our opaque Socket instance
//NOTE: the `using` keyword ensures `.Dispose()` is called automatically
using(var client = context.Dealer())
{
  // configure opaque proxy
  client.Connect("tcp://eth0:5555");
 
  // ... elsewhere ...
  // send a message
  client.Send(date.Stamp());

  // recv (and log) a message
  var msg = new byte[0];
  if(client.TryGetInput(500,out msg)) logger.Log(msg);
}

By combining useful techniques from a few different “styles” of programming, and exploiting the rich, multi-paradigm capabilities of F#, we are able to provide simple, robust wrappers over native code.


TL;DR…

A Mixed-Paradigm Recipe for Exposing Native Code

  1. Make a managed type with no public members, which proxies an unmanaged object
    • Initialize native objects in the type’s constructor
    • Clean-up native objects in the type’s finalizer
    • Expose the finalizer via the IDisposable interface
  2. Use the abstract data type pattern to provide an API
    • Define functions which have “privileged” access to the native objects inside the opaque type from step #1
    • Group said functions into a module named after the opaque type from step #1

Bonus: make the ADT friendly for C# consumers

  • Use ExtensionAttribute to make ADT functions seem like method calls
  • Use CompiledNameAttribute to respect established naming conventions

(This post is part of the 2015 F# Advent.)