KC Sivaramakrishnan - Scalable Functional Programming

Shrinking the OxCaml js_of_ocaml bundle: 285 MB to 4 MB

2026-05-10T11:00:00+00:00

In the previous post on capsules, I cheated. The lecture I was adapting (from my CS6868 course on language abstractions for parallelism) used Await_capsule.Mutex.with_lock, the recommended non-deprecated way to acquire a capsule mutex, but the post shipped Capsule_blocking_sync.Mutex instead with the deprecation alert silenced. The reason was bundle size: the await library, once we chased its transitive dependencies through base, sexplib0, base_quickcheck and the rest of Jane Street’s runtime, would have ballooned the in-browser toplevel by roughly 285 MB. The right API would not even fit through GitHub’s 100 MB per-file push limit, let alone be reasonable to send to a reader’s browser.

This post is the story of how we got from 285 MB down to 4 MB and made the resulting bundle compose cleanly with the in-browser toplevel, so the lecture’s Await_capsule form works end-to-end in the cell at the bottom of this post. Most of the work happened on a branch of ocsigen/js_of_ocaml, with a smaller piece in art-w/x-ocaml, the WebComponent that powers the cells.

Why bundle size matters

I teach two OCaml-heavy courses at IIT Madras: CS3100, the undergraduate functional programming course, and CS6868, the more recent graduate course on language abstractions for parallelism. The lecture notes, examples and homework for both would be much more useful as interactive books that a student can read, edit and run entirely client-side, with no local installation. The same shape would help us when we run hands-on OCaml and OxCaml workshops, where the first session routinely gets eaten by the installation hump: getting opam, the compiler and the required libraries working on every attendee’s machine over patchy conference WiFi, before the teaching can begin.

The broader effort to make installation painless is the OCaml Platform roadmap, which we have been working on at Tarides as a “zero to OCaml in one click” experience. That roadmap targets a developer who wants a real local toolchain, with the full editor, debugger and project-management story, and a generous latency budget since this is a one-time setup. A workshop attendee has a much narrower target: just enough OCaml to complete the exercises in front of them. The client-side x-ocaml toplevel fits that target naturally, because everything ships as static assets and there is no installation step. The bundle, in this setting, is the latency budget: 285 MB makes the in-browser path unshippable, 4 MB makes it a realistic alternative to a local toolchain for a 90-minute session.

Why 285 MB?

The recipe x-ocaml already had for “load extra libraries into a running in-browser toplevel” goes like this. For each cma you want to ship, run

$ js_of_ocaml --toplevel <library>.cma -o lib.js

then concatenate the per-cma outputs into a single bundle and load it via <script src-load=...>. Each per-cma output is kind=cma: it registers the cma’s modules into the existing toplevel without clobbering anything, the modules light up, and you can open them from a cell. This works, and it is what the previous two OxCaml posts already use.

The trouble is that dead code elimination runs one cma at a time. If you ship base, you ship all of base, because the per-cma DCE pass never gets to look at the linked-together program and notice that the await library you actually want only touches a small slice of sexplib0. So the bundle for the closure await + capsule + basement ends up being the union of every cma in the transitive dependency, fully linked, which comes out to roughly 285 MB after the OxCaml compiler’s normal optimisations and before any of the JavaScript-side cleverness has had a chance to run.

In other words, the await-based blessed API is unshippable not because the bundling tooling is broken, but because per-cma DCE is the wrong granularity for this problem.

The other recipe, and why it doesn’t compose

It turns out js_of_ocaml already has a second recipe that does perform cross-cma DCE, which I learned about when Ricky Vetter pointed me at the --export route on X. The recipe has two steps:

Build a single bytecode that links every library you want, with -linkall so nothing gets pruned at the bytecode level.
Hand that single bytecode to js_of_ocaml --toplevel --export units.txt. The export list names the compilation units that should remain visible to the toplevel; everything else is fair game for DCE on the unified intermediate representation.

For the same await + capsule + basement set, this recipe produces a 4 MB bundle, almost two orders of magnitude smaller than the per-cma concatenation. The link-step DCE works at function granularity over the whole linked program, so the unused parts of sexplib, base, base_quickcheck and the rest of the dependency closure get pruned away cleanly.

So why aren’t we already using this recipe?

Because the artifact that comes out the other end is kind=exe, that is, a self-contained executable rather than a library. When you load such a bundle into a Web Worker that already has a running toplevel, its initialisation runs in caml_main style and starts by overwriting the host’s globals:

caml_global_data.symbols    = <bundle's symbol table>
caml_global_data.sections   = <bundle's bytecode sections>
caml_global_data.prim_count = <bundle's primitive count>
caml_global_data.aliases    = <bundle's alias table>

Those four assignments overwrite the host toplevel’s tables. After the bundle loads, caml_get_global_data().symbols is the bundle’s symbols, not the worker’s, and anything in the host toplevel that does name-based symbol resolution (Toploop, hover-types lookup, open Stdlib) now consults a table that does not know about the modules the worker had already linked. The toplevel survives, but its typing environment is wrong, and cells stop being able to open anything. We hit this dead end in the capsules post; the bundle was correct and the sizes were great, but the integration step that connects an exe-shaped bundle to an existing toplevel was the missing piece.

The patch

The fix is a flag, --toplevel-extend, that I added on a branch of ocsigen/js_of_ocaml. When it is set, js_of_ocaml --toplevel --export … emits exactly the same DCE output as before, with the same bytes for the registered modules and the same .cmi files embedded under /static/cmis/, but with three small changes:

packed as ~standalone:false, so there is no globalThis polyfill IIFE wrapping the output,
with the four caml_js_set writes to caml_global_data from above skipped, and
tagged as kind=cma in the buildInfo header.

The bundle’s modules still register themselves on load via the ordinary caml_register_global(n, v, name), which the runtime correctly merges via symidx into the host’s existing tables. The result is additive, not destructive: the host toplevel’s symbol table, sections, primitives and aliases all survive intact, and its typing environment continues to resolve Stdlib, Toploop and everything else that was already linked. The new modules from the bundle simply show up as new symbols on top.

The initial diff is small: parse_bytecode.ml gates the caml_js_set block on the new flag, and cmd_arg.ml/compile.ml thread it through. That gets the bundle composable. The actual debugging is what came afterwards, when we tried to make the composed bundle behave the way the host expected, and that is what the rest of this post is about.

Wiring it through `x-ocaml`

On the x-ocaml side, the --dce flag drives the single-bytecode + --export build, invoking js_of_ocaml --toplevel-extend --export units.txt for the bundles we ship. The oxcaml branch of my x-ocaml fork carries the change; the patched js_of_ocaml needs to be on PATH when the build runs.

Numbers

For the full closure of libraries the lecture’s gensym example uses, the two paths come out very differently:

	Per-cma	`--dce --toplevel-extend`	Ratio
basement + capsule0 only	1.0 MB	1.0 MB	1×
+ capsule + await + portable	285 MB	4.0 MB	~70×

The basement + capsule0 row is essentially a wash, because the bundle size at that scale is dominated by the .cmi files and there is very little JavaScript code for DCE to chew on. Once await and the curated capsule API enter the picture the per-cma path balloons, because it has to ship every cma in the transitive closure of base, sexplib0, bin_prot, base_quickcheck and the ppx_* runtime libraries, while --dce keeps only the functions that are actually reachable from the export list, plus the .cmi files the toplevel needs to elaborate the signatures the user types into a cell.

A few more snags

Getting the bundle to be kind=cma was the easy part. Composing it with an already-running toplevel turned out to surface a small zoo of follow-on issues, each of which had a short fix once we understood it. They all live on the same kc-toplevel-extend branch; the commit messages have the gory details.

Predefined-exception identity drifts across bundles. The bundle’s re-allocated Not_found, Sys_error and friends are physically distinct from the host’s copies, so a try ... with Not_found -> in stdlib code (the first place we hit this was Hashtbl.randomized_default reading OCAMLRUNPARAM) fails to catch the host’s Not_found raised by the runtime. The fix is to bind each predef-exn variable in the bundle to a runtime caml_get_global_data lookup so the bundle reuses the host’s instances.
The pseudo-filesystem raises on duplicate .cmi registrations. The bundle wants to re-emit /static/cmis/stdlib.cmi, which the host has already registered at boot, and MlFakeDevice.register refuses to overwrite. Making register idempotent removes the conflict without losing anything: the two copies of stdlib.cmi agree, since they come from the same opam switch.
Stdlib re-registration overwrites host modules. Without a guard, caml_register_global was cheerfully replacing the host’s caml_global_data["Format"] (and every other stdlib module the bundle’s bytecode happens to link in) with the bundle’s freshly initialised copy. Adding an early return when the name is already known fixes this without changing the behaviour for any name the host does not yet have.
Domain.DLS slot collisions silently broke hover types. The bundle re-runs stdlib’s module init when it loads, and Stdlib__Domain.DLS’s let key_counter = Atomic.make 0 re-allocates the counter and starts it from zero. The bundle’s Format.stdbuf_key then ends up at a low DLS index the host had already assigned to its Format.stdbuf_key, and DLS.set overwrites the host’s entry in the shared caml_domain_dls array. The host’s Format.flush_str_formatter then reads from the bundle’s empty buffer, merlin’s type-enclosing printer (which flushes through Format.str_formatter) returns "" for every query, and hover-on-identifier tooltips come up blank. The fix is at the bundle-load boundary in build_portable_js_extend.sh: snapshot caml_domain_dls_get () before the bundle’s IIFE runs, and restore the host-owned slots afterwards. The bundle’s new high-index slots are left alone; only the host’s previously populated slots get restored. I spent a while convinced this was a Format DCE bug before tracing through the OCaml 5+ Domain.DLS init path.
Bundle build: the curated capsule and await.capsule APIs both open! Base at the top of their files, so their interfaces mention Base.unit rather than Stdlib.unit. To elaborate those signatures the host toplevel needs a small chain of base and sexplib0 .cmi files at /static/cmis/. We ship three base cmis (base.cmi, base__.cmi, base__Unit.cmi) and two sexplib0 cmis via js_of_ocaml’s --file=<src>:/static/cmis/ flag, which embeds the file directly without putting base on the bytecode-link line (which would drag the whole of base back in).

The cell

The cell below uses the lecture’s gensym_capsule.ml shape directly: Await_capsule.Mutex.with_lock taking an Await_kernel.Await.t witness, with Capsule_expert.Data.create and Capsule_expert.Data.unwrap for the brand-locked counter. No [@@@alert "-deprecated"], no shim. Press Run.

let make_gensym () = let (P mutex) = Await_capsule.Mutex.create () in let counter = Capsule_expert.Data.create (fun () -> ref 0) in let fetch_and_incr (w : Await_kernel.Await.t) = Await_capsule.Mutex.with_lock w mutex ~f:(fun access -> let c = Capsule_expert.Data.unwrap ~access counter in incr c; !c) in fun w prefix -> prefix ^ "_" ^ string_of_int (fetch_and_incr w) let gensym = make_gensym () let w = Await_blocking.await Await_kernel.Terminator.never let () = Printf.printf "%s %s\n" (gensym w "x") (gensym w "y")

We cannot actually demonstrate parallelism in the browser worker (it is single-domain), so we hand the body the trivial blocker Await_blocking.await Terminator.never. The mode-checking is the same brand/local/once dance as in the capsules post; what is new is that the same dance now type-checks against the blessed Await_capsule API directly, without the Capsule_blocking_sync shim.

What next?

The fork is small enough that --toplevel-extend is plausibly upstreamable into ocsigen/js_of_ocaml. The DLS-snapshot dance would also be cleaner as a proper runtime primitive than as a wrapper around the bundle’s IIFE. The branch is open for review.

For x-ocaml itself, the obvious next step is to broaden the set of extension bundles. The current portable.js covers basement + capsule0 + capsule + await + portable, which is just enough for the capsule and parallelism material in CS6868; the CS3100 material would want a different slice, and an async/Eio-flavoured bundle would let the same cells host concurrency examples. Once a small library of these bundles exists, turning a lecture set or a workshop tutorial into an interactive book becomes mostly a matter of picking the right bundle, which is the workshop-scale “zero to OxCaml” story I want to get to.

x-ocaml is one of Arthur Wendling’s hacking expeditions, and it remains a pleasure to build on. Thanks to Ricky Vetter for the --export pointer that got the whole thing started, and to the OxCaml team for the libraries.

Written together with Claude Opus 4.7 (1M context). The js_of_ocaml diff against the +ox base and the x-ocaml integration diff are both on GitHub.

Capsules: compile-time lock discipline in OxCaml

2026-05-08T10:00:00+00:00

In the previous post we fixed the racy gensym with Portable.Atomic. That worked because the shared state was a single integer with atomic primitives. What about state that needs a hash table, a multi-step update, or any structure where atomics aren’t enough? OxCaml’s answer is the capsule: a way to bundle state with its lock so the lock discipline becomes a type-checker job rather than a programmer convention.

The example here is the capsule version of gensym from my CS6868 lecture (handout). Capsules don’t introduce a new mode axis; they’re a small library that composes the axes we’ve already met: contention and portability, uniqueness, and linearity. The cells below run in the same in-browser OxCaml toplevel as the previous post; the A note on the API section at the end explains why we use a slightly older flavour of the capsule API (it’s a bundle-size thing, not pedagogical).

Why atomics aren’t enough

For a single integer counter, atomics are perfect. Drop the size constraint and they’re not. Suppose gensym had to consult a hash table of already-issued names, or update two counters in lockstep, or do a multi-step modify-then-record. None of that fits an atomic word. The standard OCaml answer is Mutex.t:

let mutex = Mutex.create ()
let table = Hashtbl.create 16

let safe_insert k v =
  Mutex.lock mutex;
  Hashtbl.add table k v;
  Mutex.unlock mutex

(* Nothing stops you from forgetting to lock: *)
let unsafe_insert k v =
  Hashtbl.add table k v   (* races, but compiles. *)

Two things are wrong here. First, the mutex and the data are separate values; nothing at the type level connects mutex to table. Second, “always lock before access” is a programmer convention. The compiler can’t tell what mutex you meant for what state, and an unlocked Hashtbl.add is a runtime data race that nothing catches.

Yes, OxCaml’s contention rule from the previous post would reject direct mutation of a @ contended table in a parallel context. But that only helps if the type system can see that table is contended in the first place. Once you have a top-level shared mutable, you need a structural reason for the type checker to believe that “you are holding the lock right now.” That’s exactly what a capsule provides.

Gensym in a capsule

The capsule pattern packs the counter inside a brand-locked container, making the lock the only handle to the state:

[@@@alert "-deprecated"] module Cap = Capsule_expert module Mtx = Capsule_blocking_sync.Mutex let make_counter () = let Cap.Key.P key = Cap.create () in let counter = Cap.Data.create (fun () -> ref 0) in let mutex = Mtx.create key in fun () -> Mtx.with_lock mutex ~f:(fun password -> Cap.access ~password ~f:(fun access -> let c = Cap.Data.unwrap ~access counter in incr c; !c)) let next = make_counter () let gensym prefix = prefix ^ "_" ^ string_of_int (next ()) let () = Printf.printf "%s %s\n" (gensym "x") (gensym "y")

Run it; you get two distinct ids, and the toplevel binds val next : unit -> int and val gensym : string -> string. No handle to the inner ref is in scope outside Mtx.with_lock. Let’s read what’s making that true.

The brand `'k`: an existential tied to a fresh capsule

Cap.create () returns a key wrapped in an existential constructor Cap.Key.P:

type packed = P : 'k Key.t -> packed [@@unboxed]
val create : unit -> packed

The let Cap.Key.P key = Cap.create () pattern unwraps it and introduces a fresh type variable, call it $k, in the surrounding scope. key is then bound at type $k Cap.Key.t. A second Cap.create () would introduce $k2, distinct and unrelated. Each P pattern brings a new type into the world.

That brand is the static glue between capsule, data, and mutex. When we call Cap.Data.create (fun () -> ref 0) next, the compiler unifies the data’s type parameter with the brand of the key it’ll eventually be unwrapped under, so counter : (int ref, $k) Cap.Data.t. The mutex created from key (consuming it as @ unique) is also branded $k. Henceforth this data only opens under this mutex’s lock.

(Why is the existential pattern wrapped inside make_counter () rather than at the top level? OCaml refuses top-level let-bindings that introduce existentials, so we hide the unwrap inside a function. The closure returned by make_counter carries the $k-branded mutex and counter in its environment.)

The bare `ref` is unreachable from outside the capsule

Look at where the ref 0 lives:

let counter = Cap.Data.create (fun () -> ref 0)

That ref has no binding outside the closure handed to Cap.Data.create. The only handle out is the Cap.Data.t value branded $k. There is no aliased reference to smuggle out: the ref’s reachability is single-pathed. This is exactly the configuration the uniqueness post was about, a unique reference with no aliases to it, but here we get it for free from how the API is shaped, without writing @ unique anywhere ourselves. The capsule API is exploiting uniqueness as a construction principle.

`Cap.Data.t` mode-crosses contention and portability

Just like Portable.Atomic.t from the previous post, the type ('a, 'k) Cap.Data.t carries the kind annotation value mod contended portable. A closure that captures a capsule value stays @ portable, and any domain may hold the capsule. So the closure returned by make_counter (which captures both mutex and counter) type-checks at portable mode and is safe to send to another domain.

This is the same mode-crossing move we saw with Portable.Atomic, just applied to a more general container.

`with_lock` produces an `access` token, briefly

Mtx.with_lock is the only way into the critical section. Its signature:

val with_lock
  : 'k t
  -> f:('k Cap.Password.t @ local -> 'a @ once unique) @ local once
  -> 'a @ once unique

Three different things in this signature are doing real work.

Brand 'k: the password’s brand matches the mutex’s brand. A password from a different mutex (different $k1) cannot unwrap our counter. The type checker refuses to unify the two existentials and the program does not compile.
@ local: the password is local to the body’s region. It cannot escape f by being returned, stored in a global, or stuffed into a closure that outlives the call. When with_lock returns the lock is released, and there’s no password left in scope to attempt to touch the data with. (Locality is the stack-allocation / no-escape axis we covered; same mechanism, different use.)
@ once on the body: f can be called at most once. This is the linearity guarantee. The body cannot be re-entered with the same password, and the runtime cannot smuggle it out for later. Each with_lock use is a single shot.

Inside f, Cap.access ~password ~f:(fun access -> ...) converts the password into a brand-matching 'k Access.t for the inner body. Cap.Data.unwrap ~access counter then accepts the access and returns the underlying int ref at @ uncontended. We hold the lock; no other domain can be touching the data. The next two lines, incr c; !c, are ordinary OCaml mutation; the contention rule is satisfied by construction because unwrap promised uncontended access.

Outside the lock body, neither password nor access exists in scope, so there is no path from the outer program to incr c that doesn’t go through with_lock.

Brand mismatch is a type error

The cell below tries the genuinely unsound case: protecting one Cap.Data.t with two distinct mutexes. If this compiled, two threads holding distinct mutexes could enter critical sections on the same data simultaneously. The type checker refuses, and reports the two existentials don’t match.

[@@@alert "-deprecated"] let abuse () = let Cap.Key.P key1 = Cap.create () in let Cap.Key.P key2 = Cap.create () in let data = Cap.Data.create (fun () -> ref 0) in let m1 = Mtx.create key1 in let m2 = Mtx.create key2 in (* First unwrap pins data's brand to key1's $k. *) let _ = Mtx.with_lock m1 ~f:(fun password -> Cap.access ~password ~f:(fun access -> !(Cap.Data.unwrap ~access data))) in (* This second unwrap fails: data is branded with the FIRST key's $k, not the second's $k1. *) Mtx.with_lock m2 ~f:(fun password -> Cap.access ~password ~f:(fun access -> !(Cap.Data.unwrap ~access data)))

The error names the two existentials and points out they cannot unify. The handout’s capsule_abuse.ml walks through two more attempted escape hatches:

Leak the inner ref through a top-level Portable.Atomic. The store compiles, but anything inside a portable atomic is @ contended, and the contention rule from the previous post forbids reading or writing a mutable field of a contended value. So you can park an alias, but you can’t dereference it.
One mutex protecting two Cap.Data.t values. Allowed, and correctly so. Both data values are branded with the same $k, and a single critical section can update both. (Equivalent to one mutex guarding two fields of a struct.)

The first closes the obvious leak path (via the same uniqueness-of-paths property that protects the bare ref); brand mismatch closes the aliased-mutex case; the one-mutex-two-data case shows the rule isn’t gratuitously restrictive.

What each mode is doing

Pulling the threads together, with one line per axis:

Contention ensures shared mutable state can’t be read or written from outside a critical section. Inside with_lock we get an @ uncontended view via unwrap; outside, the contents aren’t in scope at all.
Portability lets the closure cross domain boundaries. Cap.Data.t mode-crosses, so the closure capturing it stays @ portable.
Locality confines the password (and the access derived from it) to the body of with_lock. No way to keep a token past lock release.
Linearity (@ once on the body) makes the critical section single-shot. The body cannot be re-entered with the same password.
Uniqueness isn’t an annotation we wrote; it’s the property that the inner ref has exactly one path of access, through the capsule. The API shape forces it.

A note on the API

Two API choices in the cells above are worth flagging for anyone transcribing this to a real .ml file.

Curated vs expert capsule API. The full capsule opam library wraps the brand-based primitives in a curated layer (Capsule.Isolated, Capsule.Guard, Capsule.Shared, Capsule.Data) that hides most of the Key.P/password/access plumbing for common patterns. For most application code you reach for those first. We use Capsule_expert directly here because the curated wrapper pulls in base, sexplib0, and friends that would balloon the in-browser bundle by ~270 MB. The lecture’s Capsule.Data calls are syntactically identical to ours.

Mutex flavour. The lecture uses Await_capsule.Mutex.with_lock, the recommended (non-deprecated) primitive that hands the body an Access.t directly. Bundling the await library into the in-browser toplevel pulls in roughly 280 MB of transitive deps (sexplib, stdio, ppx_*, base.shadow_stdlib, …), so we use the deprecated-but-equivalent Capsule_blocking_sync.Mutex with the deprecation alert silenced. It hands the body a 'k Password.t @ local, which we convert to a 'k Access.t via Capsule_expert.access. One extra line of plumbing; same modes are doing the work. In a real .ml file with await available, swap Capsule_blocking_sync.Mutex for Await_capsule.Mutex and drop the Capsule_expert.access step; the lecture’s verbatim form in the handout is what you want.

What’s left

The capsule pattern is enough to put a hash table or a more elaborate shared structure under a single mutex with a compile-time guarantee that nothing touches it without the lock. To actually run gensym from two domains in parallel we need the parallel scheduler: Parallel.fork_join2 and the @ portable once story for closures crossing the fork boundary. That’s the next post.

The point of stopping here is that the hard part of safe shared mutable state in OxCaml isn’t the parallel primitive, it’s the lock discipline. And the lock discipline turns out to be a small composition of the mode axes we already had to learn anyway. The new library is small; the framework was already in place.

Data race freedom in OxCaml

2026-05-07T10:00:00+00:00

A while back I wired up x-ocaml so this blog could embed live, editable OCaml notebooks. That post used a vanilla OCaml 5 toplevel. Today the toplevel running in your browser is built from OxCaml, the Jane Street fork of the compiler. That means we can prove a small parallel program is data-race free, interactively, without ever spawning a thread.

The examples below are adapted from the OxCaml team’s excellent Intro to Parallelism (Part 1) tutorial, by way of my recent CS6868 lecture on OxCaml (handout). The tutorial is the canonical reference and is worth reading in full; this post tries to capture the essence in a more bite-sized form, focused on the two new mode axes that together rule out data races at compile time — and on one subtlety about portability that’s easy to misread.

A quick aside: data races in OCaml are less scary

Before we dive in, it’s worth pausing on why we want to rule out data races at all in OCaml. In C, C++, or unsafe Rust, a data race is catastrophic: the standard licenses the compiler to do anything, including silent memory corruption, on the basis that your program had no defined meaning to begin with. OCaml is considerably gentler. OCaml’s memory model guarantees that even programs with races preserve type safety and memory safety — a racy program may observe weakly-consistent values across domains, but it will not crash, won’t read uninitialised memory, and won’t violate the type system’s invariants.

So a race in OCaml is a logic bug, not a runtime footgun. Why bother catching them statically, then? Because once a program is data-race free, you get sequential consistency: any observed behaviour can be explained as some interleaving of the operations of the different domains, with each domain’s own operations executed in program order. That’s still concurrent reasoning — there are many possible interleavings — but it’s the simplest model concurrent code can have, and equational reasoning, induction, and the usual program-logic tools all transfer over to each interleaving. Drop race freedom and you lose this: observed behaviours can only be justified by allowing intra-thread reorderings as well, where a single domain’s operations appear to execute out of program order to other domains. Sequential consistency is the real prize. Race freedom is what buys it.

Hello, OxCaml

A quick sanity check that your browser really is running an OxCaml toplevel. The @ local annotation is OxCaml-only syntax; on a stock OCaml parser it wouldn’t even parse.

let use_locally (x @ local) = x + 1 let () = Printf.printf "use_locally 42 = %d\n" (use_locally 42)

A racy `gensym`

Here’s the example we’ll keep coming back to: a symbol generator that hands out distinct ids by incrementing a captured counter. Sequentially it works; the cell prints two ids. The last line tries to ship gensym to another domain, and that’s where the type checker stops us.

[@@@alert "-do_not_spawn_domains"] let gensym = let count = ref 0 in fun prefix -> count := !count + 1; prefix ^ "_" ^ string_of_int !count let () = Printf.printf "%s %s\n" (gensym "x") (gensym "x") let _ = Domain.Safe.spawn (fun () -> gensym "x")

Run it from two domains in parallel and you’d tick every box on the four-ingredient recipe for a data race: two domains, a shared count, both writing, and count is a plain ref — not atomic. On stock OCaml 5, the compiler would happily let you ship this closure to another domain. OxCaml refuses, before anything runs. The error names two things — nonportable and portable — that aren’t in vanilla OCaml’s vocabulary. What do they mean?

Two new modes for data race freedom

OxCaml extends OCaml’s type system with several modes — annotations that describe how a value can be used, orthogonal to its type. I’ve already written about two of them on this blog: uniqueness, which tracks whether a value has at most one reference, and linearity, which tracks how often a value may be used. Today’s post is about a different pair — the two modes that, together, rule out data races at compile time:

contention — uncontended / contended: tracks whether multiple domains can simultaneously access a value. A contended value might be in another domain’s hands right now, so the type system limits what you can do with it.
portability — portable / nonportable: tracks whether a value can safely cross a domain boundary at all. Closures that are sent to other domains must be portable.

The pair is enough to catch the gensym race. Let’s look at each restriction in isolation, then come back to the closure.

Contention rejects mutable writes

A contended value might be mutated by another domain right now. So OxCaml refuses to let you read or write its mutable fields:

type mood = Happy | Neutral | Sad type thing = { price : float; mutable mood : mood } (* Reading an immutable field of a contended value is fine. *) let price_contended (t @ contended) = t.price (* Writing a mutable field is not. *) let cheer_up_contended (t @ contended) = t.mood <- Happy

The error on the last line names exactly the rule: “This value is contended but is expected to be uncontended because its mutable field mood is being written.” Even reading a mutable field on a contended value is rejected — another domain might be mid-write at the same instant.

Portability rejects captured refs

Portability is about closures. A @ portable closure is one the compiler has verified is safe to ship to another domain, with one critical catch: every value the closure captures from its enclosing scope is treated as contended inside the closure body. A pure function that doesn’t mutate anything is trivially portable — there’s nothing the contention rule can object to:

let test_portable () = let (f @ portable) = fun x y -> x + y in f 1 2 let () = Printf.printf "test_portable () = %d\n" (test_portable ())

Capturing a ref is fine on its own; mutating one that’s been captured is what falls afoul of the rule:

let test_nonportable () = let r = ref 0 in let (counter @ portable) () = incr r; !r in counter ()

Read the error carefully — it tells you exactly why counter isn’t portable. The closure is @ portable, so the captured r is treated as contended inside the body. But incr r is a mutation, and writing through a ref requires it to be uncontended. The two rules collide. And now the original gensym rejection makes sense: it does exactly the same thing — mutates a captured count.

The trap, and the actual rule

Read those last two cells together and it’s tempting to draw the wrong moral: that to make a function safe to ship to another domain, you have to give up side effects. That would be far too restrictive.

The actual rule is narrower, and the difference matters.

Portability constrains what a closure captures from its enclosing scope. Captured values become contended — that’s the rule, and it’s why gensym got rejected. But a closure’s parameters are not captures. They’re handed in fresh at each call, so they can be passed at any mode the type spells out — including @ uncontended. The obligation to provide an uncontended argument shifts to whoever is doing the calling. That’s a much weaker requirement than “no side effects.”

Captured vs parameter, in code

Here’s the example that makes the distinction concrete. loop is @ portable, and its body mutates an int ref. That works because the ref is a parameter annotated @ uncontended, not something captured:

let (factorial_portable @ portable) n = let a = ref 1 in let rec (loop @ portable) (a @ uncontended) i = if i > 0 then begin a := !a * i; loop a (i - 1) end in loop a n; !a let () = Printf.printf "factorial_portable 10 = %d\n" (factorial_portable 10)

Two @ portable annotations are doing work here. The inner one says loop is shippable to another domain — that’s the interesting one, and it works because a is loop’s parameter, not a capture: when loop is eventually called from somewhere parallel, that somewhere has to prove the a it passes in is uncontended. Portability didn’t ban mutation; it pushed the proof to the call site. The outer annotation says the whole factorial_portable is portable too — it allocates a fresh a on each call and captures nothing from outside, so we can ship the whole function to another domain. The compiler verifies both annotations as part of accepting the program.

A formal aside: defaults and submoding

We’ve been writing @ contended and @ portable as if they were the interesting modes and their absence was nothing in particular. There’s actually a small lattice on each axis, with a default and a direction of “how strong” the guarantee is. The handout summarises it like this:

Axis	Modes (`⊑`)	Default
Contention	`uncontended` ⊑ `shared` ⊑ `contended`	`uncontended`
Portability	`portable` ⊑ `nonportable`	`nonportable`

The relation A ⊑ B is the submoding order: a value at mode A may be used wherever mode B is expected, because A carries the stronger guarantee and B is the looser expectation. An uncontended value satisfies a @ contended parameter (we just promised the callee might let other domains touch it; if no other domain ever does, that promise is trivially compatible). A portable closure satisfies a @ nonportable slot. But not the other way around — submoding goes one way only.

The defaults are what you get when you write plain OCaml. Every value starts out uncontended — no other domain has it, so reads and writes are unrestricted. Every closure starts out nonportable — we make no claim about shipping it elsewhere. That’s why ordinary OCaml code keeps type-checking under OxCaml: defaults are the most permissive end of each axis, and you only meet the new restrictions when you (or a parallel API) ask for a stronger guarantee. The post uses only the two endpoints of the contention chain — uncontended and contended — and skips shared, which permits read-only access across domains and isn’t needed for the gensym story.

Back to `gensym`: the fix

The captured-vs-parameter distinction is enough to fix simple loops, but gensym is a different shape: there’s a single counter that must be shared across calls. We need a counter type that’s itself portable — something whose values mode-cross both contention and portability, so a closure capturing it stays portable. That’s exactly Portable.Atomic.t, from OxCaml’s portable library: a 'a Portable.Atomic.t is always portable and always uncontended, regardless of where it lives.

Wrap the counter and gensym in a module so the toplevel can see the whole bundle as portable, then actually do what got us in trouble at the start: ship gensym to a freshly-spawned domain. The compiler accepts it (it verified Gen is portable), and the program runs:

[@@@alert "-do_not_spawn_domains"] module Gen = struct open Portable let count = Atomic.make 0 let gensym prefix = let n = Atomic.fetch_and_add count 1 in prefix ^ "_" ^ string_of_int n end let d = Domain.Safe.spawn (fun () -> Gen.gensym "y") let s1 = Gen.gensym "x" let s2 = Domain.join d let () = Printf.printf "%s %s\n" s1 s2

The toplevel reports module Gen : sig ... end @@ portable — mode inference noticed every value inside is portable and made the whole module portable, so Gen.gensym survives extraction at portable mode. You might wonder why we don’t just write let (gensym @ portable) prefix = ... at the top level, the same shape we used for factorial_portable above. That’s a quirk of the toplevel: a bare let lands in the implicit toplevel module, which itself sits at the default nonportable mode, so when Domain.Safe.spawn later reads gensym back out, it sees a nonportable binding and refuses — even though the closure was verified portable at its binding site. The factorial_portable cell got away with the simpler form only because nothing else tried to extract the binding at portable mode. Wrapping in module Gen gives the closure its own portable home, which is what lets it actually cross a domain boundary. In a real .ml file the whole compilation unit is a module that mode inference can mark portable wholesale, so this dance isn’t necessary.

(A note on the Portable.Atomic you see here: in a real program you’d get it by open Portable from the portable opam library. To keep the page weight reasonable, this notebook ships only basement — the small library that provides the actual atomic operations — and a hidden setup cell at the top of the page wraps it with the kind annotation : value mod contended portable. That kind is what tells the compiler this type mode-crosses both axes — a closure capturing one stays portable, and any domain may touch it.) The handout notes that for state more elaborate than a single atomic counter, the right answer is capsules — a structural way to bundle mutable state with its access protocol so the whole package is portable.

What’s left

The race-freedom guarantee here is independent of any test run: every rejection above happened at the same compile step an OxCaml binary on disk would go through, before any code executed. The final spawn is a demonstration, not a proof — it’s the compiler accepting the program in the first place that tells us no race is possible. Two mode axes, a kind annotation, and a clear rule about what “portable” means were enough to make data-race freedom a property the type checker enforces.

The lecture this is drawn from goes further into capsules (for shared state more elaborate than an atomic) and Parallel.fork_join2 (for structured parallelism). Material for another post.

From Convergence to Confidence: Push-button verification for RDTs

2026-04-28T10:00:00+00:00

What does it mean for a replicated data type to be correct? For most of the literature, my own prior work included, the answer has been convergence: two replicas that have applied the same operations end up in the same state. I argued in my PaPoC 2026 keynote last week that for many useful data types convergence is not enough, and agentic proof-oriented programming can help close the gap between convergence and confidence.

A stronger answer, replication-aware linearizability from Wang et al. (PLDI 2019), asks for full functional equivalence with a sequential specification: the merged state should behave as if the operations everyone did had run in some sequential interleaving. RA-linearizability is what our verification work has aimed at for the past few years, and for a class of useful data types it still falls short: the ones where the state is a grow-only bag and the interesting semantics live in the read function. This post is the longer-form version of the keynote argument, plus a digest of recent work on Sal.

PaPoC 2026 was co-located with EuroSys in Edinburgh, and was a wonderful workshop. Thanks to the program chairs, Lindsey Kuper and José Orlando Pereira, for running it. A permanent entry for the keynote is on my talks page.

Convergence is not enough

The canonical example is the OR-Set. The state is two sets, an adds set and a tombstones set. Adding an element tags it with a fresh id and stores the (element, id) pair in adds. Removing an element does not delete anything; it adds a tombstone entry for each currently observed (element, id) pair to tombstones. Reading filters adds against tombstones, and merging unions both components componentwise. The data type converges, but tombstones accumulate without bound. The fix is a bit cleverer than just tagging: each replica tracks causality explicitly, by maintaining a context of operations it has observed, and the merge uses that context to recognise when an element was removed at another replica without having to keep a tombstone for it. The result is the “efficient” OR-Set.

In practice this is harder than it sounds. Riak’s riak_dt, a production CRDT library, shipped an optimisation to its OR-Set Map that broke monotonicity (issue #79). Christopher Meiklejohn, one of the authors of the Riak DT Map paper, later wrote about how easy it is to get the inflationary, deterministic, least-upper-bound conditions wrong, and noted that the team “have even got this wrong a few times” themselves. Martin Kleppmann’s 2019 PaPoC paper on collaborative-text anomalies had errata of the same flavour, appended to the publication page a few years after the fact: the non-interleaving condition proposed in §2.1 “cannot be guaranteed by any algorithm,” and the algorithm in §3.1 does not guarantee convergence.

A well-defined join and a join that does what the user expects are not the same thing, and the gap between the two is where most of these bugs live.

A short history of how we tried to close that gap

My group has been chipping at this for a few years. In Peepul (PLDI 2022) we tried to capture intent axiomatically: write the spec as a fold over a partially ordered event graph, then prove the implementation simulates it. The proofs closed, but the cost was telling. The queue MRDT case study took 1,123 lines of proof spread over 75 lemmas against a 32-line implementation, with each F* verification run taking on the order of 4,753 seconds. F*’s SMT-aided proofs were brittle (z3 upgrades broke them, and the solver only ever returned yes / no / timeout with no counterexample), and the methodology did not really scale past a handful of carefully-chosen examples.

Neem (OOPSLA 2025) replaced the axiomatic spec with RA-linearizability, with one twist: instead of taking a separate sequential specification as input, Neem treats the operational definition of do_ itself as the spec. The merge is correct if the resulting state behaves like some sequential interleaving of the updates under that definition. Neem also gave us a fixed schedule of 24 verification conditions (six on do_, twelve on merge, six on conflict resolution) that, if all closed, certify RA-linearizability. That was a real upgrade. But it was still in F*, still SMT-fragile, and there was a class of useful RDTs for which closing the 24 VCs left the correctness statement itself near-vacuous.

The pattern that exposes that vacuity is recognisable. When porting an op-based CRDT to a state-based one, a common move is to dump every operation into a grow-only set and leave the read function to reconstruct the result. The merge is then set union, any sequence of operations is trivially its own linearisation, and the 24 RA-linearizability VCs close without much work. The data type can still be wrong, because nothing in that proof has said anything about intent: which characters are bold, which key sits at the head of the queue. Peepul did try to capture intent, but it expressed the spec over a partially ordered graph of events with no further structure, and that turned out to be both an awkward medium to write specs in and a brittle one to prove against.

We have started calling these vacuous-convergence cases Tier-C RDTs, in a three-tier taxonomy that we settled on this past month:

Tier A: state is the semantic content. LWW-Register, PN-Counter, Bounded-Counter, MAX-Map. The RA-linearizability VCs reduce to lattice arithmetic and cover what one wants to know.
Tier B: merge does non-trivial computation. Multi-Valued-Register, LWW-Element-Set. Still substantive.
Tier C: state is a grow-only bag, and the semantics live in the read. OR-Set, RGA, Add-Win Priority Queue, and Peritext (which the rest of the post comes back to).

Roughly half the Sal suite, and most of the RDTs people actually want to use, sit in Tier C. For Tier C, “verified” without read-side theorems is an overclaim.

What Sal actually is

Sal is the work I presented at PaPoC, joint with Pranav Ramesh and Vimala Soundarapandian. The wider line of work this builds on also involves Adarsh Kamath, Aseem Rastogi, and Kartik Nagar. It ports Neem to Lean 4 and adds a multi-modal proof tactic. The repo is at https://github.com/fplaunchpad/sal.

The Lean choice is not original to us. Ilya Sergey is the one who convinced me to take Lean seriously, and Sal’s multi-modal tactic stack is directly inspired by Loom, the multi-modal proof-orchestration layer his group has been building. Velvet, the Lean library it sits inside, verifies imperative programs; Sal verifies RDTs. The toolchain is shared.

When you write by sal in front of a verification condition, three things happen in sequence. First, dsimp + grind: pure rewriting plus Lean’s bit-level automation. If this closes the goal, the proof term is checked by the Lean kernel and the trusted base is just Lean. If it doesn’t, Sal hands the goal to Z3 via lean-blaster. Fast, but Z3 is now in the trusted base, and Sal records this with a Blaster_admit annotation so the borrowed trust is visible. If that does not close either, an AI agent (Claude Code or Aristotle) tries to write a proof term, which Lean then kernel-checks. The TCB shrinks back to just Lean.

For the LWW-Register all 24 VCs close at stage 1. The whole specification fits on a screen:

abbrev concrete_st := ℕ × ℕ
def init_st : concrete_st := (0, 0)

def lex_max (a b : ℕ × ℕ) : ℕ × ℕ :=
  if a.1 > b.1 then a
  else if b.1 > a.1 then b
  else if a.2 ≥ b.2 then a else b

def do_ (s : concrete_st) : op_t → concrete_st
  | (ts, _, .Write v) => lex_max s (ts, v)

def merge (a b : concrete_st) : concrete_st := lex_max a b

theorem merge_comm (a b : concrete_st) : eq (merge a b) (merge b a) := by sal
theorem merge_idem (s : concrete_st) : eq (merge s s) s            := by sal
-- ... 22 more, all closed by `by sal`

Across the current Sal suite (28 RDTs, 17 CRDTs and 11 MRDTs, 648 verification conditions) the breakdown from the paper was roughly 69% closed at stage 1, 28% via Z3, and 3% via AI-completed ITP. That is the “push-button” claim, and for Tier A and B it holds up well.

Two practical notes. First, Aristotle has been the workhorse at stage 3: LWW-Map, Shopping-Cart, and MAX-Map each had VCs that neither grind nor Z3 closed, and Aristotle produced kernel-checked intermediate lemmas that did. Second, on the brittleness-of-tooling worry, the suite was bumped from Lean 4.26 to 4.28 between the paper draft and PaPoC. One Shopping-Cart obligation drifted under the new grind and needed a tweak; everything else carried over. That is a different experience from the F* / z3-upgrade story above.

Tier C is where things get more involved.

The Peritext story

Peritext is a CRDT for collaborative rich-text editing, by Geoffrey Litt, Sarah Lim, Martin Kleppmann, and Peter van Hardenberg (CSCW 2022). The design has four state components (characters, parent pointers, tombstones, formatting marks). What makes the paper unusually pleasant to mechanise is the care its authors put into specifying intent. §3 walks through eight worked examples of intent preservation, each spelled out as a small concrete scenario, and appendix §A.2 catalogues them as a single table for reference.

In one, Alice bolds the entire sentence “The fox jumped” while Bob inserts the word “brown” in the middle, and the result should be The brown fox jumped. In another, working with the same sentence, Alice bolds “The fox” while Bob bolds “fox jumped”, and the overlapping word “fox” ends up bold under both. A third concerns link spans: a link has a hard right edge, so a concurrent insert at the end of the link should not expand it. And so on for the rest of the eight. The paper argues informally that the design handles each example correctly, and ships a TypeScript reference plus property-based tests, but no machine-checked proofs.

The 24 RA-linearizability VCs are trivial here. Peritext’s state is four grow-only components and componentwise union commutes; by sal closes them in seconds. That work lives in Peritext_CRDT.lean (571 lines, mostly state plumbing), and at the level of RA-linearizability the data type is verified.

The interesting questions, however, are about the read: the function that takes the state and renders it as formatted text. That function lives in Peritext_ReadSide.lean, currently 1,316 lines. It answers questions like: do the start- and end-side anchor bits matter? do overlapping bolds compose? does tombstoning a character preserve the formatting of everything else? do insertions at a span boundary fall inside or outside the span under the paper’s expand-vs-contract rules? None of these are anywhere in the 24 VCs.

The eight worked examples in §3 turn out to be a much better starting point than the VCs for actually mechanising the data type. Each example is small enough to write down as a concrete state and a concrete claim about its read. Nik Swamy and Shuvendu Lahiri have a recent post calling this kind of artifact a Small Proof-Oriented Test, or SPOT: “a small, concrete, verified test case, proving that the test always succeeds.” By writing §3 the way they did, the Peritext authors had effectively done the specification-engineering half of that job for us; all that was left was to translate each example into Lean and prove it against the implementation. So we did.

Take Example 1 from the paper. Alice bolds the entire sentence “The fox jumped” while Bob concurrently inserts the word “brown” in the middle. Convergence is not the question: both replicas will agree on a state once they merge. The question is what the rendered text looks like. Peritext’s expand-on-the-bold-side rule says the inserted text should fall inside Alice’s bold span, so the merged result reads “The brown fox jumped”, with “brown” formatted bold along with everything else. In Sal we simplify Bob’s insert to a single character ‘b’ (enough to exercise the rule, and the proof stays tractable), and the SPOT translates almost directly:

-- Alice (rid 0) types "The fox jumped" and bolds the whole thing.
def ex1_pre : Scenario :=
  the_fox_jumped.bold 0
    ['T','h','e',' ','f','o','x',' ','j','u','m','p','e','d']

-- Bob (rid 1) concurrently inserts 'b' between "The " and "fox".
def ex1_post : Scenario := ex1_pre.insertCharAfter 1 (4, 0) 'b'

-- Alice's bold mark, opId (15, 0), spans the original sentence.
def ex1_mark : MarkOp := Mark.bold (15, 0) (1, 0) (14, 0)

-- The intent claim: Bob's 'b' ends up inside Alice's bold span.
example : in_span_visible ex1_post.state ex1_mark (16, 1) := by ...

The other seven worked examples take the same shape: an English description in the paper, a few lines of DSL builder to set up the concrete scenario, and a one-line theorem statement that captures the paper’s intent claim. Peritext_SPOT.lean is 323 lines for all eight. The proofs themselves are unremarkable; the work was in setting up the DSL so the scenario reads like the paper.

The read-side theorems in Peritext_ReadSide.lean are universally quantified versions of the same statements: instead of “Bob’s ‘b’ is in Alice’s bold span at this concrete state,” the theorem insert_within_span_in_span_visible says “for any state, any mark, any character inserted strictly inside the span at a fresh opId, the inserted character is in the span.” Each SPOT then closes via that universal theorem applied to its concrete state. Generalisation of the SPOT is the read-side proof, which I find a useful way to think about the relationship.

We have since written SPOTs for all 28 RDTs in the suite, and the generalised read-side theorems for the Tier-C ones. The Tier-A SPOTs are mostly there as documentation that a reader can confirm the read function does what the lattice arithmetic suggests it does.

It is worth saying how an early draft of the Peritext read-side went wrong, since the failure is a sharp version of the talk’s title. I had Claude draft a predicate called in_span_boundary from a careful reading of paper §3.3, with four cases (start of span, end of span, after-of-start, after-of-end) and a boolean side bit. The proofs went through. About 400 lines of theorems closed across both the CRDT and the MRDT versions, and I was about to commit. Re-reading §3.3 with the predicate in front of me, the predicate turned out to be backward. The after_of c endId → endSide clause encoded the opposite of the link-contract case: it included post-endId inserts in the span where the paper says they should be excluded. The proofs validated the proof, not the spec. Tests pass, code is wrong. The bug was caught only when I wrote an alternative predicate (in_span_visible) and the two disagreed on Example 8.

The fix was a four-rule inductive characterising the RGA visible order (parent-child, sibling via opid_max, left-descendant-of-older-sibling, transitive) plus a separate bold_expand_reach predicate for the bold-expand semantics in Example 7. About 800 lines of the buggy parallel track were deleted in the process. Eight _visible theorems now map one-to-one to the paper’s eight worked examples; there is a crosswalk for anyone who wants the full table.

The piece of methodology I would recommend to someone starting out: the SPOTs catch this kind of failure if the SPOT itself is faithful to the paper’s example. Had I started by formalising the worked examples as SPOTs and only then generalised, the disagreement on Example 8 would have shown up immediately rather than after 400 lines of proofs against the wrong predicate. John Regehr’s zero-degree-of-freedom LLM coding post argues a similar point in a different domain: pin the agent with fast, deterministic, executable oracles, and the agent has nowhere to drift. The Peritext SPOTs are an executable oracle for the read-side spec.

Beyond Peritext

Peritext is one example of a broader pattern. PaPoC 2026 had several other talks where the hard work was not the merge but in writing down what the data type was supposed to mean in the presence of concurrent edits. Three I want to flag, because they suggest the intent-formalisation problem has finally become the bottleneck people are willing to talk about openly.

AegisSheet (Florian Pfeil, David Scandurra, and Julian Haas, TU Darmstadt) studies collaborative spreadsheets. Their starting move is honest empirical work: they tabulate how Google Sheets and Notion behave under all combinations of concurrent EditCell / InsertR/C / RemoveR/C / MoveR/C and classify each outcome as desirable, suboptimal, or destructive. Most of the cells in those tables are red. They then design a compositional spreadsheet CRDT that turns the red cells green. The CRDT is the easy half; the table of intended semantics is the hard half, and they did it.

ERA (Kegan Dougal, Element Creations) addresses the duelling admins problem in group-management CRDTs in Matrix and Keyhive: two equally-permissioned admins concurrently revoke each other’s permissions. Whose revocation wins? Kleppmann’s PaPoC 2025 keynote suggested seniority ranking, but a Byzantine admin can backdate events to roll back their own revocation. ERA proposes epoch events: an external arbiter batches concurrent events into epochs and imposes a bounded total order, which gives finality without sacrificing availability. Like the spreadsheet case, the contribution is mostly about specifying what correct looks like under adversarial concurrency, with the data structure designed to match.

The closing lightning talk by Florian Jacob, Johanna Stuber, and Hannes Hartenstein (KIT) sketches a path to formal verification of local-first access control. Same shape: write the intent down precisely, then prove an implementation matches.

None of these are RDT-specific. Each is a different domain (rich-text, spreadsheets, group membership, access control) where the merges are easy and the intent is hard. SPOTs, oracles, and the multi-modal Lean stack are all reactions to the same shift in where the difficulty lives.

The past three weeks

Since the Sal paper was finalised, the repo has had something like 200 commits, overwhelmingly mine, with Pranav having landed the architectural heavy lifting earlier on. The headline change is in coverage: 13 new CRDTs and 2 new MRDTs since the paper, taking the suite from 13 RDTs to 28. Most of the additions are smaller scalar, set, and map types (MAX-Register, MIN-Register, LWW-Register, LWW-Map, MAX-Map, Grow-Only-Set, Grow-Only-Multiset, LWW-Element-Set, Shopping-Cart, and a few more) that fill out Tier-A and Tier-B coverage and serve as documentation that the obvious read functions do the obvious thing. The three additions that took real work, and that the rest of this section is about, are the paper-only designs that finally got machine-checked Lean ports.

Peritext is the largest of the three: 1,316 lines of read-side, 571 of CRDT, 291 of a small DSL for the §3 worked examples, and 323 of SPOTs. Eight intent-preservation theorems matching the paper’s worked examples one-to-one, plus a wf_afters acyclicity invariant and a bold_expand_reach predicate for Example 7. Roughly two days of agent-paired work, mostly spent on the spec rather than on the proofs.

The Add-Wins Priority Queue of Zhang et al. (Internetware 2023) is a 391-line CRDT plus a 325-line read-side. The translation is interesting in its own right. The paper has a per-record count field that ties commutativity in knots; in Sal we flatten it into a separate I component for increment records and use a snapshot-in-op-payload trick (the Rmv op carries the observed-set as a parameter), which gives us rc := Either everywhere and lets the RA-linearizability VCs drop out without any SMT.

The Bounded Counter of Balegas et al. (SRDS 2015), in the state-based formulation that Sypytkowski wrote up in 2019, is a 465-line CRDT structurally a PN-Counter plus a transfer matrix. The 24 VCs are trivial. The bound itself is not part of the verified model: it is enforced operationally at the client boundary (a replica refuses to emit a Dec that would push its quota negative), and that part is currently unverified. The headline number alone might suggest more than is there, so worth calling out.

A separate cleanup is worth noting. About 85 Blaster-admits across the suite closed via a single pattern, per-component decomposition: instead of one monolithic SMT call on the full state, split the state into components, prove each component independently, then combine. The OR-Set MRDT alone went from 20 VCs trusting Z3 down to 3 (closing 17 in a single commit). The pattern was human-found; the agent applied it consistently across files.

The qualifier from the Peritext section applies to all three of these ports as well: the RA-linearizability VCs are easy, and the work that took time was the spec.

For readers who want to play with the data types directly, fplaunchpad.org/sal hosts an interactive playground for every RDT in the suite. The CRDT pages let you drive two replicas in parallel and merge one into the other; toggling “show concrete state” exposes the lattice that the convergence proofs are about. The MRDT pages render the operation history as a git-style commit DAG and let you do three-way merges over the lowest common ancestor. None of this is load-bearing for the verification story, but it is a more direct way to see what the data types actually do than reading the proofs.

Closing

Agents have made proof-engineering noticeably cheaper. Spec engineering still sits with the human author, and SPOTs, executable oracles, and the multi-modal Lean stack each approach that from a different direction. None of them is sufficient on its own; in combination they look like a workable methodology.

The slides are here, the repo is at https://github.com/fplaunchpad/sal, and the question I closed the talk with was: what will you prove next?

Foundations for hacking on OCaml

2025-11-10T10:35:00+00:00

How do you acquire the fundamental computer skills to hack on a complex systems project like OCaml? What’s missing and how do you go about bridging the gap?

There are many fundamental systems skills that go into working on a language like OCaml that only come with soaking in systems programming. By systems programming, I mean the ability to use tools like the command-line, editors, version control, build systems, compilers, debuggers, bash scripting, and so on. This is often something that one takes for granted when working on such projects, but is often inscrutable for new contributors, who may not have had the opportunity to develop these skills.

I struggle with this in my own research group. Students approach me to work on the OCaml compiler because they have studied OS, Compilers and Computer Architecture in class. But once they understand that working on OCaml involves actually hacking on systems, they are often lost. How do you build the compiler from source? How do you manage your changes? Do I have to build the entire compiler if I make a small change in the runtime system? The compiler crashes with a segfault – how do I debug it? Worse, the students do not even know what questions to ask, and come back with “This is all new to me, I don’t know where to begin. ChatGPT doesn’t help.”

The CS education in India often lacks a focus on these practical systems skills, which can make it challenging for new contributors to get involved in systems programming. Looking at my own past, my undergraduate CS education, like many others in India (and potentially elsewhere), had mandatory OS and Compiler Construction courses. But neither had a dedicated lab component. It is natural that these theoretical courses do not prepare the students for the practical aspects of systems programming.

I was privileged to have a computer at my school, an IBM PC AT Model 5170 and later an IBM PC 340, and surprisingly, had an education where I got to do programming from a very young age. There was lots of BASIC programming but also just tinkering with the system, learning how to use DOS, and later Windows 3.1, 95, and of course playing games (Doom and Prince of Persia, mostly). This early exposure to computers and systems programming gave me a head start. Many students, especially those from less privileged backgrounds, do not have this early exposure. They may have learned some programming, but not had the time to tinker with systems for extended periods of time.

This challenge of bridging the gap between theoretical CS education and practical systems programming skills is a common one faced by professors working in the broad systems area. The problem is compounded by the fact that these skills are difficult to teach in a traditional classroom setting—they require hands-on experience, experimentation, and often many hours of frustration and debugging. These are skills that come from doing, not from reading or watching lectures. I would be curious to hear from others about their experiences and how they have addressed this challenge.

That said, there are resources available online that can help new contributors acquire these skills. This list is biased to the areas of the compiler that I work on. I mainly work on the backend and the runtime system. The only reason I usually touch the frontend is to lower the features that I care about to the backend. Here are some I have found useful for working on the OCaml compiler:

Systems programming
- Course: MIT Missing Semester: This is a fantastic resource that covers a wide range of topics related to systems programming, including command-line tools, version control, editors, and more. The course is available online for free and includes video lectures, notes, and exercises. I encourage you to read the motivation for this course.
- Course: Stanford CS45: CS45 is an extended version of the MIT course, and delves into the topics in more detail.
- Video: CppCon 2015: Greg Law “Give me 15 minutes & I’ll change your view of GDB”: The talk explores GDB’s less-known features and sheds light on some advanced debugging techniques.
- Tool: rr - Lightweight Recording and Deterministic Debugging: rr is a powerful tool for recording and replaying program execution, which can be invaluable for debugging complex issues in systems programming. I’ve stopped using gdb directly for anything non-trivial and have switched to rr.
OCaml
- Course: CS3100 Paradigms of Programming: The course covers a significant chunk of the OCaml language. You should be able to self-study the course to get a good understanding of the language. That said, the course deliberately does not cover the build system (dune), package manager (opam), command-line tools for the compiler (ocamlc, ocamlopt), editor integration (merlin, ocaml-lsp, ocamlformat), etc.
- Book: Real World OCaml: The book has a section on the compiler and the runtime system, which gives a great overview of the memory representation, garbage collection, and other aspects of the runtime system.
Diving deeper
- Book: Systems Performance: Enterprise and the Cloud, 2nd Edition: This book provides an in-depth look at systems performance, covering topics such as CPU architecture, memory hierarchy, storage systems, and networking. It is a valuable resource for understanding the underlying principles of systems programming and performance optimization.
- Book: The Garbage Collection Handbook: This book offers a comprehensive overview of garbage collection techniques, algorithms, and implementations. It is an essential resource for understanding memory management in programming languages like OCaml.
- Book: The Art of Multiprocessor Programming: This book provides a deep dive into concurrent programming and synchronization techniques, which are crucial for understanding multi-threaded runtime systems like OCaml 5’s multicore runtime and the programming model.

I will probably keep editing this post as I find more resources. If you have suggestions for other useful resources or experiences to share, please feel free to reach out to me.

Testing x-ocaml, OCaml notebooks as a WebComponent

2025-06-20T10:00:00+00:00

Can we have OCaml notebooks as pure client-side code? Can these notebooks have rich editor support (highlighting, formatting, types on hover, autocompletion, inline diagnostics, etc.)? Can you take packages from OPAM and use them in these notebooks?

The answer to all of these turns out to be a resounding yes thanks for x-ocaml. This post is my experiment playing with x-ocaml and integrating that into this blog.

The most wonderful thing about programming is that it lets you experiment freely. You can try out an idea, get instant feedback, and learn by doing—much like playing with Lego bricks or sketching on a canvas. The two main courses that I teach at IITM, CS3100 and CS6225, both involve me live-coding during every lecture. However, blogging about OCaml where the code is static and non-interactive always felt a bit unsatisfying.

Enter x-ocaml, which allows for a way to embed OCaml notebooks into any webpage thanks to WebComponents. All you need to do is to load some JavaScript in your webpage and you can start embedding code cells using <x-ocaml> tag. The snippet below:

<x-ocaml>
print_endline "Hello, world"
</x-ocaml>

renders to:

print_endline "Hello, world"

The code is interpreted in the browser thanks to the OCaml interpreter compiled to JavaScript through the Js_of_ocaml compiler. There is also support for Merlin and OCamlformat in the code editor. Try hovering over the functions and writing some code. You should see inferred types and auto-completion suggestions. It turns out that this solution integrates well with Jekyll, which is what I use for this blog.

Reverse-mode AD using Effects

Js_of_ocaml also supports effect handlers. Here’s the implementation of reverse-mode algorithmic differentiation, implemented using effect handlers running in the browser.

open Effect open Effect.Deep module F : sig type t val mk : float -> t val ( +. ) : t -> t -> t val ( *. ) : t -> t -> t val grad : (t -> t) -> float -> float val grad2 : (t * t -> t) -> float * float -> float * float end = struct type t = { v : float; mutable d : float } let mk v = { v; d = 0.0 } type _ eff += Add : t * t -> t eff type _ eff += Mult : t * t -> t eff let run f = ignore (match f () with | r -> r.d <- 1.0; r; | effect (Add(a,b)), k -> let x = {v = a.v +. b.v; d = 0.0} in ignore (continue k x); a.d <- a.d +. x.d; b.d <- b.d +. x.d; x | effect (Mult(a,b)), k -> let x = {v = a.v *. b.v; d = 0.0} in ignore (continue k x); a.d <- a.d +. (b.v *. x.d); b.d <- b.d +. (a.v *. x.d); x) let grad f x = let x = mk x in run (fun () -> f x); x.d let grad2 f (x, y) = let x, y = (mk x, mk y) in run (fun () -> f (x, y)); (x.d, y.d) let ( +. ) a b = perform (Add (a, b)) let ( *. ) a b = perform (Mult (a, b)) end

Here are some tests.

(* XXX KC: `assert` from standard library doesn't work. Why? *) let assert' c = if not c then raise (Failure "assertion failed!") let test1 = (* f = x + x^3 => df/dx = 1 + 3 * x^2 *) for x = 0 to 10 do let x = float_of_int x in let d1 = F.(grad (fun x -> x +. (x *. x *. x)) x) in let d2 = 1.0 +. (3.0 *. x *. x) in Printf.printf "%f %f\n" d1 d2; assert' ( d1 = d2 ) done let test2 = (* f = x^2 + x^3 => df/dx = 2*x + 3 * x^2 *) for x = 0 to 10 do let x = float_of_int x in assert' ( F.(grad (fun x -> (x *. x) +. (x *. x *. x)) x) = (2.0 *. x) +. (3.0 *. x *. x)) done let test3 = (* f = x^2 * y^4 => df/dx = 2 * x * y^4 df/dy = 4 * x^2 * y^3 *) for x = 0 to 10 do for y = 0 to 10 do let x = float_of_int x in let y = float_of_int y in assert' ( F.(grad2 (fun (x, y) -> x *. x *. y *. y *. y *. y) (x, y)) = (2.0 *. x *. y *. y *. y *. y, 4.0 *. x *. x *. y *. y *. y)) done done

Using other libraries

x-ocaml also supports loading any js_of_ocaml compatible library into the webpage. Let’s use digestif.

For any library that you want to export, install the library using opam. x-ocaml provide a command-line utility to export the library.

$ x-ocaml --effects digestif.ocaml -o digestif.js

This produces the JavaScript artifact that can be used in the webpage. It may be instructive to look at the source of this post to see how the compiler and the libraries are integrated into this blog post. There is a little script at the top of the file:

<script async
  src="{{ '/assets/x-ocaml.js' | absolute_url }}"
  src-worker="{{ '/assets/x-ocaml.worker+effects.js' | absolute_url }}"
  src-load="{{ '/assets/digestif.js' | absolute_url }}"
></script>

let hash = Digestif.MD5.(digest_string "hello" |> to_hex)

What next?

There is a number of rough edges to x-ocaml. This is expected since this project appears to be one of Arthur’s hacking expeditions (which, as usual, is pushing the state of the art forward).

It would be fun to use this for teaching CS3100 and also other OCaml tutorials. Perhaps even have an interactive version of Real World OCaml book.

Not all OCaml libraries can be compiled to JavaScript. The common reason being that they may depend on features not available on JavaScript. In writing this post, I unsuccessfully tried for a long time to get mirage-cypto working. mirage-crypto has a large C dependency, which does not work with Js_of_ocaml. Js_of_ocaml promises to take any opam library installed on your opam switch and compiles that to JavaScript. However, at that point, we’re really cross compiling the opam packages installed on your switch to JavaScript since the installed package may make some assumptions about the platform that it is supposed to run on. Hence, JavaScript compilation of arbitrary OCaml packages is unlikely to work in the general case. Unfortunately, the error was difficult to debug since the failure was at runtime, and was not apparent in the error messages (at least for me, who has little JavaScript experience). It would be nice to have the opam packages explicitly say whether they are JavaScript compatible, and have build tooling that reports errors like these early.

Linearity and uniqueness

2025-06-04T10:00:00+00:00

In the last post, we looked at uniqueness mode and how uniqueness may be used to optimise. As we will see, uniqueness alone is insufficient in practice, and we also need a concept of linearity for uniqueness to be useful.

Capturing unique values

Let’s start with an example. Recall the signature of the unique reference module.

module type Unique_ref = sig
  type 'a t
  val alloc : 'a -> 'a t @ unique
  val free : 'a t @ unique -> unit
  val get : 'a t @ unique -> 'a Modes.Aliased.t * 'a t @ unique
  val set : 'a t @ unique -> 'a -> 'a t @ unique
end

Assume that we also have an implementation of the module:

module Unique_ref : Unique_ref

Consider the following example, which works fine:

let works () =
  let t = alloc 42 in (* Allocate a unique reference *)
  free t (* free it *)

Now consider this modified example:

let wat () =
  let t = alloc 42 in (* Allocate a unique reference *)
  let f () = free t in (* capture free in a closure *)
  f (); (* free it *)
  f () (* free it again??? *)

Observe that f has captured t in the closure, and when called frees t. It should be clear that calling f more than once is bad – leads to a double-free issue! What property do we want of f? Uniqueness is insufficient; we have a unique reference to f in this program, with which we call f twice.

What we want to enforce is that f can be called at most once. The compiler has a linearity mode which captures the idea of how many times a value can be used. We have two modes in the linearity axis – once, which stands for “at most once” and many (the default one for all values), which allows values to be used arbitrary number of times.

Whenever a unique value is captured by a closure, the closure gets a once mode, which allows the closure to be called at most once. This program rightly gets rejected by the compiler.

File "./unique_ref.ml", line 32, characters 2-3:
32 |   f () (* free it again??? *)
       ^
Error: This value is used here,
       but it is defined as once and has already been used:
File "./unique_ref.ml", line 31, characters 2-3:
31 |   f (); (* free it *)
       ^

A linear ref

Now, one might wonder whether the unique reference that we’ve implemented may be implemented with the linear mode. The answer is yes.

module type Linear_ref = sig
  type 'a t
  val alloc : 'a -> 'a t @ once
  val free : 'a t @ once -> unit
  val get : 'a t @ once -> 'a * 'a t @ once
  val set : 'a t @ once -> 'a -> 'a t @ once
end

module Linear_ref : Linear_ref = struct
  type 'a t = { mutable value : 'a }
  let alloc x = { value = x }
  let free t = ()
  let get t =
    t.value, t
  let set t x =
    t.value <- x;
    t
end

This works as expected:

open Linear_ref

let works () =
  let r = alloc 42 in
  let v,r = get r in
  let r = set r (v + 1) in
  let v,r = get r in
  print_int v;
  free r;
  ()

let fails () =
  let r = alloc 42 in
  free r;
  get r (* fails here *)

with the error message:

File "./linear_ref.ml", line 34, characters 6-7:
34 |   get r (* fails here *)
           ^
Error: This value is used here,
       but it is defined as once and has already been used:
File "./linear_ref.ml", line 33, characters 7-8:
33 |   free r;

Why both linearity and uniqueness?

Given this example, you might be wondering, if the safe reference may be implemented equivalently using both uniqueness and linearity, why do we need both? Obviously, there’s something interesting going on where unique values captured in a closure needs linearity. Does that mean linearity is sufficient?

It turns out that only recently was the relationship between the two formally studied in the same type system. While linear types and uniqueness types have a long history of being studied independently, Marshall et al. in their paper, “Linearity and Uniqueness: An Entente Cordiale”, present the ideas in the same type system. They provide some key insights.

The first insight is that

in a setting where all values must be linear, we can also guarantee that every value is unique, and vice versa! Intuitively, if it is never possible to duplicate a value, then it will never be possible for said value to have multiple references.

In our Unique_ref and Linear_ref every operation that operates on the ref requires uniqueness or linearity, respectively. Hence, they seem almost equivalent in expressive power.

It is when we also have the ability for unrestricted use (non-linear/non-unique) that differences between linearity and uniqueness begin to arise, as we will soon see.

In our language, we do have the ability for unrestricted use. That is, in the linearity axis, many is the default mode attributed to all the values not tagged or inferred as once. Similarly, aliased is the default mode attributed to all the values not tagged or inferred as unique.

The type system has submoding: values may move freely to greater modes (which typically restrict what can be done with those values) but not to lesser modes. For example, a many value may be safely use in a context where a once value is expected.

let works () =
  let set_to_20 (r @ once) =
    r := 20
  in
  let r @ many = ref 10 in
  set_to_20 r (* [r @ many] is passed to a function that expects [int ref @ once] *)

Similarly, you can use a unique value in a context where an aliased value is expected.

let dup r = r,r

let works () =
  let r = Unique_ref.alloc 42 in
  let a,b = dup r in
  a,b

The type of the works function is val works : unit -> int Unique_ref.t * int Unique_ref.t, which crucially lacks the fact that the references are at unique mode. We can’t call any functions from the Unique_ref module with these references, all of which expect a reference with unique mode.

Uniqueness is more appropriate for safe refs

In our running example of implementing a safe ref, it turns out that uniqueness is more appropriate. Consider the type signature of free in Unique_ref:

val free : 'a t @ unique -> unit

The type signature says that there are no other aliases to this reference. Hence, its memory may be safely deallocated. However, consider the Linear_ref.free signature:

val free : 'a t @ once -> unit

The signature says that this reference must be used at most once. In particular, just by looking at the signature, we cannot conclude that there are no other aliases to this reference. But we know that the API is safe since the only way to create a safe reference is through the alloc function, which returns a once-usable reference, and every other operation also expects and returns a once-usable reference.

The correctness of the linear version depends on reasoning over the whole API, whereas the unique version can be concluded to be safe just by looking at the signature of the free function. This modular reasoning makes uniqueness more appropriate for our safe reference API.

Past and the future

In a sense, uniqueness and linearity are duals of each other. Uniqueness talks about the past – whether a value may be aliased in the past. It is okay to alias a unique value in the future and lose the uniqueness mode. Linearity talks about the future – whether a value may be used more than once in the future. You can take any value and ascribe a linear mode to it, restricting its use in the future. However, there may be other aliases to this value in the past.

Conclusions

The code examples are available here. Section 2.1 of Marshall et al.’s paper is quite readable and explains the distinction between linearity and uniqueness with some historical context. I highly recommend it.

Acknowledgements

Thanks to Richard Eisenberg for the discussions which spurred this post.

Uniqueness for Behavioural Types

2025-05-29T17:56:00+00:00

Jane Street has been developing modal types for OCaml – an extension to the type system where modes track properties of values, such as their scope, thread sharing, and aliasing. These modes restrict which operations are permitted on values, enabling safer and more efficient systems programming. In this post, I focus on the uniqueness mode, which tracks aliasing, and show how it can eliminate certain runtime checks.

My intention in this post is not to explain how the different modes work. There are a number of blog posts and academic papers written about modes. I recommend the interested reader to have look at them. The following table summarizes the main properties tracked by modes, the corresponding mode names, and resources for further reading:

Property	Modes	Resources
Scope	Locality	Blog, Paper
Sharing between threads	Portability, Contention	Blog, Paper
Aliasing	Uniqueness, Linearity	Blog, Paper

The OCaml compiler extended with modes is developed in the open, and is used in production at Jane Street. The repo also has some documentation of the extensions.

Be warned that the compiler and the language features are fast evolving. The code examples presented in the blog and the paper referenced above are likely not to work. I expect the same for the code examples in this post in the near future, but that’s what one should expect with these bleeding-edge features.

Behavioural types and runtime overhead

A couple of years ago, I wrote a post on behavioural types where the types capture the sequence of operations that may be performed on the values with those types. The correctness of the system depended on the linear use of the resources. Since OCaml does not provide support for enforcing linearity statically, the implementation uses a dynamic check, using a fresh ref cell that gets consumed every time the type state changes. If we are guaranteed that the resource is not aliased statically, then there’s no need for the dynamic check. This is where uniqueness helps.

Uniqueness mode allows the OCaml compiler to statically guarantee that certain values are not aliased. This enables optimizations and eliminates the need for some runtime checks, which is particularly valuable in systems programming for ensuring memory safety and efficient resource management.

Setting up OCaml with modes

An opam repository with the modes extensions and packages supporting modes is available here. Here’s how you can set up the new compiler:

# this will take time
opam switch create 5.2.0+flambda2 --repos with-extensions=git+https://github.com/janestreet/opam-repository.git#with-extensions,default
eval $(opam env --switch 5.2.0+flambda2)

An explicitly memory-managed reference

Suppose you want to implement a mutable reference whose memory is explicitly managed (not managed by the GC), you may go for the following interface:

module type S = sig
  type 'a t
  val alloc : 'a -> 'a t
  val free : 'a t -> unit (* unsafe *)
  val get : 'a t -> 'a
  val set : 'a t -> 'a -> unit
end

This interface provides an explicit free, which releases the memory associated with this reference. This opens up the possibility of memory safety bugs such as use-after-free and double-free. We can use uniqueness modality to get a safe API. Here’s the interface:

module type S = sig
  type 'a t
  val alloc : 'a -> 'a t @ unique
  val free : 'a t @ unique -> unit
  val get : 'a t @ unique -> 'a Modes.Aliased.t * 'a t @ unique
  val set : 'a t @ unique -> 'a -> 'a t @ unique
end

The unique annotation states that the value is not aliased. The operations on the reference expect that this reference is not aliased. Observe that get and set take in the unique reference and also return them unlike the original interface. You can use this like so:

# let okay r =
    let v, r = get r in
    let r = set r 20 in
    free r;;
val okay : int M.t @ unique -> unit = <fun>

The key bit is that free consumes the unique reference; you can no longer produce a unique handle to the same reference and hence, you cannot call free, get or set on this reference which has been freed.

# let wont_work r =
    free r;
    get r
  ;;
Error: This value is used here, but it has already been used as unique:
Line 2, characters 7-8:

Modes.Aliased.t

Uniqueness applies deeply. If a value is marked as unique, then the transitive closure of the reachable parts of the object is also expected to be unique. The return value of get is a pair, which is marked as unique¹. Hence, both the components of the pair are expected to be unique. However, we don’t want to impose uniqueness of the value stored in the reference. The language allows parts of the value to be marked as aliased. Modes.Aliased.t is defined as:

module Aliased : sig
  type 'a t = { aliased : 'a @@ aliased } [@@unboxed]
end

The language allows record fields to be annotated as aliased, while the record itself may be uniquely referenced.

Implementation

Here’s an implementation of that satisfies the signature.

module M : S = struct
  type 'a t = { mutable value : 'a }
  let alloc x = { value = x }
  let free t = ()
  let get t =
    let a = Modes.Aliased.{aliased = t.value } in
    a, t
  let set t x =
    t.value <- x;
    t
end

There’s nothing surprising about this implementation. Note that the compiler is doing a lot of work behind the scenes to ensure that the functions do in fact satisfy the uniqueness requirements. For example, if you change the implementation of set to do something innocuous where the compiler cannot prove that the value is not aliased, the program no longer compiles:

# let set t x =
    t.value <- x;
    let t' = Fun.id t in (* compiler cannot prove [t'] is not aliased *)
    t'
Error: <snip>
Values do not match:
 val set : 'a t -> 'a -> 'a t
is not included in
 val set : 'a t @ unique -> 'a -> 'a t @ unique
The type 'a t -> 'a -> 'a t is not compatible with the type
 'a t @ unique -> 'a -> 'a t @ unique

Refs that explain their work

The earlier blog post used polymorphic variants to encode the protocol of operations that are permitted on a ref cell. The implementation is reproduced below:

module type Ref =
sig
  type ('a, 'b) ref constraint 'b = [>]

  val ref   : 'a -> ('a, 'b) ref
  val read  : ('a, [`Read of 'b]) ref
              -> 'a * ('a, 'b) ref
  val write : ('a, [`Write of 'b]) ref
              -> 'a
              -> ('a, 'b) ref
end
module Ref : Ref =
struct

  type ('a, 'b) ref =
    {contents     : 'a;
     mutable live : bool} (* For linearity *)
     constraint 'b = [>]

  let ref v = {contents = v; live = true}

  let check r =
    if not r.live then raise LinearityViolation;
    r.live <- false

  let fresh r = {r with live = true}

  let read r =
    check r;
    (r.contents, fresh r)

  let write r v =
    check r;
    { contents = v; live = true }

  let branch r _ = check r; fresh r
end

Observe that we use a dynamic check to enforce linearity. It requires a fresh ref cell for each operation performed on this reference. With uniqueness, we can enforce this statically, avoiding the dynamic check and the fresh ref cell requirement.

module type Ref =
sig
  type ('a, 'b) ref constraint 'b = [>]
  (* 'b is the behavioural type variable *)

  val ref   : 'a -> ('a, 'b) ref @ unique
  val read  : ('a, [`Read of 'b]) ref @ unique
              -> 'a Modes.Aliased.t * ('a, 'b) ref @ unique
  val write : ('a, [`Write of 'b]) ref @ unique
              -> 'a
              -> ('a, 'b) ref @ unique
  val branch : ('a, [>] as 'b) ref @ unique
               -> (('a, [>] as 'c) ref @ unique -> 'b)
               -> ('a, 'c) ref @ unique
end

module Ref : Ref =
struct
  type ('a, 'b) ref = {mutable contents : 'a} constraint 'b = [>]

  let ref v = {contents = v}

  let read r =
    let c = Modes.Aliased.{aliased = r.contents} in
    c, Obj.magic_at_unique r

  let write r v =
    r.contents <- v;
    Obj.magic_at_unique r

  let branch r _ = Obj.magic_at_unique r
end

The only changes necessary in the signature were a number of uniqueness and aliasing annotations. Notice that the implementation no longer needs the dynamic check! Obj.magic_at_unique has the type 'a @ unique -> 'b @ unique, and is the version of Obj.magic with uniqueness annotation. We use it to advance the protocol type state.

Where next

The rest of the examples in the original post should also benefit from uniqueness annotations to remove the runtime overheads.

The complete code examples are available here. You can also play with the code examples directly in the browser thanks to Patrick Ferris’ OCaml with extensions js_of_ocaml top-level.

Since the modes features are constantly evolving, there are no stability guarantees yet. However, I’m excited about the possibility of modes improving how we do safe systems programming in OCaml.

Addendum

Looks like there’s a part 2 of this post.

Footnotes

Unclear whether it is possible to return a pair where one of the components is unique, but the other one is not. ↩

Joining my group

2025-04-28T12:10:00+00:00

Recently, I posted on X and LinkedIn that I am always looking for excellent people to join my group. I received a lot of enquiries, some of which led to internship hires (yay!). But mostly, I seemed to offer similar advice. I thought I’d write a post that summarise my responses.

At IIT Madras, my research group develops programming language abstractions to solve systems problems. The group is composed of research associates (fixed-term project staff), PhD, MS and MTech students, undergraduate research students (who are typically BTech students from IIT Madars) and interns. I made the following post a few weeks ago, for which I received a lots of enquiries, and I have been busy writing similar responses to many of them, which I summarise below.

PSA: I'm always looking for excellent folks to join my research group at IIT Madras to work on building "functional" systems. This includes internships, MS and PhD studentships, research staff positions, and post-baccalaureate fellowships.

Reach out to me if you are keen!
— KC Sivaramakrishnan (@kc_srk) April 15, 2025

Internship positions

Internship enquiries are the most frequent ones that I receive. Here’s how you can make it work. Please do go through my web page to look at what areas I work on. Write to me about what interests you and what your goals are.

My group works on systems. To make the internship work well, we require that you have demonstrable systems building experience. Do build projects that go beyond your coursework. Make sure that the projects are developed publicly on GitHub or other similar platforms so that one can take a look at what you’ve built. Even better is contributions to other open-source projects.

The group solves systems problems with functional programming. If you have prior experience with functional programming, such as building small projects with OCaml, Haskell, Scala, Scheme or other languages, it is easier for me to assess your interest. That said, if you are great at any programming language, having built non-trivial projects in any language, then you have the right skills for internships in my group. Generally, I expect the interns to have done course work on OS, compilers and computer architecture. Significant projects in any of those areas is a huge plus.

I should clarify that my recommendation letters for graduate programs will reflect my honest assessment of the internship. I will decline writing a recommendation letter if I think I may not be able to provide a strong one.

I do not work on projects that are primarily AI/ML or Web Development. If you write to me looking for projects in those areas, it is very likely that you won’t hear from me. Please don’t bulk email faculty CCing or BCCing everyone in the department. It is likely that no one will read such an email.

PhD/MS/MTech positions

For academic positions, please have a look at https://research.iitm.ac.in/. There are alternative ways to enter MS and PhD positions by being a reserach associate and completing some coursework at IITM. For more information, see here.

Contributing to the OCaml community

A significant chunk of the enquiries were from folks who hold full-time positions looking to be involved in the research group. Unfortunately, making part-time positions work is a challenge for both sides. I would encourage contributions to the wider OCaml community.

There are several great ways to get involved with the community. Here’s what I usually recommend.

Learn the basics.
- Go through the OCaml part of my CS3100 course. The course has a YouTube playlist and programming assignments. Complete the programming assignments.
- Read the Real World OCaml book.
- There are lots of other resources at OCaml.org, the official website of the OCaml community and the ecosystem.
Join the community.
- OCaml discord and discuss are great places to hang out with other OCaml folks and ask questions.
- Discord is better for quick clarifications and discuss for longer form discussions.
Look for “good first issues” in the OCaml projects and work on them
- Check out the core platform tools under the OCaml github org. See OCaml compiler, dune build system, opam package manager, ocaml.org, etc.
- Across the wider ecosystem – SemGrep, OpenGrep, Rocq, etc.
Work on self-directed projects. Here is my list of ideas.

OCaml community also participates in Outreachy internships. Outreachy internships are paid internships for underrepresented groups. It is a great way to contribute to the community while being mentored by folks from the OCaml community. Here’s a nice intro (in Tamil) to the impact that Outreachy program had on an Outreachy intern. Look out for announcements about Outreachy internships in the OCaml discuss forum.

Research Associate positions

This is for folks who want to contribute to the core research programme but do not see themselves joining academic programs. The expectation here is that you are an experiened systems engineer, who should see themselves easily qualifying for the internship positions in the group.

One useful way to look at this position is similar to a research software development engineer who helps build out the systems used for research or translate research to practice. In the past, research associates have helped upstream multicore OCaml. The easiest way to get into this role would be to do an internship, see whether you like this area, do well in the internship and then choose to apply to research associate position.

Another variant is a post-bacc or a pre-doc position aimed at highly motivated recent graduates, who are looking to build research experience. The expectation here is that we get papers into top venues in PL and Systems. For such students, I recommend going through my CS6225 Programs and Proofs course, watch the video lectures and complete the assignments. The course is not an easy one, but will expose you to the broad area of PL and specifically to deductive program verification. At the very least, you will come out with an understanding of what it is to think rigorously about program correctness.

Research associate positions are fixed-term positions. In order to make this work, the tenure should be at least 18 months to make it work.

Summary

While I may not be hiring actively all the time, do reach out to me if you are interested in any of hte above. Please follow me on LinkedIn, X or Bluesky, where I am likely to announce any open positions.

Off-CPU-time analysis

2024-07-24T09:48:00+00:00

Off-CPU analysis is where the program behavior when it is not running is recorded and analysed. See Brendan Gregg’s eBPF based off-CPU analysis. While on-CPU performance monitoring tools such as perf give you an idea of where the program is actively spending its time, they won’t tell you where the program is spending time blocked waiting for an action. Off-CPU analysis reveals information about where the program is spending time passively.

Installation

Install the tools from https://github.com/iovisor/bcc/.

Enabling frame pointers

The off-CPU stack trace collection, offcputime-bpfcc, requires the programs to be compiled with frame pointers for full backtraces.

OCaml

For OCaml, you’ll need a compiler variant with frame pointers enabled. If you are installing a released compiler using opam, you can create one the following switch command opam switch create 5.2.0+fp 5.2.0 ocaml-option-fp. Change out 5.2.0 for your preferred OCaml version.

Instead, if you are building the OCaml compiler from source, configure the compiler with --enable-frame-pointers option:

$ ./configure --enable-frame-pointers

Lastly, there is an option to create an opam switch with the development branch of the compiler. The instructions are in ocaml/HACKING.adoc. In order to create an opam switch from the current working directory, do:

$ opam switch create . 'ocaml-option-fp' --working-dir

glibc

The libc is not compiled with frame pointers by default. This will lead to many truncated stack traces. On Ubuntu, I did the following to get a glibc with frame pointers enabled:

Install glibc with frame pointers
```
$ sudo apt install libc6-prof
```

LD_PRELOAD the glibc with frame pointers

$ LD_PRELOAD=/lib/libc6-prof/x86_64-linux-gnu/libc.so.6 ./myapp.exe

Running

On one terminal run the program that you want to analyze:

$ LD_PRELOAD=/lib/libc6-prof/x86_64-linux-gnu/libc.so.6 ./ocamlfoo.exe

On another terminal run offcputime-bpfcc tool:

$ sudo offcputime-bpfcc --stack-storage-size 2097152 -p $(pgrep -f ocamlfoo.exe) 10 > offcputime.out

The command instruments the watches for 10s and the writes out the stack traces corresponding to blocking calls in offcputime.out. We use a large stack storage size argument so as to not lose stack traces. Otherwise, you will see many [Missing User Stack] errors in the back traces.

Caveats

offcputime-bpfcc must run longer than the program being instrumented by a few seconds so that the function symbols are resolved. Otherwise you may see [unknown] in the backtrace for function names.

Oddities

I still see an order of magnitude difference between the maximum pauses observed using offcputime-bpfcc and olly trace. Something is off.

Getting Started with GDB on OCaml

2024-01-20T15:16:00+00:00

A number of folks who regularly use OCaml were surprised to learn that you can reasonably debug OCaml programs using gdb. The aim of the post is to show the first steps in using gdb on OCaml programs.

Let’s consider the following program:

(* fib.ml *)
let rec fib n = 
  if n = 0 then 0
  else if n = 1 then 1
  else fib (n-1) + fib (n-2)

let main () = 
  let r = fib 20 in 
  Printf.printf "fib(20) = %d" r

let _ = main ()

Let’s compile this program. I’m using OCaml version 5.1.1.

$ ocamlopt --version
5.1.1
$ ocamlopt -g -o fib.exe fib.ml
$ $ ./fib.exe 20
fib(20) = 6765

As you can see, the program prints the 20th Fibonacci number. Let’s examine this program under gdb. Before we venture any further, I highly recommend watching this 15-minute video that shows a number of gdb tricks. Let’s start a gdb session.

$ gdb ./fib.exe

Setting breakpoints

Let’s set a break point at the fib function. When OCaml functions are compiled, their names are mangled. OCaml 5.1.1 uses the following mangling scheme caml<MODULE_NAME>.<FUNCTION_NAME>_<NNN> where NNN is a randomly generated number. For the fib function, since it is under the file fib.ml, the module name is Fib. Since we can’t guess NNN, we use tab completion to help identify the function.

(gdb) break camlFib.fib_ #press tab
(gdb) break camlFib.fib_269 #269 happens to be the randomly generated number
                            #on my machine.
(gdb) Breakpoint 1 at 0x3d160: file fib.ml, line 1.

You can also set a breakpoint using gdb’s file name and line number combination. Let’s set another break point at the main function, which is at line number 6 in fib.ml.

(gdb) break fib.ml:6
Breakpoint 2 at 0x3d1d0: file fib.ml, line 6.

Let’s run the program.

(gdb) r
Starting program: /home/kc/temp/fib.exe 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 2, camlFib.main_271 () at fib.ml:6
6       let main () =

The program execution starts in the gdb session and we stop at the breakpoint installed at main. gdb has a nice TUI mode for stepping through the file. This can be activated with ctrl+x+a key combination, which should show a screen similar to the following.

Notice that we can see both the breakpoints installed in this file. The current line is highlighted.

Examining the stack

You can step through the OCaml program with gdb commands n and s. After a few ns, you can examine the backtrace using the bt command.

(gdb) bt
#0  camlFib.fib_269 () at fib.ml:1
#1  0x00005555555911a1 in camlFib.fib_269 () at fib.ml:4
#2  0x00005555555911a1 in camlFib.fib_269 () at fib.ml:4
#3  0x00005555555911a1 in camlFib.fib_269 () at fib.ml:4
#4  0x00005555555911a1 in camlFib.fib_269 () at fib.ml:4
#5  0x00005555555911f1 in camlFib.main_271 () at fib.ml:7
#6  0x000055555559129a in camlFib.entry () at fib.ml:10
#7  0x000055555558eb0b in caml_program ()
#8  <signal handler called>
#9  0x00005555555dd306 in caml_startup_common (pooling=<optimised out>, argv=0x7fffffffe008) at runtime/startup_nat.c:132
#10 caml_startup_common (argv=0x7fffffffe008, pooling=<optimised out>) at runtime/startup_nat.c:88
#11 0x00005555555dd37f in caml_startup_exn (argv=<optimised out>) at runtime/startup_nat.c:139
#12 caml_startup (argv=<optimised out>) at runtime/startup_nat.c:144
#13 caml_main (argv=<optimised out>) at runtime/startup_nat.c:151
#14 0x000055555558e8f2 in main (argc=<optimised out>, argv=<optimised out>) at runtime/main.c:37

As you can see the backtrace includes the recursive calls to the fib function, the main function in fib.ml, followed by a number of functions from the OCaml runtime, and finally ending at the main function.

Note that <signal handler called> is a misnomer and is not an actual signal handler. OCaml 5 supports effect handlers with the help of runtime managed stack segments for the OCaml stack. There is also a single C stack that is used by all the fibers that run on a domain, our unit of parallelism. The <signal handler called> represents a frame where the control switches between the C stack (managed by the OS) and the OCaml stack (managed by the OCaml runtime). The OCaml runtime marks these frames where the stack are split as signal handler frames so that gdb doesn’t complain about stack corruption; gdb expects stacks to grow down, which may not be true if the stack segments are in different parts of the memory address space. You will also find such <signal handler called> frames between OCaml fibers (when using effect handlers) and when OCaml calls into the (C) runtime. You can find more details about the stack layout in the PLDI 2021 paper on OCaml effect handlers.

Examining values

There isn’t good support for examining OCaml values in gdb unlike C. That said, given the uniform value representation of OCaml, with a bit of information about the OCaml calling convention, we can start to examine the values. It is useful to note that OCaml 5.1.1 on x86 passes the first 10 arguments in registers. In particular, the first argument is in the register rax. So the argument to the fib function should be in the rax register. We also know that the argument to fib is an integer. OCaml uses 63-bit tagged integers (on 64-bit machines) with the least-significant bit is 1. Given a machine word or a register holding an OCaml integer, the integer value is obtained by right shifting the value by 1.

Putting it all together, we can get the argument value of fib at the breakpoint at the entry to fib as follows:

(gdb) p $rax >> 1
$2 = 12

Given that we’ve already stepped through the program several times, the current call for me corresponds to fib(12). Let’s see what’s the next argument by continuing the program until we hit the breakpoint again.

(gdb) c
Continuing.

Breakpoint 1, camlFib.fib_269 () at fib.ml:1
(gdb) p $rax >> 1
$3 = 10

Observe that this corresponds to the recursive call fib(10), which must mean that the RHS recursive call is the one being invoked. Note that the evaluation order of arguments in OCaml is unspecified. The 5.1.1 implementation does right-to-left evaluation of arguments (to the (+) function in this case), which can be confirmed with the following program:

$ cat eval_order.ml
let _ =
  (print_endline "hello"; 0) + (print_endline "world"; 1)
$ ocamlopt.opt -g -o eval_order.exe eval_order.ml
$ ./eval_order.exe
world
hello

Advanced printing

As you can observe, examining values this way is cumbersome. The OCaml compiler distribution has some rudimentary scripts to make it easier to examine OCaml values in gdb. Note that this was developed by OCaml maintainers to develop the compiler, and was not designed to serve end user needs. That said, let’s dive in.

Since we are on OCaml 5.1.1, let’s check out the source code for 5.1.1 first.

# I'm in ~/repos directory on my machine *)
$ git clone https://github.com/ocaml/ocaml --branch 5.1.1

Let’s start a new gdb session, load the gdb script and get to the desired breakpoint.

$ gdb ./fib.exe
(gdb) source ~/repos/ocaml/tools/gdb_ocamlrun.py
(gdb) break fib.ml:1
Breakpoint 1 at 0x3d160: file fib.ml, line 1.
(gdb) r
Starting program: /home/kc/temp/fib.exe 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, camlFib.fib_269 () at fib.ml:1
1       let rec fib n =

As earlier, the first argument is in rax register. We can examine the value now with the help of the script.

(gdb) p (value)$rax
$1 = I(20)

value is the type of OCaml values defined in OCaml runtime. The script tools/gdb_ocamlrun.py installs a pretty printer for the values of type value. Here, it prints that the argument is the integer 20.

We can also print other kinds of OCaml values. In order to illustrate this, consider the following program:

$ cat test_blocks.ml
(* test_blocks.ml *)

type t = {s : string; i : int}

let main a b =
  print_endline "Hello, world!";
  print_endline a;
  print_endline b.s

let _ = main "foo" {s = "bar"; i = 42}

Let’s compile, start a gdb session and break at the main function.

$ ocamlopt -g -o test_blocks.exe test_blocks.ml                                                                                                               
$ gdb ./test_blocks.exe
(gdb) break camlTest_blocks.main_272 
Breakpoint 1 at 0x16ed0: file test_blocks.ml, line 5.
(gdb) r
Starting program: /home/kc/temp/test_blocks.exe 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, camlTest_blocks.main_272 () at test_blocks.ml:5
5       let main a b =
(gdb) source ~/repos/ocaml/tools/gdb_ocamlrun.py 

Let’s examine the two arguments to main.

(gdb) p (value)$rax
$1 = String_tag("foo", NOT_MARKABLE)

The first argument is a string “foo”. NOT_MARKABLE is one of the GC colours used by OCaml 5, and represents objects that are not traced by the mark-and-sweep (major) GC. The string happens to be allocated in the data section of the address space, and is not traced by the GC.

(gdb) info symbol $rax
camlTest_blocks.4 in section .data of /home/kc/temp/test_blocks.exe

Let’s examine the second argument.

(gdb) p (value)$rbx
$2 = Block(0, wosize=2, NOT_MARKABLE)

The second argument, which is passed in the register rbx, is a record with two fields. Hence, the pretty printer says that it is a block with 2 fields. We can print both values using gdb’s support for printing a range of values.

(gdb) p *(value*)$rbx@2                                                                 
$3 = {String_tag("bar", NOT_MARKABLE), I(42)}

We cast rbx to an array of values and print the first two fields in the array. This shows that the fields are the string “bar” and integer 42.

More for later

There is a lot more to be said about debugging OCaml programs using gdb. We shall see them in subsequent posts if there is interest.

Teaching OCaml and Prolog through Jupyter Notebooks

2020-01-19T15:16:00+00:00

Last semester at IIT Madras, I taught a revamped core course CS3100 Paradigms of Programming, which introduces 3rd-year students to functional and logic programming paradigms. While the course had been traditionally offered in Lisp and Prolog, I introduced OCaml instead of Lisp. All of the lectures were delivered through interactive Jupyter notebooks. The assignments were also distributed as Jupyter notebooks and evaluated through autograder facility in Jupyter. There has since been several requests to replicate this setup elsewhere. Hence, I thought I should write about the set up and experience of teaching through Jupyter notebooks.

Course Content

Having never taken a functional programming course, there was the question of what I wanted the students to take away from the course. I wanted the course to be a mixture of functional programming concepts (types and lambda calculus) as well as advanced yet pragmatic concepts that one would find in modern functional programming languages (such as GADTs and Monads). The OCaml part of the course is based on the excellent CS3110 from Cornell and AFP from Cambridge Computer Laboratory. In particular, I would highly recommend the CS3110 book for anyone taking first steps into functional programming. Lambda calculus lectures were based on Peter Selinger’s lecture notes on lambda calculus.

The Prolog part of the course were modelled on Prolog lectures from Cambridge Computer Laboratory and the wonderful The Art of Prolog book.

Teaching functional and logic programming in the same course allowed me to develop interesting content that intersected both of the paradigms. In the functional programming part of the lecture, I had introduced simply typed lambda calculus. In the logic part of the course, we developed a type checker for simply typed lambda calculus in Prolog. Merely encoding type checking rules for simply typed lambda calculus in Prolog, type inference with polymorphic types falls out. With a tiny bit of coaxing, Prolog synthesizes programs for the given type. In the last assignment, the students were asked to implement a Prolog interpreter in OCaml. There was indeed some value in teaching multiple paradigms in the same course, not just for a comparative study of strengths and weaknesses, but to be able to teach the students to pick the right tool for the job.

Course Delivery

I had a clear idea that the course will have to be interactive where programs are developed during the lectures. There was the option of using pdf slides and switching to utop for interactive development. But this solution lacked the uniformity that the students would like when reviewing the course materials. Moreover, switching between two mediums made it difficult for me to plan the lectures and was a distraction for the students.

Jupyter Notebooks

Hence, I decided to use Jupyter Notebooks for the course. Jupyter is a collection of open source standards and software for interactive development. Jupyter supports a variety of languages. For OCaml, I used akabe/ocaml-jupyter, an OCaml kernel for Jupyter notebooks. This uses utop, an advanced OCaml top-level in the backend and hence provides excellent interactive top-level support. The situation for Prolog was not so great. Eventually, I zeroed in on targodan/jupyter-swi-prolog but ended up improving the solution a bit kayceesrk/jupyter-swi-prolog (TODO KC: upstream fixes). Jupyter supports mathjax, which allows typesetting LaTeX in the notebooks. This was great for writing the lectures on lambda calculus.

RISE for slideshow

Jupyter notebooks are webpages that mixes text and code. For lectures, I much prefer slides since they let you focus on a particular images, statement or an inference rule. While Jupyter allows the conversion of notebooks to slides out-of-band, RISE is an Jupyter notebook extension that lets turn your Jupyter notebook into a slideshow. Adding RISE to the setup makes the Jupyter experience compatible with traditional slides based lectures.

Course Distribution

Apart from delivering the lectures through the notebooks, I also wanted the students to be able to go through the notebooks and be able to run the snippets. Installing all the required software (OPAM, OCaml, Prolog, Jupyter and its extensions, Jupyter Kernels for OCaml and Prolog) and correctly was not something I wanted the students to go through. I wasn’t even sure if this software combination works on various Mac, Windows and Linux distributions. Hence, everything was packaged as a Docker file, and the latest version of the image uploaded to docker hub. In order to review the course, the students only had to install Docker and Git and run exactly 4 commands.

Docker is generally supported on all major OSes. Packaging up the course content as a docker image and pushing it to dockerhub is insurance against the software combination not working in the next offering of the course; if for some reason one of the dependency does not work next year, I can always fallback to the docker image while I find a fix. One of my TAs ran a tutorial on basic Docker and Git in the first week of the course to ensure that everyone was setup. I would consider Docker and Git as essential tools for modern software development as well as research. After that, the students did not ever have to do anything on the command line.

Assignments

nbgrader is a tool that facilitates creating and grading assignments in Jupyter notebook. It uses language-agnostic logic to identify failing cells, which meant that it was easy to set up nbgrader for OCaml and Prolog. The assignments were released as Jupyter notebooks, which the students filled in and submitted. nbgrader has support for unit tests which allowed the students to get instant feedback as they were developing the solutions.

Wish List

Overall, the students felt that the Jupyter notebooks were better than slidedecks. However, not everything was perfect with the Jupyter notebook based lecturing. Here are some of the things that could be improved.

There is no good diagramming + animation support for Jupyter notebooks. The best I could find was egal whose user interface I did not find intuitive. Even for simple diagrams, it was much more effort making diagrams there compared to Keynote, PowerPoint or OmniGraffle. Eventually, I used draw.io to make the diagrams and include the images in the slides for a few of the cases where I actually needed to make diagrams.
Docker for Windows does not work on Windows Home or Student. Support for OPAM on Windows is slowly improving, but it is not yet for novices. Hence, I had to recommend the students to run an Ubuntu VM on their Windows machines in which they ran the course’s docker container.
nbgrader had several bugs which caused the autograder to award marks even for failing cells. The TAs had to go through a few of the assignments manually to ensure that students were awarded grades correctly. This is something that should be fixable easily.
RISE doesn’t easily let you change the size of the font. One has to edit the CSS to change the font size. And the default style wastes too much space. This meant that not much content can be fit into a single slide. Hence, I’ve had to artificially split content into multiple slides or zoom out several steps to show content that was cut off on the bottom.
The support for Prolog is not so great. There are a few advanced features in Prolog for which the Prolog setup fails. I had to switch to SWI-Prolog top-level for a few lectures. That said, the Prolog support is mostly there and the issues can be fixed with some effort.

Conclusion

I have started working on fixing some of these issues and upstreaming the solutions. Hopefully the fixes should be ready for the next iteration of the course. If you would like to replicate this setup for your course, do feel free to utilise the course materials.

Multicore OCaml Jobs

2019-09-16T11:59:00+00:00

Multiple Research Software Engineer positions are available in the Department of Computer Science and Engineering at the Indian Institute of Technology, Madras to develop Multicore OCaml and enable Tezos ecosystem to benefit from Multicore OCaml.

A dog, a deer and a monkey walk into a coffee shop...

Background

The Multicore OCaml project aims to add native support for scalable concurrency and shared memory parallelism in OCaml. At its core, Multicore OCaml extends the OCaml programming language with effect handlers for expressing scalable concurrency and a high-performance concurrent garbage collector aimed at responsive networked applications. Multicore OCaml is also the first industrial-strength language to come equipped with an efficient yet modular memory model, allowing high-level local program reasoning while retaining performance. Multicore OCaml is actively being developed and core features are being upstreamed to OCaml.

Tezos is an open-source smart contract platform for decentralized applications and assets. Tezos uses a self-amending cryptographic ledger - It achieves consensus not just about the state of a ledger, but about the state of its own protocol. The primary protocol of Tezos utilizes proof of stake and supports Turing complete smart contracts in a domain-specific language called Michelson. Tezos codebase is written in OCaml and extensively uses OCaml ecosystem libraries and tools such as Lwt, OPAM, and Irmin.

Roles

There are two roles:

Compiler Engineer: Runtime system improvements to the OCaml programming language in order to make it compatible with multicore support. Implementing new features in Multicore OCaml compiler.
Application Engineer: Developing core OCaml libraries that take advantage of multicore support. Adding parallelism support for Tezos ecosystem libraries and tools such as Lwt, Irmin, and dune.

Positions

The positions available are:

Role	Minimum Qualification	Pay Range (Per Month)
Project Engineer	BE / BTech / Master’s in Science/MCA or equivalent	Rs.21,500 to Rs.75,000
Senior Project Engineer	ME / MTech (or) BE / BTech / Master’s in Science / MCA or equivalent with 2 years experience	Rs.27,500 to Rs.1,00,000
Senior Project Officer / Post-doctoral Researcher	Ph.D. in Engineering or Sciences (or) ME / MTech with 3 years experience (or) BE / BTech / Master’s in Science / MCA or equivalent with 5 years experience	Rs.35,000 to Rs.1,50,000
Principal Project Officer	Ph.D. in Engineering or Sciences with 7 years experience (or) ME / MTech with 10 years experience (or) BE / BTech / Master’s in Science / MCA or equivalent with 12 years experience	Rs. 48,000 to Rs.2,25,000

The appointment will be made for 6 months initially and can be extended up to 2 years. The project engineers have the option of enrolling in the MS program at CSE, IIT Madras after 6 months. Such candidates may appear to the interview directly, without having to write GATE.

You will work closely with the OCaml Labs group, University of Cambridge, UK and Tarides, France. All of the work done will be made available as liberally licensed open-source software.

Skills

Compiler Engineer

Necessary:

Excellent working knowledge of C, concurrent and parallel programming
Knowledge of compilers (not necessarily of functional programming languages), operating systems, x86 & ARM assembly programming

Desired:

Experience developing and/or maintaining performant software systems
Experience with a functional programming language such as Haskell, OCaml, Scala, Scheme, Elm, or Elixir.
Track record of open source contributions.
Understanding of benchmarking techniques and analyzing results

Applications Engineer

Necessary

Excellent working knowledge of operating systems, concurrent and parallel programming
Experience with a functional programming language such as Haskell, OCaml, Scala, Scheme, Elm, or Elixir.

Desired

Track record of contributions to large open-source software systems
Understanding of benchmarking techniques and analyzing results

Apply

Write to kcsrk@iitm.ac.in with the subject “IITM Multicore OCaml 2019: Compiler Engineer” or “IITM Multicore OCaml 2019: Application Engineer” based on the role to express interest. Please include:

Curriculum Vitae
A summary of your experience in relevant technologies and software
Any open source contributions

Deterministically debugging concurrent GC bugs with rr

2019-04-28T00:00:00+00:00

Multicore OCaml comes with a concurrent garbage collector, where the garbage collector and the mutator threads run concurrently. Debugging concurrent GC bugs has been the most frustrating / satisfying (when fixed) part of Multicore OCaml development. rr, a record and replay tool has made debugging concurrent GC bugs a sustainable exercise. In this short post, I’ll describe why.

A particularly tricky concurrent GC bug is one which occurs once every 10 to 100 runs due to non-determinism and any attempt to instrument the program to isolate the bug (simplifying the program, adding print statements, etc.) makes it disappear. The bug may only appear relatively late in the program run – after a few major GC cycles, where the program might have allocated 10s of gigabytes of memory by then. The bug usually manifests as a segfault due to illegal memory access, but the source of the bug may lie in the previous GC cycle and perhaps due to actions of a different thread than the one that is throwing up the error. gdb often doesn’t help since finding the illegal memory access may not give any clue as to when the heap was corrupted.

rr to the rescue. rr is an enhancement over gdb with support for recording an execution and debugging in reverse. Once a failing execution is recorded, the execution can be replayed multiple times deterministically. This removes the non-determinism from debugging session. gdb does support record and replay, but not on multi-threaded targets.

The fact that the program can be run in reverse is the key for debugging heap corruptions. An illegal access typically appears as a load or store to a illegal memory address obtained from a heap object. When such an illegal access is found, I set a hardware watchpoint on the heap address containing the illegal address and continue the program in reverse. rr runs the program in reverse until the write that stored the illegal address in the heap object! Usually, several transitive reverse runs are necessary to get to the source of the bug, but this is just mechanics.

While rr supports multi-threaded programs, it runs every thread on the same core. This usually makes the bug disappear. Luckily, rr comes with support for forcing a context switch after a certain number of CPU ticks (measured in terms of the number of retired conditional branches). Even with this option, you will need many runs before rr comes across a buggy execution. So I use the following command:

for i in {1..10000}; do rr record -c 10000 <program> <args>; if (( $? == 0 )); then echo "done $i"; else break; fi; done

which runs <prog> <args> under rr where a thread is allowed to execute for a maximum of 10,000 ticks before a context switch. rr runs are repeated until a crash is found or 10,000 rr runs are successfully completed. Depending on the program being debugged, I leave it running overnight. If rr had in fact found a crash, I can perform replay debugging with rr replay the following morning and have a deterministic and reversible recorded execution to work with.

rr has save countless hours in the development of Multicore OCaml, and rr should be a essential tool in every GC hacker’s toolbox.

ML Family Workshop 2019: Call for presentations

2019-04-22T16:00:00+00:00

I am chairing the PC for ML family workshop this year. The PC is happy to invite submissions for the workshop to be held during the ICFP conference week on Thursday 22nd August 2019.

ML family workshop invites submissions touching on the programming languages traditionally seen as part of the “ML family”. However, we are also keen to receive submissions from other related language groups. If you have questions about the suitability of your work for the workshop, please feel free to write an email.

The detailed CFP is available on the ICFP website:

https://icfp19.sigplan.org/home/mlfamilyworkshop-2019#Call-for-Presentations

Important dates

Thu 16 May 2019 – (AoE)Submission deadline
Sun 30 Jun 2019 – Author Notification
Thu 22 Aug 2019 – ML Family Workshop

Program Committee

Aggelos Biboudis – EPFL, Switzerland
Andreas Rossberg – Dfinity, Germany
Atsushi Igarashi – Kyoto University, Japan
Avik Chaudhuri – Facebook, USA
Cyrus Omar – University of Chicago, USA
David Allsopp – University of Cambridge, UK
Edwin Brady – University of St. Andrews, UK
Jacques-Henri Jourdan – CNRS, LRI, Université Paris-Sud, France
KC Sivaramakrishnan – IIT Madras, India
Lars Bergstrom – Mozilla Research, USA
Matthew Fluet – Rochester Institute of Technology, USA
Zoe Paraskevopoulou – Princeton University, USA

Submission details

We seek extended abstracts, up to 3 pages long. Submissions must be uploaded to the workshop submission website:

https://icfp19mlworkshop.hotcrp.com/

OCaml on Baremetal Shakti RISC-V processor

2019-03-29T14:00:00+00:00

It has been 3 months since I joined IIT Madras and it has been good fun so far. Along with the members of the RISE group, we’ve initiated a project to build secure applications on top of secure extensions of the open-source Shakti RISC-V processor ecosystem. Unsurprisingly, my language of choice to build the applications is OCaml. Given the availability of rich ecosystem of libraries under the MirageOS library operating system for building unikernels, we hope to minimise the amount of unsafe C code that the hardware has to contend with and protect exploits against. As a first step, we have managed to get OCaml programs to run on directly on top of the Shakti processor running in simulation under QEMU and Spike ISA simulators without an intervening operating system.

A custom bootloader performs the necessary hardware initialisation and transfers control directly to the OCaml program. We have open-sourced all of the tools necessary to build your own kernel. This handy dockerfile documents the entire process. For the impatient, an image is available in the dockerhub:

$ docker run -it iitmshakti/riscv-ocaml-baremetal:0.1.0

# Write your program
$ echo 'let _ = print_endline "A camel treads on hardware!"' > hello.ml
# Compile for Shakti
$ ocamlopt -output-obj -o payload.o hello.ml
$ file payload.o
payload.o: ELF 64-bit LSB relocatable, UCB RISC-V, version 1 (SYSV), not stripped

# Link with bootcode and build the kernel
$ make -C ../build
make: Entering directory '/root/ocaml-baremetal-riscv/build'
make[1]: Entering directory '/root/ocaml-baremetal-riscv/build'
make[2]: Entering directory '/root/ocaml-baremetal-riscv/build'
make[2]: Leaving directory '/root/ocaml-baremetal-riscv/build'
[ 64%] Built target boot
make[2]: Entering directory '/root/ocaml-baremetal-riscv/build'
make[2]: Leaving directory '/root/ocaml-baremetal-riscv/build'
[ 78%] Built target freestanding-compat
make[2]: Entering directory '/root/ocaml-baremetal-riscv/build'
make[2]: Leaving directory '/root/ocaml-baremetal-riscv/build'
[ 85%] Built target asmrun_t
make[2]: Entering directory '/root/ocaml-baremetal-riscv/build'
make[2]: Leaving directory '/root/ocaml-baremetal-riscv/build'
[ 92%] Built target nolibc_t
make[2]: Entering directory '/root/ocaml-baremetal-riscv/build'
make[2]: Leaving directory '/root/ocaml-baremetal-riscv/build'
[100%] Built target kernel
make[1]: Leaving directory '/root/ocaml-baremetal-riscv/build'
make: Leaving directory '/root/ocaml-baremetal-riscv/build'
$ file kernel 
kernel: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), statically linked, with debug_info, not stripped

# Run under spike RISC-V ISA simulator
$ spike kernel
ocaml-boot: heap@0x80042be8 stack@0x8002fbc0
A camel treads on hardware!
ocaml-boot: caml runtime returned. shutting down!

# Run under QEMU
$ qemu-system-riscv64 -machine spike_v1.10 -smp 1 -m 1G -serial stdio -kernel kernel
VNC server running on 127.0.0.1:5900
ocaml-boot: heap@0x80042be8 stack@0x8002fbc0
A camel treads on hardware!
ocaml-boot: caml runtime returned. shutting down!

The immediate next step will be getting the code to run on a Shakti softcore on an FPGA. In addition to targeting high-end FPGAs, we will also be targeting the $100 Arty A7 hobbyist board and release all of the software under liberal open-source licenses.

Further along, we will port mirage libraries to Shakti following similar to the setup in Well-typed lightbulbs and implementing hardware security enhancements in Shakti for preventing spatial and temporal attacks while running unsafe C code (with the ability to dynamically turn it off when running OCaml!), hardware-assisted compartments, etc. Lots of exciting possibilities on the horizon!

Acknowledgements

Much of this work was done by the incredible Malte, who is a visiting student at IIT Madras on a semester away from Leibniz University Hannover, Arjun, Lavanya, Ambika, Chester, and the rest of the Shakti team. The RISC-V port of OCaml is developed and maintained by Nicolás Ojeda Bär.

Continuous Benchmarking & Call for Benchmarks

2018-09-13T15:43:00+00:00

Over the past few weeks, at OCaml Labs, we’ve deployed continuous benchmarking infrastructure for Multicore OCaml. Live results are available at http://ocamllabs.io/multicore. Continuous benchmarking has already enabled us to make informed decisions about the impact of our changes, and should come in handy over the next few months where we polish off and tune the multicore runtime.

Currently, the benchmarks are all single-threaded and run on x86-64. Our current aim is to quantify the performance impact of running single-threaded OCaml programs using the multicore compiler. Moving forward, would would include multi-threaded benchmarks and other architectures.

The benchmarks and the benchmarking infrastructure were adapted from OCamlPro’s benchmark suite aimed at benchmarking Flambda optimisation passes. The difference with the new infrastructure is that all the data is generated as static HTML and CSV files with data processing performed on the client side in JavaScript. I find the new setup easier to manage and deploy.

Quality of benchmarks

If you observe the results, you will see that multicore is slowest compared to trunk OCaml on menhir-standard and menhir-fancy. But if you look closely:

these benchmarks complete in less than 10 milliseconds. This is not enough time to faithfully compare the implementations as constant factors such as runtime initialisation and costs of single untimely major GC dominate any useful work. In fact, almost half of the benchmarks complete within a second. The quality of this benchmark suite ought to be improved.

Call for benchmarks

While we want longer running benchmarks, we would also like those benchmarks to represent real OCaml programs found in the wild. If you have long running real OCaml programs, please consider adding it to the benchmark suite. Your contribution will ensure that performance-oriented OCaml features such as multicore and flambda are evaluated on representative OCaml programs.

How to contribute

Make a PR to multicore branch of ocamllabs/ocamlbench-repo. The packages directory contains many examples for how to prepare programs for benchmarking. Among these, numerical-analysis-bench and menhir-bench are simple and illustrative.

The benchmarks themselves are run using these scripts.

Dockerfile

There is a handy Dockerfile to test benchmarking setup:

$ docker build -t multicore-cb -f Dockerfile . #takes a while; grab a coffee

This builds the docker image for the benchmarking infrastructure. You can run the benchmarks as:

$ docker run -p 8080:8080 -it multicore-cb bash
$ cd ~/ocamlbench-scripts
$ ./run-bench.sh --nowait --lazy #takes a while; grab lunch

You can view the results by:

$ cd ~/logs/operf
$ python -m SimpleHTTPServer 8080

Now on your host machine, point your browser to localhost:8080 to interactively visualise the benchmark results.

Caveats

Aim to get your benchmark compiling with OCaml 4.06.1. You might have trouble getting your benchmark to compile with the multicore compiler due to several reasons:

Multicore compiler has syntax extensions for algebraic effect handlers which breaks packages that use ppx.
Multicore compiler has a different C API which breaks core dependencies such as Lwt.
Certain features such as marshalling closures and custom tag objects are unimplemented.

If you encounter trouble submitting benchmarks, please make an issue on kayceesrk/ocamlbench-scripts repo.

JFP Special Issue on Algebraic Effects and Handlers

2018-08-16T09:09:00+00:00

Andrej Bauer and I are editing a special issue of JFP on the theory and practice of algebraic effects and handlers. The CfP is below.

CALL FOR PAPERS

JFP Special Issue
on
The Theory and Practice of Algebraic Effects and Handlers

Submission Deadline: 18 January 2019
Expected Publication Date: December 2019

Scope

An important aspect of real-world languages is their support for computational effects such as raising exceptions, printing to the screen, accessing a database, non-determinism, and concurrency. In order to reason about the semantics of a programming language with computational effects, it is necessary to separate the effects out from the rest of the language. To this end, algebraic effects permit a wide class of computational effects to be specified in a pure setting using only operations that give rise to them and equations that the operations satisfy. The algebraic treatment of operations naturally leads to a novel treatment of handlers for all computational effects, not just for exceptions.

Algebraic effect handlers have been steadily gaining attention as a programming language feature since they generalise many control-flow abstractions such as exception handling, iterators, async/await, or backtracking, while ensuring that the composition of various features remains well-behaved. Indeed, there are implementations of algebraic effects and effect handlers as libraries in C, Clojure, F#, Haskell, OCaml, Scala, JavaScript, as well as full-fledged languages such as Eff, Frank, Links, Koka, and Multicore OCaml. Algebraic effect handlers have also influenced the design of software tools in industry including Facebook’s React UI library and Uber’s Pyro probabilistic programming language.

To recognise and encourage the publication of mature research contributions in this area, a special issue of the Journal of Functional Programming (JFP) will be devoted to the same theme.

Topics

Full-length, archival-quality submissions are solicited on theoretical and practical aspects of algebraic effects and handlers. Examples include, but are not limited to:

Reasoning about algebraic effects and handlers (denotational semantics, dependent types, logical relations, language support for equational reasoning)
Effect typing (subtyping, row-polymorphism, generativity, encapsulation)
Implementation of effect handlers (dynamic effects, selective CPS translations, delimited continuations)
Applications of algebraic effect handlers (probabilistic programming, event correlation, meta-programming, asynchronous I/O, debugging)

Reports on applications of these techniques to real-world problems are especially encouraged, as are submissions that relate ideas and concepts from several of these topics, or bridge the gap between theory and practice.

Papers will be reviewed as regular JFP submissions, and acceptance in the special issue will be based on both JFP’s quality standards and relevance to the theme. The special issue also welcomes high-quality survey and position papers that would benefit a wide audience.

Authors are encouraged to indicate interest in submitting by December 14, 2018, to aid in identifying suitable reviewers. The submission deadline is January 18, 2019. The expected submission length is 25-35 pages, excluding bibliography and appendices. Shorter submissions are encouraged; prospective authors of longer submissions should discuss their plans with the special issue editors in advance.

Submissions that are based on previously-published conference or workshop papers must clearly describe the relationship with the initial publication, and must differ sufficiently that the author can assign copyright to Cambridge University Press. Prospective authors are welcome to discuss such submissions with the editors to ensure compliance with this policy.

Submissions

Submissions should be sent through the JFP Manuscript Central system at https://mc.manuscriptcentral.com/cup/jfp_submit. Choose “Effects and Handlers” as the paper type, so that it gets assigned to the special issue.

For other submission details, please consult an issue of the Journal of Functional Programming or see the Journal’s web page at http://journals.cambridge.org/jid_JFP.

Tentative Schedule

14 December 2018: Expression of interest
18 January 2019: Submission deadline
22 April 2019: First round of reviews
23 August 2019: Revision deadline
15 November 2019: Second round of reviews
13 December 2019: Final accepted versions due

Guest Editors

Andrej Bauer, Faculty of Mathematics and Physics, University of Ljubljana
KC Sivaramakrishnan, Department of Computer Science and Technology, University of Cambridge

Editors in Chief

Jeremy Gibbons, Department of Computer Science, University of Oxford
Matthias Felleisen, College of Computer and Information Science, Northeastern University

A deep dive into Multicore OCaml garbage collector

2017-07-06T01:36:00+00:00

I recently gave a talk on the internals of multicore OCaml GC at Jane Street offices in NYC. The slides from the talk are available online. But I felt that the slides alone aren’t particularly edifying. This post is basically the slides from the talk annotated with notes.

Abstract

In a mostly functional language like OCaml, it is desirable to have each domain (our unit of parallelism) collect its own local garbage independently. Given that OCaml is commonly used for writing latency sensitive code such as trading systems, UIs, networked unikernels, it is also desirable to minimise the stop-the-world phases in the GC. Although obvious, the difficulty is to make this work in the presence of mutations and concurrency. In this talk, I will present the overall design of Multicore OCaml GC, but also deep dive into a few of the interesting techniques that make it work.

Slidedeck

Slide of ( ← and → arrow keys also work)

Multicore OCaml project is led from OCaml Labs within the University of Cambridge. Stephen Dolan begun the project while procrastinating on writing up his dissertation. Hurray to that!

Multicore OCaml extends OCaml with native support for concurrency and parallelism. Unlike many other languages, we clearly separate concurrency from parallelism in the language. Concurrency in Multicore OCaml is expressed through lightweight language level threads called fibers. The unit of parallelism is a domain, which maps to kernel threads. Many kernel threads may service a particular domain, but only one of those kernel threads ever runs OCaml code. A typical program is expected to have a large number of fibers mapped over a few domains. In this talk, I provide an overview of the Multicore OCaml GC design, with a few deep dives into some of the interesting and novel techniques.

It is difficult to appreciate the subtleties of the choice of GC techniques in isolation. So we shall begin with a sane setup - a GC for a sequential purely functional language. We shall subsequently extend the language with series of reprehensible extensions including mutations, parallelism and concurrency, in that order, and observe how the sane setup falls apart and what we shall do to recover sanity while retaining efficiency. The early parts of the talk should be unsurprising to someone familiar with GC internals, but is useful for setting up the latter material.

The figure shows the snapshot of the runtime state of a sequential purely functional language. We have a set of objects allocated on the heap. These objects may be pointed to be other heap objects as well as any of the registers and the current runtime stack. The simplest GC strategy is to stop the program and perform a mark-and-sweep garbage collection. The core idea is that we start from the roots of the program state i.e., current stack and registers, and perform a depth-first traversal through the object graph. Any unreachable objects are garbage and they can be reclaimed. Tri-colour marking is a standard marking algorithm. During marking, the objects are in one of three states: white(unmarked), grey(marking) and black(marked).

At the beginning of the GC, all objects are white. The GC marks grey any white object it finds and pushes it into the mark stack. Subsequently, objects are popped off the mark stack, all of its white children marked, and the object is marked black. We have the invariant that a black object does not point to a white object. This is called the tri-colour invariant. The figure shows the state when object A and its children have been marked (hence, A is black), object B has been marked grey and is on the mark stack.

GC is done when the mark stack is empty. At this point, all reachable objects are black. Any unreachable objects are white. A sweeper examines all the allocated objects, and marks any object still white as free space that can subsequently be reused.

Mark and sweep GC has several advantages. For starters, it is a very simple algorithm as GC algorithms go. The algorithm is also naturally incremental. The mutator (OCaml code) and the GC work can alternate between each other, minimizing the pause times in the GC. However, the primary disadvantage with this scheme is that one needs to maintain a free-list of objects for allocations. While there are many algorithms to find the best place to allocate an object, all of them have non-trivial overheads. Functional programming languages have high rate of allocation and would benefit fast allocations. Moreover, free-list implementations also suffer from fragmentation.

To alleviate this, functional programming languages typically implement a generational GC. Generational hypothesis says that the young objects are far more likely to die than older objects. To take advantage of this, the heap is split into two -- a small minor heap, where new objects are allocated and a larger major heap. In particular, objects in the minor heap are allocated by bumping the frontier, leading to fast allocations. When the minor heap is full, we garbage collect the minor heap.

Minor heap is GCed with a copying collector, and any object that survives the minor GC is promoted to the major GC. A nice aspect of copying collection for minor GC is that only the live objects need to be scanned, unlike mark and sweep collection where the sweeper needs to examine every allocated object. Given the generational hypothesis, we will not examine most of the objects in the minor heap. As a data point, the minor GC survival rate while compiling the OCaml compiler is around 10%.

The roots for the minor collection are the current stack and the live register set. Since our language is purely functional, there will be no pointers from the major heap to the minor heap. This is because all the objects in major heap are older than all the objects in the minor heap. The lack of mutations mean that an object can only point to an older object. Hence, we don't need to care about objects in the major heap for minor collections.

With mutations, the major heap may point to the minor heap. For example, by assigning a minor heap object to a major heap reference. We need to know these references so that we can treat them as roots for the minor collection.

Naively, we can scan the entire major heap for every minor GC to find such pointers. But this is impractical.

Instead, we intercept the writes with a write barrier that records such pointers in an auxiliary data structure called the remembered set.

Remembered set holds the set of pointers from the major heap to the minor heap, and is used as the root for minor collections. After the minor collection, the remembered set is cleared.

Mutations breaks tri-colour invariant. Suppose our heap has these three objects...

and we assign the white object B to the black object C...

and we subsequently delete the pointer from A to B. If we perform a major GC now,

A will be eventually marked as black. But the white object B is only pointed to by the black object C, which will not be marked,

and the sweeper will mark it as free. This leaves the heap in an inconsistent state.

Mutations are problematic if two conditions hold. 1. there exists a black to white pointer and 2. all references from a grey object through a chain of white objects to that white object is deleted. We can recover correctness if we disallow one of these conditions.

An insertion barrier prohibits 1. Whenever a black to white pointer may be established, the insertion barrier marks the target.

This prevents B from being GCed. The insertion barrier is said to preserve strong tri-colour invariant: A black object never points to a white object.

A deletion barrier prohibits 2. In particular, deletion barrier allows the black to white pointer from C to B.

But when the pointer from the grey A to the white B is deleted, B is marked. This prevents B from being GCed. The deletion barrier preserves weak tri-colour invariant: for any white object pointed to by black object, there exists some grey object from which through a series of white objects that white object is reachable.

Deletion barrier is used by OCaml. We extend the write barrier to mark the original object in the reference r if both r and x are in the major heap.

Let us extend the language by adding support for parallel execution. In Multicore OCaml, Domain.spawn forks off a new domain to run the thunk in parallel with the calling domain. It is reasonable to expect that most of the objects in the minor heap are in fact local to the domain which allocated the object, and are never shared between domains. Hence, it is desirable to collect each domain's young garbage independently without having to synchronize all of the domains.

This can be done if the minor heap objects are only accessed by the owning domain.

In their POPL'93 paper, Doligez and Leroy built a concurrent garbage collector for concurrent caml light which used domain local heaps. In their paper, the heap invariants imposed are that there are no pointers between the local heaps and the major heap does not point to any minor heaps.

These heap invariants are enforced with the help of a write barrier that intercepts writes, and whenever there is a pointer about to be created between the major and a minor heap, the transitive closure of the minor heap object is promoted to the major heap before the assignment.

The problem with this strategy is that we end up with false promotions. Consider a work stealing queue used for sharing work among multiple domains. It is very likely that the work-stealing queue survives the minor collection and is promoted to the major heap. Now, whenever a domain adds work to the queue, the work needs to be promoted to preserve the heap invariant, even though we expect that the domain which added the work to consume it in the common case.

Hence, we relax this invariant to allow pointers from major heap to the minor. We still do not allow pointers between minor heaps, but allow pointers from major to minor heaps.

We make sure that a domain does not access objects in a foreign domain with the help of a read barrier. The read barrier intercepts reads to mutable fields. If the value loaded is an integer, an object in the shared heap or own minor heap, then we let the read continue. Otherwise, we interrupt the domain which owns the object to promote the object closure. This operation returns the new location of the object in the major heap. This scheme ensures that the minor heaps can be independently GCed. Marlow and Peyton Jones evaluated a local heap design with similar weaker heap invariants for GHC Haskell.

There is a surprisingly efficient way to implement the read barrier checks through careful virtual memory mapping for minor heaps and a few bit twiddling tricks. Recall that the fast path for the read barrier is the value read is either an integer, or a shared heap value, or a value in own minor heap. Let us assume a 16-bit address space. The minor heap are all allocated in a contiguous power-of-2 aligned virtual memory area, where each minor heap is also a power-of-2 aligned and sized. Not all of the virtual memory area need to be allocated, but only needs to be reserved. In particular, we ensure that shared heap pages are not allocated in the minor heap area.

Addresses in this 16-bit address space can be written as 4 quads 0xPQRS. In OCaml, integer values are represented by tagging the least significant bit to be 1. Hence, in this example, integers have low bit of S to be 1. Minor heap values have PQ to be 42, and R determines the domain. We can implement the read barrier check by comparing the given address with an address from the minor heap. Luckily, we have such an address handily available in the register -- the allocation pointer. On amd64, the allocation pointer is in register r15.

Here is our read barrier implementation for amd64 architecture. Assume that the value of interest is in rax register. At the end of this sequence of instructions, if none of the enabled bits in 0xff01 are set in rax, then zero flag will be set, and we know that the value is not a pointer into a foreign minor heap. Let's see how this works for the different cases.

In the case, of integer, the least significant bit remains set, and hence zero flag will not be set. We are safe to read this value.

In the case of a shared heap address, the PQ bits will different between r15 and rax. Hence, zero flag will not be set.

In the case of an address in own minor heap, the bits PQR will be the same. Hence, the subtraction underflows and sets all the bits in PQ. Hence, the zero flag will not be set.

In the case of an address in foreign minor heap, the bits PQ will be the same. The bits in R will be different, and S will be zero in both values. After xoring, R will be non-zero and importantly, the rest of the bits are zero. Subtracting 1 from a non-zero value does not underflow, hence the rest of the bits remain zero. Now, the zero flag will be set after the test, and we know that the pointer is in the foreign minor heap.

We now have an efficient way to find out whether we need to promote. But how do we perform the promotion on a read fault? The general strategy is to interrupt the foreign domain to perform the promotion for us. There are several alternatives here.

On receiving the interrupt, the foreign domain may copy the objects from its minor heap to the major heap. While this strategy works for immutable objects, mutable objects which are copied should somehow be kept in sync on updates. This gets tricky especially with relaxed memory behaviours observed on modern multicore hardware. Moreover, this scheme breaks OCaml objects with Abstract_tag where a C library may map a C structure onto the OCaml heap and modify it transparently without the knowledge of the write barrier. Hence, the copies may go out of sync.

We may instead move the object to the major heap and perform a minor GC to fix any references to the promoted objects. However, this scheme suffers from false promotions and long pauses during reads. To avoid false promotions, we may promote the object and scan the roots and the entire minor heap to fix any references to promoted objects. However, one needs to scan all the objects in the minor heap, which even the minor collection doesn't have to do.

We make the observation that most objects promoted on read faults happen to be recently allocated. The reason being that such objects are messages placed into channels where there is a waiting receiver on the other domain which can consume the message immediately. In our experiments on a small corpus of parallel OCaml programs, we found that 95% of objects promoted on read fault are among the youngest 5%. To make use of this fact, we combine the solutions 2 and 3 from above.

If the objected to be promoted on read fault is among the youngest x% (in the current implementation x = 10, but can be dynamically chosen), then we move the transitive object closure to the major heap. We may have extant pointers into the promoted objects. We need to fix those pointers such that they point to the new location of these objects in the major heap. In particular, the pointers may be found in the roots, objects younger than promoted object in the minor heap, and older minor objects that point to younger minor objects due to mutations of older objects. For the latter, we extend the write barrier to record such minor to minor pointers in promotion_set, which is only scanned during the promotion process. Otherwise, we move the object closure and perform a minor GC.

That resolves the major pain points in the interaction of parallelism with minor GC and promotion. The next is the interplay between parallelism and major GC. Recall that OCaml's major GC is incremental -- the mutator and the GC (mark and sweep) take turns to run. If we were to extend the scheme naively to parallel execution, we would have a stop-the-world incremental collector, where all the domains have to synchronize for GC work. This would introduce significant latency overheads. Instead, we go for a concurrent collector design where the mutator and the GC thread can run in parallel.

Our design is based on the Very Concurrent Mark-&-Sweep Garbage Collector (VCGC) design from the Inferno project which allows the mutator, marker and the sweeper threads to run concurrently. In VCGC design, there is a small stop-the-world phase at the end of a cycle where the threads agree on the end of the current major GC cycle and the beginning of the next one. Multicore OCaml's major GC is mostly concurrent mark and sweep where the stop-the-world phase might need to do a small fraction of major GC work left over before the end of the cycle, not unlike the VCGC design with many mutators i.e., parallel execution.

The major heap objects are in one of the 4 states: Marked, Unmarked, Garbage and Free. The domains alternate between running the mutator and performing GC work. The GC thread performs a depth-first traversal of the heap. If it finds an Unmarked object, it changes its colour to Marked and pushes the object into a domain-local mark-stack. On the other hand, if it finds a Garbage object, it marks it as Free and adds it to the free list. Since multiple GC threads operate on the heap simultaneously, marking is racy but idempotent. In particular, there is no synchronization for marking the objects.

If any domain thinks that all the work in the current major GC cycle is done (in practice, close to being done), it calls for a global synchronization where all the domains synchronize on the barrier. Once stopped, all the domains work to actually finish marking, as some work may be left over at other domains. Once that is done, the cores agree to flip the meaning of the colours. Anything that is Unmarked is considered Garbage. Anything that is Marked becomes Unmarked for the next cycle. Garbage objects are considered Marked, but at the end of the major GC, all Garbage objects have been marked Free. Hence, no objects fall into this category. Anything that is marked Free still remains Free. This concludes the discussion on parallelism.

We introduce concurrency into the mix next. Concurrency in Multicore OCaml is expressed through fibers, which are language-level lightweight threads. Fibers are implemented as heap allocated, dynamically resized stack segments. Just like mutating a regular heap object, fibers can also be mutated by pushing and popping values. However, unlike regular objects, fiber mutations are not protected by a write barrier. This poses challenges to maintaining the heap invariants.

The main thread in Multicore OCaml is also a fiber. Hence, the GC root current stack is just a pointer to the current fiber. Since we don't have write barrier on pushing to a fiber, we need to approximate the pointers that may arise from a fiber in the major heap which points to the minor heap. We do this with the help of remembered fiber set.

This represents the set of fibers in the major heap that ran in the current minor cycle on some domain. Similar to remembered set, the remembered fiber set is also a domain local data structure. The remembered fiber set is a root for minor GCs. The remembered fiber set is cleared along with the remembered set at the end of minor GC.

We also treat fibers specially during promotions on read faults. Fibers transitively reachable are not promoted automatically in order to avoid prematurely promoting the entire world. We envision work stealing schedulers where the fiber only needs to be promoted when a different domain explicitly demands it.

In this example, when assigning x to r, instead of promoting fiber f,

we leave it on the minor heap. We record x in the remembered set.

And when a different domain demands it i.e., tries to context switch to the fiber f,

then we promote the fiber and its transitive closure.

Since we've added the domain-local remembered fiber set, at every instance the fast path is taken on a read fault, we are obligated to scan this set just after the promotion is done. But the remembered fiber set may be large, and the fiber stacks themselves can be large. Hence, scanning the stack for most promotions would introduce large pause times.

We would like to break up the large pause time if possible. We can do this by lazily scanning the fibers just before a context switch. We only need to scan a fiber once per promotion. We also expect the rate of context switches to be much smaller than the rate of promotions. Hence, in practice, the fiber only gets scanned once per batch of promotions.

How does concurrency affect the major GC? Recall that Multicore OCaml uses a deletion barrier. In a deletion barrier, whenever a major heap object loses a reference to another major heap object, we have to mark the target object. However, fiber stack pops are not protected by a write barrier. Instead, we conservatively mark all of the objects on the fiber before switching to it. This is not to dissimilar to the current stock OCaml compiler scanning the current stack (as part of the roots) at the beginning of the major GC cycle.

In VCGC, marking is racy but idempotent. Hence, the mutators can freely read and write the object while it is concurrently marked. In particular, no synchronization such as compare-and-swap or locks are required to mediate access between the mutator and the GC thread. However, this invariant does not hold for fibers. Assume that a mutator is about to context switch to a thread while a GC thread on another domain attempts to scan and mark the objects on the fiber. Since the mutator may push and pop the fiber stack, the GC thread concurrently scanning the stack may observe inconsistent state.

In order to prevent this inconsistency, before switching to a unmarked fiber, the fiber is marked and all the objects on the fiber are also marked -- fiber is made Black. And in order to prevent racy access, the fiber is locked while marking. If the GC thread holds the lock on a fiber and a mutator tries to context switch to it, the mutator blocks until the fiber is marked. If the GC thread loses the race, it can safely skip marking the fiber.

In summary, the Multicore OCaml GC optimizes for latency and minimizes the maximum pause time through independent minor GCs and a mostly concurrent mark-and-sweep collector for major GCs. The challenge is to consider the interactions between mutations, concurrency and parallelism and the various GC techniques in order to come up with a design that preserves safety and optimizes our performance goals. I will have to save the performance analysis of the GC to a different talk.

(Monadic) Reflections on Concurrency

2017-06-13T12:13:00+00:00

We recently published a paper on concurrent system programming with effect handlers. In this paper, we show that with the help of effect handlers, we could express in direct-style, various interactions of a concurrent program with OS services that typically require callbacks. The question is what do we do about legacy code that uses monadic concurrency libraries such as Lwt and Async. Surely a wholesale rewrite of all Lwt and Async code is a no go. This post is an exploration of some ideas to make Lwt and Async compatible with direct-style code.

Monadic Reflection

Andrzej Filinski introduced monadic reflection in his paper Representing Monads, characterizing the relationship between monadic style and continuation-passing style. Practically, in a language like multicore OCaml with native support for delimited continuations, any monadic style program can also be written in direct-style. Filinski introduces two operators to transform between the two styles:

reify transforms a direct-style computation to a monadic one and reflect goes the other way. In multicore OCaml, we can implement monadic reflection for any monad as¹:

We introduce an effect E which is parameterized with the monadic computation. When this effect is performed, it returns the result of performing this monadic computation. reify wraps the direct-style computation with an effect handler that handles E m and binds the monadic computation m to the rest of the direct-style computation. reflect simply performs the given monadic computation wrapped in E. The idea here is that whenever the monad does anything interesting, we perform the effect E which delegates the handling of interesting monadic behavior to the effect handler.

Monadic to Direct

We implement chameneos-redux benchmark from the computer language benchmarks game in direct-style and using concurrency monad. The benchmark is intended to evaluate the cost of context switching between tasks. The source code is available here in a single-self contained file. We implement both versions as functors (direct-style is ChameneosD (S : SchedD) (M : MVarD) and monadic-style is ChameneosM (S : SchedM) (M : MVarM)) parameterized by a scheduler and an MVar implementation. The signatures of direct and monadic style scheduler and MVars are:

Using monadic reflection on the monadic scheduler SchedM and MVar MVarM implementations, we can instantiate the direct-style functor ChameneosD:

We can even instantiate the direct-style functor ChameneosD with Lwt with no extra effort:

Thus, monadic reflection lets you utilize Lwt and Async in direct-style. Importantly, one gets back backtraces and the use of raise and try...with for exception handling.

Direct to Monadic

Lwt and Async libraries provide strong guarantees on task interleavings. In particular, both libraries provide automatic mutual exclusion – context switches between tasks only occur at bind points. In other words, any non-monadic functions, such as calls to standard library functions, are guaranteed not to context switch. With effect handlers, this is no longer the case since effects are not tracked in the types in multicore OCaml.

We can recover the type level marker with a shallow embedding:

And we can go back to direct-style using monadic reflection:

Performance

We compared the performance of different configurations for running chameneos-redux for 1 million iterations:

The results show that monadic reflection has around 9% overhead on average over the baseline monadic implementations. This is a small price to pay for the advantage for programming in direct-style.

Conclusion

We have been prototyping a multicore-capable I/O library for OCaml called Aeio, with compatibility layer for Lwt and Async built on top of this library. Monadic reflection and other techniques can help resolve the schism between monadic libraries and direct-style code.

Footnotes

Thanks to Jeremy Yallop for introducing me to monadic reflection and contributing this implementation. ↩

Building and Publishing an OCaml Package: Q1 2017

2017-03-05T13:56:00+00:00

One of the key indicators of maturity of a language ecosystem is the ease of building, managing and publishing software packages in that language. OCaml platform has made steady progress in the last few years to this end. While OPAM simplified package (and compiler) management, the developing and publishing packages remained a constant pain point. This situation has remarkably improved recently with the Topkg and Carcass. This post provides a short overview of my workflow for building and publishing an OCaml package using Topkg and Carcass.

Topkg is packager for distributing OCaml software. It provides an API for describing rules for package builds and installs. Topkg-care provides the command line tool topkg with support for creating and linting the distribution, publishing the distribution and its documentation on WWW, and making the package available through OPAM. Carcass is a library and a command line tool for defining and generating the directory structure for the OCaml package. At the time of writing this post, carcass was unreleased.

Workflow

I recently released a package for mergeable vectors based on operational transformation. The following describes my workflow to build and publish the package.

Setup

Install topkg-care and carcass:

$ opam install topkg-care opam-publish
$ opam pin add -kgit carcass https://github.com/dbuenzli/carcass

Develop

Create the directory structure

  $ carcass body topkg/pkg mergeable_vector

Init

  $ cd mergeable_vector && git init && git add . && git commit -m "First commit."
  $ git remote add origin https://github.com/kayceesrk/mergeable-vector
  $ git push --set-upstream origin master

Develop: The mergeable_vector/src directory has the source files. I use this Makefile at the root of the package.
Test the package locally with OPAM
```
  $ opam pin add mergeable_vector .
```

Publish

Update the CHANGES file for the new release.
Tag the release
```
  $ topkg tag 0.1.0
```
Build the distribution
```
  $ topkg distrib
```
Publish the distribution
```
  $ topkg publish distrib
```
This makes a new release on Github.
Publish the doc
```
  $ topkg publish doc
```
This publishes the documentation on Github.
Make an OPAM package info and submit it to OPAM repository at opam.ocaml.org.
```
  $ topkg opam pkg
  $ topkg opam submit
```
This creates a Github PR to the opam-repository. Once the PR is merged, the package becomes available to the users.

Ezirmin : An easy interface to the Irmin library

2017-02-15T13:46:00+00:00

Ezirmin is an easy interface over the Irmin, a library database for building persistent mergeable data structures based on the principles of Git. In this post, I will primarily discuss the Ezirmin library, but also discuss some of the finer technical details of mergeable data types implemented over Irmin.

Contents
Irmin and Ezirmin
Quick tour of Ezirmin
Mergeable persistent data types
Next steps

Irmin and Ezirmin

Irmin is a library for manipulating persistent mergeable data structures (including CRDTs) that follows the same principles of Git. In particular, it has built-in support for snapshots, branches and reverts, and can compile to multiple backends. Being written in pure OCaml, apps written using Irmin, as well as running natively, can run in the browsers or be compiled to Unikernels. A good introduction to the capabilities of Irmin can be found in the Irmin README file.

One of the downsides to being extremely configurable is that the Irmin library is not beginner friendly. In particular, the library tends to be rather functor heavy, and even simple uses require multiple functor instantiations¹. The primary goal of Ezirmin is to provide a defuntorized interface to Irmin, specialized to useful defaults. However, as I’ve continued to build Ezirmin, it has come to include a collection of useful mergeable data types including counters, queues, ropes, logs, etc. I will spend some time describing some of the interesting aspects of these data structures.

Quick tour of Ezirmin

You can install the latest version of Ezirmin by

$ git clone https://github.com/kayceesrk/ezirmin
$ cd ezirmin
$ opam pin add ezirmin .

Stable versions are also available through OPAM:

$ opam install ezirmin

Let’s fire up utop and get started:

$ utop
utop # #require "ezirmin";;
utop # open Lwt.Infix;;

We’ll create a mergeable queue of strings using the Git file system backend rooted at /tmp/ezirminq:

utop # module M = Ezirmin.FS_queue(Tc.String);; (* Mergeable queue of strings *)
utop # open M;;
utop # let m = Lwt_main.run (init ~root:"/tmp/ezirminq" ~bare:true () >>= master);;
val m : branch = <abstr>

m is the master branch of the repository. Ezirmin exposes a key value store, where keys are hierarchical paths and values are whatever data types is stored in the repo. In this case, the data type is a queue. Let’s push some elements into the queue:

utop # push m ["home"; "todo"] "buy milk";;
- : unit = ()
utop # push m ["work"; "todo"] "publish ezirmin";;
- : unit = ()
utop # to_list m ["home"; "todo"];;
- : string list = ["buy milk"]

The updates to the queue is saved in the Git repository at /tmp/ezirminq. In another terminal,

$ utop
utop # #require "ezirmin";;
utop # module M = Ezirmin.FS_queue(Tc.String);; (* Mergeable queue of strings *)
utop # open M;;
utop # open Lwt.Infix;;
utop # let m = Lwt_main.run (init ~root:"/tmp/ezirminq" ~bare:true () >>= master);;
val m : branch = <abstr>
utop # pop m ["home"; "todo"];;
- : string option = Some "buy milk"

For concurrency control, use branches. In the first terminal,

utop # let wip = Lwt_main.run @@ clone_force m "wip";;
utop # push wip ["home"; "todo"] "walk dog";;
- : unit = ()
utop # push wip ["home"; "todo"] "take out trash";;
- : unit = ()

The changes are not visible until the branches are merged.

utop # to_list m ["home"; "todo"];;
- : string list = []
utop # merge wip ~into:m;;
- : unit = ()
utop # to_list m ["home"; "todo"];;
- : string list = ["walk dog"; "take out trash"]

Merge semantics

What should be the semantics of popping the queue at home/todo concurrently at the master branch and wip branch? It is reasonable to ascribe exactly once semantics to pop such that popping the same element on both branches and subsequently merging the queues would lead to a merge conflict. However, a more useful semantics is where we relax this invariant and allow elements to be popped more than once on different branches. In particular, the merge operation on the queue ensures that:

An element popped in one of the branches is not present after the merge.
Merges respect the program order in each of the branches.
Merges converge.

Hence, our merge queues are CRDTs.

Working with history

Irmin is fully compatible with Git. Hence, we can explore the history of the operations using the git command line. In another terminal:

$ cd /tmp/ezirminq
$ git lg
* e75da48 - (4 minutes ago) push - Irmin xxxx.cam.ac.uk.[73126] (HEAD -> master, wip)
* 40ed32d - (4 minutes ago) push - Irmin xxxx.cam.ac.uk.[73126]
* 6a56fb0 - (5 minutes ago) pop - Irmin xxxx.cam.ac.uk.[73221]
* 6a2cc9a - (6 minutes ago) push - Irmin xxxx.cam.ac.uk.[73126]
* 55f7fc8 - (6 minutes ago) push - Irmin xxxx.cam.ac.uk.[73126]

The Git log shows that there have been 4 pushes and 1 pop in this repository. In addition to the data structures being mergeable, they are also persistent. In particular, every object stored in Irmin has complete provenance. You can also manipulate history using the Git command line.

$ git reset HEAD~2 --hard
$ git lg
* e75da48 - (8 minutes ago) push - Irmin xxxx.cam.ac.uk.[73126] (wip)
* 40ed32d - (9 minutes ago) push - Irmin xxxx.cam.ac.uk.[73126]
* 6a56fb0 - (9 minutes ago) pop - Irmin xxxx.cam.ac.uk.[73221] (HEAD -> master)
* 6a2cc9a - (10 minutes ago) push - Irmin xxxx.cam.ac.uk.[73126]
* 55f7fc8 - (10 minutes ago) push - Irmin xxxx.cam.ac.uk.[73126]

Back in the first terminal:

utop # to_list m ["home"; "todo"];;
- : string list = []

Since we rolled back the master to before the pushes were merged, we see an empty list. Ezirmin also provides APIs for working with history programmatically.

utop # let run = Lwt_main.run;;
utop # let repo = run @@ init ();;
utop # let path = ["Books"; "Ovine Supply Logistics"];;
utop # let push_msg = push m ~path;;

utop # begin
  push_msg "Baa" >>= fun () ->
  push_msg "Baa" >>= fun () ->
  push_msg "Black" >>= fun () ->
  push_msg "Camel"
end;;

utop # to_list m path;;
- : string list = ["Baa"; "Baa"; "Black"; "Camel"]

Clearly this is wrong. Let’s fix this by reverting to earlier version:

utop # let m_1::_ = run @@ predecessors repo m;; (** HEAD~1 version *)
utop # to_list m_1 path;;
- : string list = ["Baa"; "Baa"; "Black"]
utop # update_branch m ~set:m_1;;
utop # to_list m path;;
- : string list = ["Baa"; "Baa"; "Black"]

Now that we’ve undone the error, we can do the right thing.

utop # push_msg "Sheep";;
utop # run @@ to_list m path;;
- : string list = ["Baa"; "Baa"; "Black"; "Sheep"]

Reacting to changes

Ezirmin supports watching a particular key for updates and invoking a callback function when there is one.

utop # let cb _ = Lwt.return (print_endline "callback: update to home/todo");;
utop # watch m ["home"; "todo"] cb

The code above installs a listener cb on the queue at home/todo, which is run every time the queue is updated. This includes local push and pop operations as well as updates due to merges.

utop # push m ["home"; "todo"] "hang pictures";;
callback: update to home/todo
- : unit = ()

Interaction with remotes

Unlike distributed data stores, where the updates are disseminated transparently between the replicas, Ezirmin provides you the necessary building blocks for building your own dissemination protocol. As with Git, Ezirmin exposes the functionality to push² and pull changes from remotes.

#show_module Sync;;
module Sync : sig
  type remote
  val remote_uri : string -> remote
  val pull : remote -> branch -> [ `Merge | `Update ] -> [ `Conflict of string | `Error | `No_head | `Ok ] Lwt.t
  val push : remote -> branch -> [ `Error | `Ok ] Lwt.t
end

This design provides the flexibility to describe your own network layout, with anti-entropy mechanisms built-in to the synchronization protocol. For example, one might deploy the replicas in a hub-and-spoke model where each replica accepts client writes locally and periodically publishes changes to the master and also fetches any latest updates. The data structures provided by Ezirmin are always mergeable and converge. Hence, the updates are never rejected. It is important to note that even though we have a centralized master, this deployment is still highly available. Even if the master is unavailable, the other nodes can still accept client requests. The replicas may also be connected in a peer-to-peer fashion without a centralized master for a more resilient deployment.

Mergeable persistent data types

Ezirmin is equipped with a growing collection of mergeable data types. The mergeable datatypes occupy a unique position in the space of CRDTs. Given that we have the history, the design of mergeable datatypes is much simpler. Additionally, this also leads to richer structures typically not found in CRDTs. It is worth studying them in detail.

Irmin Architecture

Irmin provides a high-level key-value interface built over two lower level heaps: a block store and a tag store. A block store is an append-only content-addressable store that stores serialized values of application contents, prefix-tree nodes, history meta-data, etc. Instead of using physical memory address of blocks, the blocks are identified by the hash of their contents. As a result block store enjoys very nice properties. Being content-addressed, we get sharing for free: two blocks with the same content will have the have the same hash. This not only applies for individual blocks, but also for linked-structures. For example,

The linked list above is uniquely identified by hash h0 since h0 was computed from the content a and the hash of the tail of the list h1. No other list has hash h0. Changing c to C in this list would result in a different hash for the head of the list³. Moreover, since the block store is append-only, all previous versions of a application-level data structure is also available, and thus providing persistence. This also makes for a nice concurrency story for multiple processes/threads operating on the block store. The absence of mutations on block store mean that no additional concurrency control mechanisms are necessary.

The only mutable part of the Irmin architecture is the tag store, that maps global names to blocks in the block store. The notion of branches are built on top of the tag store. Cloning a branch creates a new tag that points to the same block as the cloned branch.

User-defined merges

The real power of Irmin is due to the user-defined merges. Irmin expects the developer to provide a 3-way merge function with the following signature:

type t
(** User-defined contents. *)

val merge : old:t -> t -> t -> [`Ok of t | `Conflict of string]
(** 3-way merge. *)

Given the common ancestor old and the two versions, merge function can either return a successful merge or mark a conflict. It is up to the developer to ensure that merges are commutative (merge old a b = merge old b a) and that the merge captures the intent of the two branches. If the merge function never conflicts, we have CRDTs.

Mergeable Counters

The simplest mergeable data type is a counter with an increment and decrement operations. Given that we have a 3-way merge function, the merge is intuitive:

Given the two new values for the counter t1 and t2, and their lowest common ancestor value old, the new value of the counter is the sum of the old value and the two deltas: old + (t1 - old) + (t2 - old) = t1 + t2 - old.

Theory of merges

While this definition is intuitive, the proof of why this strategy (i.e., computing deltas and applying to the common ancestor) is correct is quite subtle. It happens to be the case that the patches (deltas) in this case, integers under addition, form an abelian group. Judah Jacobson formalizes patches for Darcs as inverse semigroups and proves convergence. Every abelian group is also an inverse semigroup. Hence, the above strategy is correct. Merges can also be equivalently viewed as a pushout in category theory, leading to the same result. I will have to save the discussion of the category theoretic reasoning of Irmin merges for another time. But Liam O’Connor has written a concise post on the theory of patches which is worth a read.

Recursive merges

Since Ezirmin allows arbitrary branching and merging, the lowest common ancestor need not be unique. One way to end up with multiple lowest common ancestors is criss-cross merges. For example, consider the history graph below:

The counter at some key in the master was initially 0. The branch wip was cloned at this point. The counter is incremented by 1 at master and 2 at wip. At this point, both branches are merged into the other branch. The common ancestor here is the initial state of counter 0. This results in counter value of 3 in both branches. Suppose there are further increments, 2 at master and 4 at wip, resulting in counter values 5 and 7 respectively in master and wip.

If the wip branch is now merged in master, there are two lowest common ancestors: the commit with value 1 at master and 2 in wip. Since the 3-way merge algorithm only work for a single common ancestor, the we adopt a recursive merge strategy, where the lowest common ancestors are first merged resulting in a internal commit with value 3 (represented by a dotted circle). This commit is now used as the common ancestor for merging, which results in 9 as the new state of the counter. This matches the increments done in both branches. The recursive merge strategy is also the default merge strategy for Git.

Mergeable logs

Another useful data type is mergeable logs, where each log message is a string. The merge operation accumulates the logs in reverse chronological order. To this end, each log entry is a pair of timestamp and message, and the log itself is a list of entries. They are constructed using mirage-tc:

The merge function extracts the newer entries from either branches, sorts them and appends to the front of the old list.

While this implementation is simple, it does not scale well. In particular, each commit stores the entire log as a single serialized blob. This does not take advantage of the fact that every commit can share the tail of the log with its predecessor. Moreover, every append to the log needs to deserialize the entire log, append the new entry and serialize the log again. Hence, append is an O(n) operation, where n is the size of the log. Merges are also worst case O(n). This is undesirable.

Efficient mergeable logs

We can implement a efficient logs by taking advantage of the fact that every commit shares the tail of the log with its predecessor.

type log_entry = {
  time    : Time.t;
  message : V.t;        (** V.t is type of message. *)
  prev    : K.t option  (** K.t is the type of address in the block store. *)
}

Merges simply add a new node which points to the logs of merged branches, resulting in a DAG that captures the causal history. The following sequence of operations:

utop # #require "ezirmin";;
utop # open Lwt.Infix;;
utop # module M = Ezirmin.Memory_log(Tc.String);;
utop # open M;;
utop # let m = Lwt_main.run (init () >>= master);;
utop # Lwt_main.run (
  append m [] "m0" >>= fun _ ->
  append m [] "m1" >>= fun _ ->
  clone_force m "wip" >>= fun w ->
  append w [] "w0" >>= fun _ ->
  append m [] "m2" >>= fun _ ->
  merge w ~into:m >>= fun _ ->
  append w [] "w1" >>= fun _ ->
  append w [] "w2" >>= fun _ ->
  append m [] "m3" >>= fun _ ->
  append m [] "m4"
);;

results in the heap below.

Read traverses the log in reverse chronological order.

utop # read_all m [];;
- : string list = ["m4"; "m3"; "m2"; "w0"; "m1"; "m0"]

This implementation has O(1) appends and O(1) merges, resulting in much better performance. The graph below compares the blob log implementation and this linked implementation with file system backend by performing repeated appends to the log and measuring the latency for append.

Each point represents the average latency for the previous 100 appends. The results show that the append latency for linked implementation remains relatively constant while the blob implementation slows down considerably with increasing number of appends. Additionally, the linked implementation also supports efficient paginated reads.

Mergeable ropes

A rope data structure is used for efficiently storing and manipulating very long strings. Ezirmin provides mergeable ropes where for arbitrary contents, but also specialized for strings. Ropes automatically rebalance to maintain the invariant that the height of the tree is proportional to the length of the contents. The crux of the merge strategy is that given a common ancestor and the two trees to be merged, the merge algorithm works out the smallest subtrees where the modification occurred. If the modifications are on distinct subtrees, then the merge is trivial.

If the modification is on the same subtree, then the algorithm delegates to merge the contents. This problem has been well studied under the name of operational transformation (OT). OT can be categorically explained in terms of pushouts. Mergeable strings with insert, delete and replace operations are isomorphic to counters with increment and decrement. We apply a similar strategy to merge string.

First we compute the diff between the common ancestor and the new tree using Wagner-Fischer algorithm. Then we transform one patch with respect to the other using standard OT definition such that we can first apply one of the original patch to the common ancestor and then apply the transformed patch of the other branch to get the result tree.

For example,

utop # #require "ezirmin";;
utop # open Lwt.Infix;;
utop # open Ezirmin.Memory_rope_string;;
utop # let m = Lwt_main.run (init () >>= master);;
utop # let t = Lwt_main.run (
  make "abc" >>= fun t ->
  write m [] t >>= fun _ ->
  Lwt.return t
);;
utop # let w = Lwt_main.run (clone_force m "w");;
utop # let _ = Lwt_main.run (
  set t 1 'x' >>= fun t' (* "axc" *) ->
  write m [] t' >>= fun _ ->

  insert t 1 "y" >>= fun t' (* "aybc" *)->
  write w [] t' >>= fun _ ->

  merge w ~into:m >>= fun _ (* "ayxc" *) ->
  merge m ~into:w
);;
utop # Lwt_main.run (
  read m [] >>= function
  | None -> failwith "impossible"
  | Some r -> flush r >|= fun s ->
  Printf.printf "m is \"%s\"\n" s
);;
- : unit = ()
m is "ayxc"
utop # Lwt_main.run (
  read w [] >>= function
  | None -> failwith "impossible"
  | Some r -> flush r >|= fun s ->
  Printf.printf "w is \"%s\"\n" s
)
- : unit = ()
w is "ayxc"

The combination of mergeable ropes with OT gets the best of both worlds. Compared to a purely OT based implementation, diffs are only computed if updates conflict at the leaves. The representation using ropes is also efficient in terms of storage where multiple versions of the tree shares blocks. A purely rope based implementation either has the option of storing individual characters (atoms) at the leaves (and resolve conflicts based on some deterministic mechanism such as timestamps or other deterministic strategies) or manifest the conflict at the leaves to the user to get it resolved. A simple strategy might be to present both of the conflicting strings, and ask the user to resolve it. Hence, mergeable ropes + OT is strictly better than either of the approaches.

Next steps

Ezirmin is open to comments and contributions. Next steps would be:

Implement more mergeable data types
Implement generic mergeable datatypes using depyt.
Explore the data types which admit conflicts. For example, a bank account with non-negative balance does not form a CRDT with a withdraw operation. However, operations such as deposit and accrue_interest can be coordination-free.

Footnotes

Things are indeed improving with a cleaner API in the 1.0 release. ↩
Push is currently broken. But given that Irmin is compatible with git, one can use git-push to publish changes. ↩
The same principle underlies the irrefutability of blockchain. No block can be changed without reflecting the change in every subsequent block. ↩

Behavioural types

2016-06-30T09:31:00+00:00

Behavioural types such as session types, contracts and choreography describe the behaviour of a software entity as a sequence of operations on a resource such as a communication channel, web service session or a file descriptor. Behavioural types capture well-defined interactions, which are enforced statically with the help of type system machinery. In this post, I will describe a lightweight embedding of behavioural types in OCaml using polymorphic variants through a series of examples. The complete source code for the examples is available here.

The idea of encoding behavioural types using polymorphic variants comes from FuSe, which is a simple library implementation of binary sessions in OCaml. Similar to FuSe linear use of resources is enforced through dynamic checks in the following examples. We’ll raise LinearityViolation when linearity is violated.

exception LinearityViolation

Refs that explain their work

Let us define a ref type that is constrained not only by the type of value that it can hold but also by the sequence of operations that can be performed on it.

module type Ref =
sig
  type ('a, 'b) ref constraint 'b = [>]

  val ref   : 'a -> ('a, 'b) ref
  val read  : ('a, [`Read of 'b]) ref -> 'a * ('a, 'b) ref
  val write : ('a, [`Write of 'b]) ref -> 'a -> ('a, 'b) ref
end

module Ref : Ref = struct ... end

The phantom type variable 'b constrained to be a polymorphic variant ('b = [>]) describes the sequence of permitted operations. For example, a reference can only be read when the type presents the read capability [`Read of 'b]. Here, the 'b represents the behaviour of the continuation. Consequently, the result of the read operation is a tuple consisting of the value read and a reference whose type is ('a,'b) ref. It is useful to think of the read operation as changing the type of the reference. The type for write is similar.

Associating behaviours with references is quite handy. For example, below is a reference that holds an integer, which can only be written once following which a single read is permitted:

let my_ref1 : (int, [`Write of [`Read of [`Stop]]]) Ref.ref = Ref.ref 10

The behavioural types are also automatically inferred. For example,

utop # let foo1 r =
  let r = Ref.write r 20 in
  Ref.read r;;
val foo1 :
  (int, [ `Write of [ `Read of [>  ] as 'a ] ]) Ref.ref ->
  int * (int, 'a) Ref.ref

The inferred type says that foo1 writes into r and then reads it. We can apply foo1 on my_ref1 as their behaviours are compatible.

utop # let v,res_ref = foo1 my_ref1;;
val v : int = 20
val res_ref : (int, [ `Stop ]) Ref.ref

Recursive behavioural types are obtained painlessly.

utop # let rec foo2 r =
  let r = Ref.write r 20 in
  let v, r = Ref.read r in
  foo2 r;;
val foo2 : (int, [ `Write of [ `Read of 'a ] ] as 'a) Ref.ref -> 'b

The inferred types says that foo2 repeatedly writes and then reads the given reference. Incompatible references are rejected statically. For example,

utop # let my_ref2 : (int, [`Write of [`Read of [`Stop]]]) Ref.ref = Ref.ref 10;;
val my_ref2 : (int, [ `Write of [ `Read of [ `Stop ] ] ]) Ref.ref = <abstr>
utop # let _ = foo2 my_ref2;;
Error: This expression has type
         (int, [ `Write of [ `Read of [ `Stop ] ] ]) Ref.ref
       but an expression was expected of type
         (int, [ `Write of [ `Read of 'a ] ] as 'a) Ref.ref
       These two variant types have no intersection

whereas

utop # let my_ref3 = Ref.ref 10;;
val my_ref3 : (int, _[>  ]) Ref.ref = <abstr>
utop # let _ = foo2 my_ref3;;

is accepted and runs forever. It is (sometimes) useful to write programs that don’t always run forever such as foo3:

utop # let rec foo3 r = function
  | 0 ->
      print_endline "done";
      Ref.read r
  | n ->
      let r = Ref.write r 20 in
      let v, r = Ref.read r in
      foo3 r (n-1);;

which runs for n iterations, where it performs a write and a read in every iteration but the last one where it just performs a read. Unfortunately, this program does not type check:

Error: This expression has type ('a, [ `Read of [>  ] ]) Ref.ref
       but an expression was expected of type ('a, [ `Write of [>  ] ]) Ref.ref
       These two variant types have no intersection

The problem is that the behaviour of the two branches are incompatible, and the program is rightly rejected. We distinguish the branches in the type using:

val branch : ('a, [>] as 'b) ref -> (('a, [>] as 'c) ref -> 'b) -> ('a, 'c) ref

branch r f indicates branch selection in r where f is a function that is always of the form fun x -> `Tag x. The fixed version of foo3 is:

utop # let rec foo3 r = function
  | 0 ->
      print_endline "done";
      Ref.write (Ref.branch r (fun x -> `Zero x)) 0
  | n ->
      let r = Ref.write (Ref.branch r (fun x -> `Succ x)) 20 in
      let v, r = Ref.read r in
      foo3 r (n-1);;
val foo3 :
  (int,
   [> `Succ of (int, [ `Write of [ `Read of 'a ] ]) Ref.ref
    | `Zero of (int, [ `Write of [>  ] as 'b ]) Ref.ref ]
   as 'a)
  Ref.ref -> int -> (int, 'b) Ref.ref = <fun>

Observe that the inferred type captures the branching behaviour, and works as expected:

utop # let my_ref4 = Ref.ref 10 in foo3 my_ref4 32;;
done
- : (int, _[>  ]) Ref.ref = <abstr>

Implementation

The implementation is unremarkable except for the machinery to dynamically enforce linearity.

module Ref : Ref =
struct

  type ('a, 'b) ref =
    {contents     : 'a;
     mutable live : bool} (* For linearity *)
     constraint 'b = [>]

  let ref v = {contents = v; live = true}

  let check r =
    if not r.live then raise LinearityViolation;
    r.live <- false

  let fresh r = {r with live = true}

  let read r =
    check r;
    (r.contents, fresh r)

  let write r v =
    check r;
    { contents = v; live = true }

  let branch r _ = check r; fresh r
end

Behavioural types crucially depend on linear use of the resources. Since OCaml does not have linear types, there is nothing that prevents writing the following function that seemingly violates the behavioural contract.

utop # let foo (r : (int, [`Read of [`Stop]]) Ref.ref) =
         let _, _ = Ref.read r in
         Ref.read r;;
val foo :
  (int, [ `Read of [ `Stop ] ]) Ref.ref -> int * (int, [ `Stop ]) Ref.ref =
  <fun>

While the type of r says that it will be read only once, the function foo reads it twice. This non-linear use of r is caught dynamically; the second read of r raises LinearityViolation.

utop # let _ = foo (Ref.ref 10);;
Exception: LinearityViolation.

Polymorphic References

Since we can accurately track the behaviour of references, we can safely allow differently typed values to be written and read from the reference. A reference that holds a value of type t can be read multiple times at t before being written at type u. This protocol is captured by the following type:

module type PolyRef =
sig
  type ('a,'b) rw_prot
    constraint 'b = [> `Read of 'a * 'b | `Write of 'c * ('c,_) rw_prot]
  type 'c ref constraint 'c = ('a,'b) rw_prot
  ...
end

As before, the reference holds values of 'a with the behaviour given by 'b. The reference can either by read multiple times at 'a or written once at 'c after which the reference holds values of type 'c. The rest of the operations are defined as usual:

module type PolyRef =
sig
  ...
  val ref  : 'a -> ('a,'b) rw_prot ref
  val read  : ('a,[> `Read of 'a * 'b]) rw_prot ref -> 'a * ('a,'b) rw_prot ref
  val write : ('a,[> `Write of 'b * ('b,'c) rw_prot]) rw_prot ref -> 'b ->
    ('b,'c) rw_prot ref
  val branch : ('a, [>] as 'b) rw_prot ref -> (('a, [>] as 'c) rw_prot ref -> 'b) ->
    ('a, 'c) rw_prot ref
end

We can now write interesting programs:

utop # let rec foo r =
  let v,r = read r in
  let r = write r (string_of_int (v+1)) in
  let v,r = read r in
  let r = write r (int_of_string v) in
  foo r;;
val foo :
  (int,
   [> `Read of int * 'a
    | `Write of
        string *
        (string,
         [> `Read of string * 'b | `Write of int * (int, 'a) rw_prot ] as 'b)
        rw_prot ]
   as 'a)
  rw_prot PolyRef.ref -> 'c = <fun>

Observe that foo reads r as a integer, updates it as a string, reads it as a string and then finally writing an integer into it. The inferred type reflects this change from int -> string -> int. The implementation of polymorphic references uses the unsafe Obj.magic to coerce the contents. However, the behavioural types ensure that accesses are safe.

module PolyRef : PolyRef =
struct
  type ('a,'b) rw_prot
    constraint 'b = [> `Read of 'a * 'b | `Write of 'c * ('c,_) rw_prot]

  type 'a ref =
    {contents     : 'b.'b;
     mutable live : bool} (* For linearity *)
     constraint 'a = ('b,'c) rw_prot

  let ref v = {contents = Obj.magic v; live = true}

  let check r =
    if not r.live then raise LinearityViolation;
    r.live <- false

  let fresh r = {r with live = true}

  let read r =
    check r;
    (Obj.magic r.contents, fresh r)

  let write r v =
    check r;
    { contents = Obj.magic v; live = true }

  let branch r _ = check r; fresh r
end

File descriptors

We can utilise behavioural types to apply meaningful restrictions to operations on file descriptors.

module type File_descriptor = sig
  type 'a t constraint 'a = [>]

  val openfile : string -> Unix.open_flag list -> Unix.file_perm ->
    ([< `Close | `Write of 'a | `Read of 'a > `Close] as 'a) t
  val close : [> `Close] t -> unit
  val read : [> `Read of 'a] t -> bytes -> int -> int -> int * 'a t
  val write : [> `Write of 'a] t -> bytes -> int -> int -> int * 'a t
  val mk_read_only  : [> `Read of 'a] t -> ([`Close | `Read of 'a] as 'a) t
  val mk_write_only : [> `Write of 'a] t -> ([`Close | `Write of 'a] as 'a) t

  val open_stdin  : unit -> ([`Close | `Read of 'a] as 'a) t
  val open_stdout : unit -> ([`Close | `Write of 'a] as 'a) t
  val open_stderr : unit -> ([`Close | `Write of 'a] as 'a) t
end

The File_descriptor module is a thin wrapper around the file descriptors from Unix module. The file descriptor obtained through openfile permits a subset of operations to read, write and close. The precise set of capabilities is dictated by the flags supplied. For example, with O_RDONLY the type of the file descriptor obtained should be ([`Close | `Read of 'a] as 'a) t. The types of standard streams are also restricted. For example,

utop # open_stderr () |> fun fd -> write fd "hello\n" 0 6;;
hello
- : int * ([ `Close | `Write of 'a ] as 'a) t = (6, <abstr>)
utop # open_stdin () |> fun fd -> write fd "hello\n" 0 6;;
Error: This expression has type ([ `Close | `Read of 'a ] as 'a) t
       but an expression was expected of type [> `Write of [>  ] ] t
       The first variant type does not allow tag(s) `Write

File descriptors can also be made read or write only.

utop # let foo fd =
         let _, fd = write fd  "hello\n" 0 6 in
         let fd = mk_read_only fd in
         write fd "hello\n" 0 6;;
Error: This expression has type ([ `Close | `Read of 'a ] as 'a) t
       but an expression was expected of type [> `Write of [>  ] ] t
       The first variant type does not allow tag(s) `Write

The implementation of the module is straightforward.

module File_descriptor : File_descriptor = struct
  open Unix

  type 'a t =
    {fd : file_descr;
     mutable live : bool} constraint 'a = [>]

  let mk fd = {fd = fd; live = true}

  let fresh fd = {fd with live = true}

  let check fd =
    if not fd.live then raise LinearityViolation;
    fd.live <- false

  let open_stdin () = mk stdin
  let open_stdout () = mk stdout
  let open_stderr () = mk stderr

  let openfile file flags perm =
    let fd = openfile file flags perm in
    mk fd

  let close fd = check fd; close fd.fd

  let read fd buff ofs len =
    check fd;
    (read fd.fd buff ofs len, fresh fd)

  let write fd buff ofs len =
    check fd;
    (write fd.fd buff ofs len, fresh fd)

  let mk_read_only fd = check fd; fresh fd
  let mk_write_only fd = check fd; fresh fd
end

Tracking Aliases

The final example I will discuss is alias tracking.

module type Alias = sig
  type ('a,'b) t constraint 'b = [>]
  val make   : (unit -> 'a) -> ('a, [`One]) t
  val dup    : ('a, 'b) t -> ('a,[`Succ of 'b]) t * ('a, [`Succ of 'b]) t
  val merge  : ('a, [`Succ of 'b]) t -> ('a, [`Succ of 'b]) t -> ('a, 'b) t
  val free   : ('a, [`One]) t -> ('a -> unit) -> unit
  val app    : ('a,'b) t -> ('a -> unit) -> unit
end

module Alias : Alias = struct
  type ('a,'b) t =
    {v : 'a; mutable live : bool} constraint 'b = [>]

  let fresh a = {a with live = true}

  let check a =
    if not a.live then raise LinearityViolation;
    a.live <- false

  let make f = {v = f (); live = true}
  let dup x = check x; (fresh x, fresh x)
  let merge x y = check x; check y; fresh x
  let free x f = check x; f x.v
  let app x f = f x.v
end

The type variable 'b tracks aliases as a depth in the aliasing tree. New resources are initialised with make, and the resultant resource has type ('a,[`One]) t indicating that there is just one reference to this resource. Aliases are created explicitly with dup, which destroys the original reference and returns two new references, each one level deeper than the original reference. Two references from the same level can be merged together to obtain a reference at the next higher level, and in doing so destroying the original references. All of this machinery is to ensure that the resource can only be freed when there is a unique reference.

utop # let r = make (fun _ -> ref 0);;
val r : (int ref, [ `One ]) t = <abstr>
utop # let r1,r2 = dup r;;
val r1 : (int ref, [ `Succ of [ `One ] ]) t = <abstr>
val r2 : (int ref, [ `Succ of [ `One ] ]) t = <abstr>
utop # let r11,r12 = dup r1;;
val r11 : (int ref, [ `Succ of [ `Succ of [ `One ] ] ]) t = <abstr>
val r12 : (int ref, [ `Succ of [ `Succ of [ `One ] ] ]) t = <abstr>
utop # let r21, r22 = dup r2;;
val r21 : (int ref, [ `Succ of [ `Succ of [ `One ] ] ]) t = <abstr>
val r22 : (int ref, [ `Succ of [ `Succ of [ `One ] ] ]) t = <abstr>
utop # let r1 = merge r11 r22;;
val r1 : (int ref, [ `Succ of [ `One ] ]) t = <abstr>
utop # let r2 = merge r12 r21;;
val r2 : (int ref, [ `Succ of [ `One ] ]) t = <abstr>
utop # free (merge r1 r2);;
- : (int ref -> unit) -> unit = <fun>

Conclusion

Polymorphic variants are quite effective in encoding behavioural types. However, the absence of linear types in OCaml makes us resort to dynamic tests for linear use of resources. While it is possible to hide the resource under a monad, combining the use of multiple resources would require monad transformers, which is well known to be quite heavyweight in terms of programmability. Perhaps an effect system would do the trick.

Lock-free programming for the masses

2016-06-11T09:08:00+00:00

Efficient concurrent programming libraries are essential for taking advantage of fine-grained parallelism on multicore hardware. In this post, I will introduce reagents, a composable, lock-free concurrency library for expressing fine-grained parallel programs on Multicore OCaml. Reagents offer a high-level DSL for experts to specify efficient concurrency libraries, but also allows the consumers of the libraries to extend them further without knowing the details of the underlying implementation.

Motivation

Designing and implementing scalable concurrency libraries is an enormous undertaking. Decades of research and industrial effort has led to state-of-the-art concurrency libraries such as java.util.concurrent (JUC) for the JVM and System.Collections.Concurrent (SCC) for the .NET framework. These libraries are often written by experts and have subtle invariants, which makes them hard to maintain and improve. Moreover, it is hard for the library user to safely combine multiple atomic operations. For example, while JUC and SCC provide atomic operations on stacks and queues, such atomic operations cannot be combined into larger atomic operations.

On the other hand software transactional memory (STM) offers composability, but STM based data structures are generally less efficient than their lock-free counterparts, especially when there is moderate to high levels of contention. Aaron Turon introduced reagents, an expressive and composable library which retains the performance and scalability of lock-free programming. Reagents allow isolated atomic updates to shared state, as well as message passing communication over channels. Furthermore, reagents provide a set of combinators for sequential composition à la STM, parallel composition à la Join calculus, and selective communication à la Concurrent ML, while being lock-free. Reagents occupy this sweet-spot between expressivity and performance, and we believe it could serve as a great default¹ for writing fine-grained concurrent programs in Multicore OCaml.

Combinators

The basic reagents combinators are presented below.

type ('a,'b) t

(* channel communication *)
val swap : ('a,'b) endpoint -> ('a,'b) t

(* shared memory *)
val upd : 'a ref -> ('a -> 'b -> ('a * 'c) option) -> ('b,'c) t

(* sequential composition *)
val (>>>) : ('a,'b) t -> ('b,'c) t -> ('a,'c) t
(* conjunction *)
val (<*>) : ('a,'b) t -> ('a,'c) t -> ('a,'b * 'c) t
(* disjunction *)
val (<+>) : ('a,'b) t -> ('a,'b) t -> ('a,'b) t

val run : ('a,'b) t -> 'a -> 'b

A reagent value with type ('a,'b) t represents an atomic transaction that takes an input of type 'a and returns a value of type 'b. The basic atomic operations are exchanging message on an endpoint of a channel through swap and updating a shared reference through upd. The swap operation blocks the calling thread until a matching swap operation is available on the dual endpoint.

The atomic reference update operation upd, takes a function which is applied to the current value of the reference (of type 'a) and the input value (of type 'b), and is expected to return an optional pair of the new value for the reference and a return value (of type 'c). If the update function returns None, then the invoking thread blocks until the reference is updated. Reagent implementation takes care of the blocking and signalling necessary for thread wake up.

The most important feature of reagents is that it allows composition of reagent transactions in sequence >>> and in parallel <*>, and also to selectively choose one of the available operations <+>. Furthermore, these combinators being arrows, enable optimisations that cover the common case and help reagents achieve performance commensurate to hand-written implementations. Reagents library also exposes monadic combinators for convenience, at the cost of forgoing optimisation opportunities.

A lock-free stack

The following is a reagent implementation of the Treiber lock-free stack.

module R = Reagent

module TreiberStack : sig
  type 'a t
  val create  : unit -> 'a t
  val push    : 'a t -> ('a, unit) R.t
  val pop     : 'a t -> (unit, 'a) R.t
  val try_pop : 'a t -> (unit, 'a option) R.t
end = struct
  type 'a t = 'a list R.ref

  let create () = R.ref []

  let push r =
    R.upd r (fun xs x -> Some (x::xs,()))

  let try_pop r = R.upd r (fun l () ->
    match l with
    | [] -> Some ([], None)
    | x::xs -> Some (xs, Some x))

  let pop r = Ref.upd r (fun l () ->
    match l with
    | [] -> None
    | x::xs -> Some (xs,x))
end

We utilise a shared reference of type 'a list ref to represent the stack and use the upd operation to perform atomic operations on the stack. The important take away from this snippet is that the code is no more complicated than a sequential stack implementation. The logic for backoff, retry, blocking and signalling are hidden behind the reagents implementation. In particular, the pop operation blocks the calling thread until the stack is non-empty. Thus, the experts can write efficient concurrency libraries using reagents while preserving readability (and as a consequence maintainability) of code.

Furthermore, since the stack interface is exposed as reagents, the individual operations can be further composed. For example, given two Treiber stacks s1 and s2, pop s1 >>> push s2 transfers elements atomically between the stacks, pop s1 <*> pop s2 consumes elements atomically from both of the stacks, and pop s1 <+> pop s2 consumes an element from either of the stacks. Importantly, the composition preserves the optimisations and blocking/signalling behaviours, allowing the users of the library to arbitrarily combine and extend the functionality without knowing about the underlying implementation.

Feeding the philosophers

The parallel composition combinator provides an elegant way to solve the Dining Philosophers problem. The problem imagines a set of philosophers seated around a circular table, forever alternating between thinking and eating. Forks are placed between adjacent philosophers, and each philosopher can only eat after obtaining both the left and right forks. The goal is to design a solution where no philosopher will starve. The problem highlights the issues of deadlock and fairness in concurrent programming.

One way to solve this problem is to model each fork as a pair of endpoints, one for taking and another for dropping the fork.

type fork =
  {drop : (unit,unit) endpoint;
   take : (unit,unit) endpoint}

let mk_fork () =
  let drop, take = mk_chan () in
  {drop; take}

let drop f = swap f.drop
let take f = swap f.take

Now, the solution for a single round of eating can be implemented as follows:

let eat l_fork r_fork =
  ignore @@ run (take l_fork <*> take r_fork) ();
  (* ...
   * eat
   * ... *)
  spawn @@ run (drop l_fork);
  spawn @@ run (drop r_fork)

We use take l_fork <*> take r_fork to atomically take both of the forks. Reagents ensure that the protocol does not deadlock. After eating, we release the forks by spawning lightweight threads. In the next round, the philosophers race for the available forks. If the thread scheduler is fair, then the protocol provides fairness among the philosophers. The complete solution is available here.

Implementation

The key idea behind the implementation is that the reagent transaction executes in two phases. The first phase involves collecting all the compare-and-swap (CAS) operations necessary for the transaction, and the second phase is invoking a k-CAS operation (emulated in software). The failure to gather all the available CASes constitutes a permanent failure, causing the thread to explore other alternatives in the case of a selective communication or block otherwise. The failure in the second phase means that there is active interference from other concurrent threads, in which case the transaction is retried.

Performance of the Reagents depends critically on having fine-grained control over threads and schedulers for implementing backoff loops, blocking and signalling. However, one of the main ideas of multicore OCaml is not to bake in the thread scheduler into the compiler but rather describe them as libraries. To this end, the reagents library is functorized over the following generic scheduler interface:

module type Scheduler = sig
  (* continuation *)
  type 'a cont
  effect Suspend : ('a cont -> 'a option) -> 'a
  effect Resume  : 'a cont * 'a -> unit
end

The interface itself only describes the scheduler’s effects, whose behaviour is defined by the handlers. perform (Suspend f) applies f to the current continuation, and allows the Reagent library to stash the thread on the unavailable resource’s wait queue. The return type of f is an option to handle the case when the resource might have become available while suspending. If f returns None, then the control returns to the scheduler. Once the resource becomes available, the reagent library performs the Resume effect to resume the suspended thread.

Comparison to STM

Reagents are less expressive than STM, which provides serializability. But in return, Reagents provide stronger progress guarantee (lock-freedom) over STM (obstruction-freedom)². A reagent transaction operating more than once on the same memory location will fail at runtime. Abstractly, this behaviour is disallowed since it cannot be represented as a k-CAS operation. Due to this restriction, the transaction pop s1 >>> push s1 always fails, and prohibits important patterns such as atomically pushing or popping multiple values from the same stack. I am currently working on extending the reagents semantics to relax this invariant. The resultant behaviour will be similar to a version of snapshot isolation. While this is weaker than serializability semantics offered by the STM, we will retain the benefit of lock-freedom.

Contribute!

Using the reagents library, we have implemented a collection of composable concurrent data and synchronization structures such as stacks, queues, countdown latches, reader-writer locks, condition variables, exchangers, atomic counters, etc. There is great opportunity here to build a standard library for fine-grained parallelism for Multicore OCaml, incorporating the latest developments in lock-free data structures. There is still work to be done optimising the implementation to remove allocations in the fast path, and fine-tuning the reagents core.

Contributions to the library are most welcome, and is a great way to contribute to the Multicore OCaml effort. Please do file those issues and submit pull-requests.

Reagents is just a library, and you can implement your own favourite concurrent programming library. ↩
And then there are good arguments to why the semantics should be even weaker. ↩

Armed with Reason

2016-05-16T10:00:05+00:00

This is a short tutorial on how to build Reason apps for an ARM target with the help of Docker. I am using Docker for Mac, which is still under beta program. Using Docker for development has two important advantages over traditional cross-compilation. First, the Reason toolchain comes packaged as a Docker image and hence no local installation is necessary. Secondly, cross-compilers are often tricky to get right. Docker for Mac comes with multiarch support and hence removes the need for traditional cross-compilation.

Setup

I will be testing using a Cubietruck running Linaro Desktop. But these instructions should also work for Raspbian, a Debian optimized for the Raspberry pi hardware.

Build

First get the dockerfile for Reason toolchain and build the image.

$ mkdir /tmp/reason_arm
$ cd /tmp/reason_arm
$ wget https://gist.githubusercontent.com/kayceesrk/dc37a6ffeeda2dea338550dd4e8ad7ec/raw/8e136b8b8170758bd5e9c0cacf70fed4f9ce3df1/Dockerfile
$ docker build -t reason-arm .

All set! Let’s build a “Hello, World!” program.

$ mkdir /tmp/reason_arm_hello
$ cd /tmp/reason_arm_hello
$ echo 'print_endline "Hello, Reason!"' > hello.re
$ docker run -it -v `pwd`:/src reason-arm
$ cd /src
$ rebuild hello.native
^C

The build artifacts are found in the host machines /tmp/reason_arm_hello/_build directory.

$ file _build/hello.native
_build/hello.native: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, not stripped

We can now transfer the file to the cubietruck and run it. My cubietruck’s IP address is 192.168.0.9.

$ scp _build/hello.native linaro@192.168.0.9:
$ ssh linaro@192.168.0.9
Welcome to Linaro 13.04 (GNU/Linux 3.4.61+ armv7l)

* Documentation:  https://wiki.linaro.org/
Last login: Fri May 20 08:35:01 2016 from 192.168.0.3
linaro@cubietruck:~$ ./hello.native
Hello, Reason!

Profiling the stack

2015-10-27T17:29:30+00:00

In the last post, I described a flat allocation profiler for OCaml 4.02 bytecode interpreter. In this post, I’ll describe further developments which add support for call stack information and better location information. Lets dive straight to the usage:

Enabling stack profiling

Stack profiling is enabled by setting the environment variable CAML_PROFILE_STACK to the intended depth of stack. Suppose we would like to attribute any allocation to the current function, we would set CAML_PROFILE_STACK=1. To do the same to the current function and its caller, we would set CAML_PROFILE_STACK=2. CAML_PROFILE_STACK=<INFINITY> should give you stack profile all the way down to the first function.

Why should I care about the stack depth?

Because it affects the program performance. Enabling stack profiling walks the stack for every allocation. This has the potential to severely affect the program performance. Most often, with a flat profile, you’ve tracked the offending allocation to some function in the standard library such as¹:

File "bytes.ml", line 59, characters 7-81:
  C_CALL1 caml_create_string

File "src/bigstring.ml", line 98, characters 20-37:
  C_CALL1 caml_create_string

And all you want is to find out the caller of that standard library function in your code. A stack depth of a small number should provide you this information. You might have to play around with the stack depth to identify what you are looking for.

Profiling N-queens

You can obtain and install the profiling enabled OCaml 4.02 here. Let us obtain the flat profile first.

$ wget http://caml.inria.fr/pub/old_caml_site/Examples/oc/basics/queens.ml
$ ocamlc -o queens -g queens.ml
$ CAML_PROFILE_ALLOC=queens.preprof ./queens
Chess boards's size ? 8
The 8 queens problem has 92 solutions.

Do you want to see the solutions <n/y> ? n
$ ./tools/allocprof queens.preprof queens.prof
$ head -n10 queens.prof
Total: 77,863 words
Instr   Words   % of total      Location
-----   -----   ----------      --------
2488    31440   40.38%          file "list.ml", line 55, characters 32-39
27681   31440   40.38%          file "queens.ml", line 61, characters 46-52
27775   5895    7.57%           file "queens.ml", line 38, characters 2-113
27759   4112    5.28%           file "queens.ml", line 40, characters 33-41
27687   3930    5.05%           file "queens.ml", line 61, characters 14-59
2403    86      0.11%           file "pervasives.ml", line 490, characters 8-63
5391    44      0.06%           file "list.ml", line 20, characters 15-29

Observe that we now have the precise location information directly in the profile, whereas earlier one had to manually identify the source location using the instruction information. In this profile, we see that most allocations were in list.ml:55, which is the List.map function. However, we cannot pin down the source of these allocations in queens.ml from this profile since the profile is flat. Let us now obtain the stack allocation profile, which will reveal the source of these allocations in queens.ml.

$ CAML_PROFILE_ALLOC=queens.preprof CAML_PROFILE_STACK=10000 ./queens
Chess boards's size ? 8
The 8 queens problem has 92 solutions.

Do you want to see the solutions <n/y> ? n
$ ./tools/allocprof queens.preprof queens.prof --sort-stack
$ head -n10 queens.prof
Total: 77,863 words
Instr   Current Cur %   Stack   Stack % Location
-----   ------- -----   -----   ------- --------
27836   0       0.00%   76911   98.78%  file "queens.ml", line 100, characters 33-42
27549   0       0.00%   76870   98.72%  file "queens.ml", line 85, characters 17-36
27466   0       0.00%   76473   98.21%  file "queens.ml", line 45, characters 18-31
27715   0       0.00%   65117   83.63%  file "queens.ml", line 62, characters 4-22
27694   0       0.00%   62880   80.76%  file "queens.ml", line 61, characters 31-59
2487    0       0.00%   55020   70.66%  file "list.ml", line 55, characters 32-39
2483    0       0.00%   31440   40.38%  file "list.ml", line 55, characters 20-23

I’ve chosen a stack depth of 10000 to obtain the complete stack profile of the program. The option --sort-stack to allocprof sorts the results based on the stack allocation profile. We can now clearly see the stack of functions that perform most allocations. The line

27836   0       0.00%   76911   98.78%  file "queens.ml", line 100, characters 33-42

says that 98.78% of all allocations were performed by the function at queens.ml:100, characters 33-42, and its callees. This isn’t surprising since this function is the top-level main function! More interesting is the 98.21% of allocations on queens.ml:45. This is the recursive call to the concmap function, which in turn invokes the List.map function on queens.ml:61. We’ve now pinned down the source of the allocation in list.ml:55 to queens.ml:61.

Caveats and conclusions

Unlike stack profiles of C programs, OCaml’s stack profile does not include all the functions in the call stack since many calls are in tail positions. Calls to functions at tail position will not have a frame on the stack, and hence will not be included in the profile.

Please do submit issues and bug-fixes. Pull-requests are welcome! Also, here is my trimmed down (yay \o/!), non-exhaustive wish list of features:

Dump the profile every few milliseconds to study the allocation behavior of programs over time.
Save the location information in the object header and dump the heap at every GC to catch space leaks.

Thanks to trevorsummerssmith for the example. ↩

An Allocation Profiler for OCaml Bytecode Interpreter

2015-09-23T09:51:30+00:00

This post describes a simple flat allocation profiler for OCaml 4.02 bytecode interpreter.

OCaml is a strongly typed functional language with automatic memory management. Automatic memory management alleviates the need to manually deal with memory memory management, and by construction, avoids a large class of bugs. However, abstractions are not free in OCaml. Unlike MLton, a whole-program optimizing Standard ML compiler, which I used to hack on in an earlier life, in OCaml, one needs to be particularly aware of the cost of introducing abstractions such as higher-order functions and modules. This is often at odds with desirable programming patterns one tends to embrace in a higher-order modular functional language. Writing performance sensitive code in OCaml remains a skill that is acquired gradually through experience.

There are of course, excellent resources available to understand the performance implications of OCaml abstractions. However, often times, I simply need a way to profile and uncover performance bottlenecks in my program, before I can apply any targeted optimizations. Profiling along the following three axes are particularly useful: time, counts and allocations. OCaml has good support for two of these. While ocamlcp with ocamlprof gives you count profile, one can use the standard Unix profiler gprof for time profiling. However, these do not necessarily help with identifying the cost of abstractions, for which one needs an allocation profiler¹.

The state of allocation profiling in OCaml

While allocation profiler is not part of the standard OCaml distribution, several alternatives do exist. Memprof from OCamlPro provides “non-intrusive memory profiler for OCaml applications”, with a simple online version and a commercial version with fine-grained tracing. Mark Shinwell has an allocation profiler for OCaml 4.02 native code programs generated by ocamlopt. Unfortunately, neither of these options were suitable for me as the Multicore OCaml currently only supports bytecode compilation, and has a markedly different GC. So I decided to implement my own for the multicore compiler. Since the allocation profiler will be useful in general, I have also ported it to OCaml 4.02. This post talks about the vanilla OCaml allocation profiler.

Bytecode allocation profiler

The idea of this allocation profiler is to record the allocations and associate them with the position in the code where the corresponding block or closure was allocated. In particular, we do not record the call stack that led to the allocation point, which would have provided us a more accurate picture. One can get pretty far with just the flat profile. Running the bytecode program under the modified interpreter produces a profile, which is then analyzed offline.

The bytecode interpreter of OCaml is remarkably simple, as is the patch for the allocation profiler. In this section, I will detail the implementation of the profiler. If you are interested in just using the profiler, do skip right to the instructions.

When the bytecode is loaded by the interpreter in caml_load_code, it allocates an array for the bytecode. caml_start_code points to the start of this array. The program counter pc is a pointer into this array. We maintain a distinct code pointer profile_pc that always points to the instruction and never its operands. The offset of profile_pc from caml_start_code uniquely identifies a instruction in the bytecode executable. We will use this offset to record the allocation points.

We allocate an array caml_profile_counts of unsigned integers whose length is equal to the length of the code, into which we will store the allocation counts. There are two main ways in which OCaml allocates memory; Alloc_small for allocating in minor heap, and caml_alloc_shr for allocating in major heap. We modify both to record the allocations at a given instruction. We modify interp.c to update profile_pc for instructions which potentially allocate. Allocations for arrays and strings are performed in their corresponding C functions through caml_alloc. Such allocations are covered by recording the instruction in Setup_for_c_call.

caml_alloc_shr is also used by the GC for promoting live minor heap objects to major heap at the end of a minor GC cycle. Allocations by GC is ignored by resetting profile_pc to NULL before minor collections. Hence, the profiler only counts allocations by the mutator. Finally, the interpreter outputs the profile at the end of execution of the program.

#Using the profiler

In order to use the profiler, compile the OCaml programs with the bytecode compiler ocamlc with -g option to record the debugging information. This will be used to interpret the profile. When using ocamlbuild it is necessary to compile and link with -g (with -cflag -g -lflag -g).

First, get OCaml 4.02 with the allocation profiler, and build it using opam-compiler-conf:

$ git clone https://github.com/kayceesrk/ocaml
$ cd ocaml
$ git checkout 4.02-profile-alloc
$ opam compiler-conf configure
$ make world.opt
$ opam compiler-conf install

Let us profile the Eight Queens program. Profiling is enabled by setting the CAML_PROFILE_ALLOC to the output filename of the profile.

$ wget http://caml.inria.fr/pub/old_caml_site/Examples/oc/basics/queens.ml
$ ocamlc -o queens -g queens.ml
$ CAML_PROFILE_ALLOC=queens.preprof ./queens
Chess boards's size ? 8
The 8 queens problem has 92 solutions.

Do you want to see the solutions <n/y> ? n
$ ./tools/allocprof queens.preprof > queens.prof
$ head -n5 queens.prof
Total: 80,433 words
Instr   Words   % of total
-----   -----   ----------
2488    31440   39.09%
27681   31440   39.09%

allocprof is a small python script that post-processes the profile. The post-processed profile shows the total number of words allocated, and is followed by the instruction number, words allocated and the percentage of total allocation that it represents. The instruction number can be linked back to the source code by dumping the bytecode executable with dumpobj.

$ ./tools/dumpobj queens > queens.dumpobj
$ vim queens.prof queens.dump queens.ml

We can see that the program spent 39.09% of allocations for appending to lists in queens.ml line 61. For the curious, the other 39.09% was spent in List.map function.

Dealing with early termination

The profiler normally writes out the profile at the end of the standard program termination, when the interpreter has run to completion. However, programs may terminate early by explicitly invoking exit. In such cases, the runtime does not get a chance to output the profile. Hence, a function output_profile: unit -> unit is provided to explicitly request the profile to be written out to the filename provided in CAML_PROFILE_ALLOC. The following example illustrates the use case in a program that uses the Async library:

(* foo.ml *)
open Core.Std
open Async.Std

let main () =
  printf "Hello!\n";
  (* Without this call, profile isn't written out *)
  output_profile ();
  return ()

let () =
  Command.async_basic
    ~summary:"foo"
    Command.Spec.(empty)
    main
  |> Command.run

The program is compiled and run as follows:

$ ocamlbuild -use-ocamlfind foo.byte -package core -package async -tag thread -tag debug
Finished, 3 targets (0 cached) in 00:00:00.
$ CAML_PROFILE_ALLOC=foo.preprof ./foo.byte
Hello!
$ ls foo.preprof
foo.preprof

Thanks to trevorsummerssmith for the motivation and the example.

Conclusion

The allocation profiler has been quite useful for optimizing small programs. It would be interesting to see whether it scales to larger ones. Also, here is my (non-exhaustive) wish list of features:

Improve tooling. Avoid the need to manually search through text files.
Record stack allocation. This is especially important in multicore OCaml since stacks are heap allocated.
Record the call stack information for allocations to get an informative profile.
Dump the profile every few milliseconds to study the allocation behavior of programs over time.
Save the location information in the object header and dump the heap at every GC to catch space leaks.

Profiling for time does give you the time that the program spends in garbage collection functions such as minor GC cycles and major GC slices, but are not helpful for pinpointing allocation bottlenecks. ↩

Experiment with OCaml Multicore and Algebraic Effects

2015-09-10T13:11:00+00:00

I recently gave a talk on Algebraic Effects in OCaml at the OCaml Workshop 2015. The extended abstract and the slides from the talk are available here. The slides should provide a gentle introduction to programming with algebraic effects and handlers in OCaml. The examples from the talk (and many more!) are available here.

Algebraic effects in OCaml are available as a part of the multicore OCaml. The experimental compiler could easily be installed using the OCaml Labs opam development repo.

$ opam remote add ocamllabs -k git https://github.com/ocamllabs/opam-repo-dev
$ opam switch 4.02.2+multicore

If you are interested in contributing, please do experiment with algebraic effects, and report any inevitable bugs or feature requests to the multicore OCaml issue tracker.

We are also quite interested in hearing interesting applications of algebraic effects such as the encoding of monadic reflection and one-shot multi-prompt delimited control. Feel free to submit pull requests with your examples.

Pearls of Algebraic Effects and Handlers

2015-05-27T14:06:00+00:00

In the previous post, I presented a simple cooperative multithreaded scheduler written using algebraic effects and their handlers. Algebraic effects are of course useful for expressing other forms of effectful computations. In this post, I will present a series of simple examples to illustrate the utility of algebraic effects and handlers in OCaml. Some of the examples presented here were borrowed from the excellent paper on Eff programming language¹. All of the examples presented below are available here.

State

We can use algebraic effects to model stateful computation, with the ability to retrieve (get) and update (put) the current state:

module type STATE = sig
  type t
  val put : t -> unit
  val get : unit -> t
  val run : (unit -> unit) -> init:t -> unit
end

The function run runs a stateful computation with the given initial state. Here is the implementation of the module State which provides the desired behaviour:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
module State (S : sig type t end) : STATE with type t = S.t = struct
  type t = S.t

  effect Put : t -> unit
  let put v = perform (Put v)

  effect Get : t
  let get () = perform Get

  let run f ~init =
    let comp =
      match f () with
      | () -> (fun s -> ())
      | effect (Put s') k -> (fun s -> continue k () s')
      | effect Get k -> (fun s -> continue k s s)
    in comp init
end

The key idea here is that the handler converts the stateful computation to functions that accept the state. For example, observe that if the function f returns a unit value (line 13), we return a function which accepts a state s and returns unit. The handler for effect Get (line 15) passes the current state s to the continuation k. The expression continue k s returns a function that accepts the current state and returns unit. Since fetching the current state does not modify it, we apply this function to s, the original state. Since Put modifies the state (line 14), the function returned by continue k () is applied to the new state s'. We evaluate the computation by applying it to the initial state init (line 16).

Observe that the implementation of the handler for the stateful computation is similar to the implementation of State monad in Haskell. Except that in Haskell, you would have the stateful computation f have the type State t (), which says that f is a stateful computation where t is the type of state and () the type of return value. Since multicore OCaml does not have a effect system, f simply has type unit -> unit as opposed to being explicitly tagged with the effects being performed. While the OCaml type of f under specifies the behaviour of f, it does allow you to combine various kinds of effects directly, without the need for monad transformer gymnastics². For example, the following code snippet combines an int and string typed state computations, each with its own handler:

module IS = State (struct type t = int end)
module SS = State (struct type t = string end)

let foo () : unit =
  printf "%d\n" (IS.get ());
  IS.put 42;
  printf "%d\n" (IS.get ());
  IS.put 21;
  printf "%d\n" (IS.get ());
  SS.put "hello";
  printf "%s\n" (SS.get ());
  SS.put "world";
  printf "%s\n" (SS.get ())

let _ = IS.run (fun () -> SS.run foo "") 0

which prints:

0
42
21
hello
world

References

We can expand upon our state example, to model ML style references:

module State : sig
    type 'a t

    val ref  : 'a -> 'a t
    val (!)  : 'a t -> 'a
    val (:=) : 'a t -> 'a -> unit

    val run  : (unit -> 'a) -> 'a
  end = struct

  type 'a t = {inj : 'a -> Univ.t; prj : Univ.t -> 'a option}

  effect Ref : 'a -> 'a t
  let ref v = perform (Ref v)

  effect Read : 'a t -> 'a
  let (!) = fun r -> perform (Read r)

  effect Write : 'a t * 'a -> unit
  let (:=) = fun r v -> perform (Write (r,v))

  let run f =
    let comp =
      match f () with
      | v -> (fun s -> v)
      | effect (Ref v) k -> (fun s ->
          let (inj, prj) = Univ.embed () in
          let cont = continue k {inj;prj} in
          cont (inj v::s))
      | effect (Read {inj; prj}) k -> (fun s ->
          match find prj s with
          | Some v -> continue k v s
          | None -> failwith "Ref.run: Impossible -> ref not found")
      | effect (Write ({inj; prj}, v)) k -> (fun s ->
          continue k () (inj v::s))
    in comp []
end

The idea is to represent the state as a list of universal typed values, references as a record with inject and project functions to and from universal type values, assign as appending a new value to the head of the state list, and dereference as linear search through the list for a matching assignment. The universal type implementation is due to Alan Frisch.

Transactions

We may handle lookup and update to implement transactions that discards the updates to references in case an exception occurs:

  let atomically f =
    let comp =
      match f () with
      | x -> (fun _ -> x)
      | exception e -> (fun rb -> rb (); raise e)
      | effect (Update (r,v)) k -> (fun rb ->
          let old_v = !r in
          r := v;
          continue k () (fun () -> r := old_v; rb ()))
    in comp (fun () -> ())

Updating a reference builds up a rollback function that negates the effect of the update. In case of an exception, the rollback function is evaluated before re-raising the exception. For example, in the following code snippet:

exception Res of int

let () = atomically (fun () -> (* T0 *)
  let r = ref 10 in
  printf "T0: %d\n" (!r);
  try atomically (fun () -> (* T1 *)
    r := 20;
    r := 21;
    printf "T1: Before abort %d\n" (!r);
    raise (Res !r);
    printf "T1: After abort %d\n" (!r);
    r := 30)
  with
  | Res v -> printf "T0: T1 aborted with %d\n" v;
  printf "T0: %d\n" !r)

the updates to reference r by transaction T1 are discarded on exception and the program prints the following:

T0: 10
T1: Before abort 21
T0: T1 aborted with 21
T0: 10

From Iterators to Generators

An iterator is a fold-function of type ('a -> unit) -> unit, that iterates a client function over all the elements of a data structure. A generator is a function of type unit -> 'a option that returns Some v each time the function is invoked, where v is the next-element in the data structure. The function returns None if the traversal is complete. Unlike an iterator, the generator hands over control of the traversal to the client of the library.

Gabriel Scherer’s insightful article on generators, iterators, control and continuations nicely distinguish, motivate and provide implementation of different kinds of iterators and generators for binary trees. While the iterator implementation is obvious and straight-forward, the generator implementation requires translating the code to CPS style and manually performing simplifications for efficient traversal. Since algebraic effects handlers give us a handle to the continuation, we can essentially derive the generator implementation from the iterator.

Let us consider a binary tree with the following type:

type 'a t = Leaf | Node of 'a t * 'a * 'a t

We can define an iterator that traverses the tree from left to right as follows:

let rec iter f = function
  | Leaf -> ()
  | Node (l, x, r) -> iter f l; f x; iter f r

From this iterator, we derive the generator as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
let to_gen (type a) (t : a t) =
  let module M = struct effect Next : a -> unit end in
  let open M in
  let step = ref (fun () -> assert false) in
  let first_step () =
    try
      iter (fun x -> perform (Next x)) t;
      None
    with effect (Next v) k ->
      step := continue k;
      Some v
  in
    step := first_step;
    fun () -> !step ()

At each step of the iteration, we perform the effect Next : a -> unit (line 7), which is handled by saving the continuation to a local reference and returning the value (line 9 - 11). Since the effect handlers are provided with the continuation, we are able to invert the control from the library to the client of the library. This avoids the need to perform manual CPS translation.

Direct-style asynchronous IO

Since the effect handler has access to the continuation, we can implement minimal asynchronous IO in direct-style as opposed to the monadic style of asynchronous IO libraries such as Lwt and Async. Our asynchronous IO library has the following interface:

module type AIO = sig

  val fork  : (unit -> unit) -> unit
  val yield : unit -> unit

  type file_descr = Unix.file_descr
  type sockaddr = Unix.sockaddr
  type msg_flag = Unix.msg_flag

  val accept : file_descr -> file_descr * sockaddr
  val recv   : file_descr -> bytes -> int -> int -> msg_flag list -> int
  val send   : file_descr -> bytes -> int -> int -> msg_flag list -> int
  val sleep  : float -> unit

  val run : (unit -> unit) -> unit
end

Observe that the return type of the non-blocking function calls accept, recv, send and sleep are the same as their blocking counterparts from Unix module.

The asynchronous IO implementation works as follows. For each blocking action, if the action can be performed immediately, then it is. Otherwise, the thread performing the blocking task is suspended and add to a pool of threads waiting to perform IO:

(* Block until data is available to read on the socket. *)
effect Blk_read  : file_descr -> unit
(* Block until socket is writable. *)
effect Blk_write : file_descr -> unit
(* Sleep for given number of seconds. *)
effect Sleep : float -> unit

let rec core f =
  match f () with
  ...
  | effect (Blk_read fd) k ->
      if poll_rd fd then continue k ()
      else (Hashtbl.add read_ht fd k;
            dequeue ())
  | effect (Blk_write fd) k ->
      if poll_wr fd then continue k ()
      else (Hashtbl.add write_ht fd k;
            dequeue ())
  | effect (Sleep t) k ->
        if t <= 0. then continue k ()
        else (Hashtbl.add sleep_ht (Unix.gettimeofday () +. t) k;
              dequeue ())

let accept fd =
  perform (Blk_read fd);
  Unix.accept fd

let recv fd buf pos len mode =
  perform (Blk_read fd);
  Unix.recv fd buf pos len mode

let send fd bus pos len mode =
  perform (Blk_write fd);
  Unix.send fd bus pos len mode

The scheduler works by running all of the available threads until there are no more threads to run. At this point, if there are threads that are waiting to complete an IO operation, the scheduler invokes select() call and blocks until one of the IO actions becomes available. The scheduler then resumes those threads whose IO actions are now available:

(* When there are no threads to run, perform blocking io. *)
let perform_io timeout =
  let rd_fds = Hashtbl.fold (fun fd _ acc -> fd::acc) read_ht [] in
  let wr_fds = Hashtbl.fold (fun fd _ acc -> fd::acc) write_ht [] in
  let rdy_rd_fds, rdy_wr_fds, _ = Unix.select rd_fds wr_fds [] timeout in
  let rec resume ht = function
  | [] -> ()
  | x::xs ->
      enqueue (Hashtbl.find ht x);
      Hashtbl.remove ht x;
      resume ht xs
  in
  resume read_ht rdy_rd_fds;
  resume write_ht rdy_wr_fds;
  if timeout >= 0. then ignore (wakeup (Unix.gettimeofday ())) else ();
  dequeue ()

The program implements a simple echo server. The server listens on localhost port 9301. It accepts multiple clients and echoes back to the client any data sent to the server. This server is a direct-style reimplementation of the echo server found here, which implements the echo server in CPS style:

(* Repeat what the client says until the client goes away. *)
let rec echo_server sock addr =
  try
    let data = recv sock 1024 in
    if String.length data > 0 then
      (ignore (send sock data);
       echo_server sock addr)
    else
      let cn = string_of_sockaddr addr in
      (printf "echo_server : client (%s) disconnected.\n%!" cn;
       close sock)
  with
  | _ -> close sock

The echo server can be tested with a telnet client by starting the server and on the same machine running telnet localhost 9301.

Conclusion

The aim of the post is to illustrate the variety of alternative programming paradigms that arise due to algebraic effects and handlers, and hopefully kindle interest in reasoning and programming with effects and handlers in OCaml. Algebraic effects and handlers support in OCaml is in active development within the context of multicore OCaml. When you find those inevitable bugs, please report them to the issue tracker.

Effective Concurrency with Algebraic Effects

2015-05-20T14:04:00+00:00

Algebraic effects and handlers provide a modular abstraction for expressing effectful computation, allowing the programmer to separate the expression of an effectful computation from its implementation. In this post, I will present an extension to OCaml for programming with linear algebraic effects, and demonstrate its use in expressing concurrency primitives for multicore OCaml. The design and implementation of algebraic effects for multicore OCaml is due to Leo White, Stephen Dolan and the multicore team at OCaml Labs.

Motivation

Multicore-capable functional programming language implementations such as Glasgow Haskell Compiler, F#, Manticore and MultiMLton expose one or more libraries for expressing concurrent programs. The concurrent threads of execution instantiated through the library are in turn multiplexed over the available cores for speed up. A common theme among such runtimes is that the primitives for concurrency along with the concurrent thread scheduler is baked into the runtime system. Over time, the runtime system itself tends to become a complex, monolithic piece of software, with extensive use of locks, condition variables, timers, thread pools, and other arcana. As a result, it becomes difficult to maintain existing concurrency libraries, let alone add new ones. Such lack of malleability is particularly unfortunate as it prevents developers from experimenting with custom concurrency libraries and scheduling strategies, preventing innovation in the ecosystem. Our goal with this work is to provide a minimal set of tools with which programmers can implement new concurrency primitives and schedulers as OCaml libraries.

A Taste of Effects

A Simple Scheduler

Let us illustrate the algebraic effect extension in multicore OCaml by constructing a concurrent round-robin scheduler with the following interface:

(* Control operations on threads *)
val fork  : (unit -> unit) -> unit
val yield : unit -> unit
(* Runs the scheduler. *)
val run   : (unit -> unit) -> unit

The basic tenet of programming with algebraic effects is that performing an effectful computation is separate from its interpretation¹. In particular, the interpretation is dynamically chosen based on the context in which an effect is performed. In our example, spawning a new thread and yielding control to another are effectful actions, for which we declare the following effects:

type _ eff +=
| Fork  : (unit -> unit) -> unit eff
| Yield : unit eff

The type 'a eff is the predefined extensible variant type for effects, where 'a represents the return type of performing the effect. For convenience, we introduce new syntax using which the same declarations are expressed as follows:

effect Fork  : (unit -> unit) -> unit
effect Yield : unit

Effects are performed using the primitive perform of type 'a eff -> 'a. We define the functions fork and yield as follows:

let fork f = perform (Fork f)
let yield () = perform Yield

What is left is to provide an interpretation of what it means to perform fork and yield. This interpretation is provided with the help of handlers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
let run main =
  let run_q = Queue.create () in
  let enqueue k = Queue.push k run_q in
  let rec dequeue () =
    if Queue.is_empty run_q then ()
    else continue (Queue.pop run_q) ()
  in
  let rec spawn f =
    match f () with
    | () -> dequeue ()
    | exception e ->
        print_string (to_string e);
        dequeue ()
    | effect Yield k ->
        enqueue k; dequeue ()
    | effect (Fork f) k ->
        enqueue k; spawn f
  in
  spawn main

The function spawn f (line 8) evaluates f in a new thread of control. f may return normally with value () or exceptionally with an exception e or effectfully with the effect performed along with the delimited continuation² k. In the pattern effect e k, if the effect e has type 'a eff, then the delimited continuation k has type ('a,'b) continuation, i.e., the return type of the effect 'a matches the argument type of the continuation, and the return type of the delimited continuation is 'b.

Observe that we represent the scheduler queue with a queue of delimited continuations, with functions to manipulate the queue (lines 2–6). In the case of normal or exceptional return, we pop the scheduler queue and resume the resultant continuation using the continue primitive (line 6). continue k v resumes the continuation k : ('a,'b) continuation with value v : 'a and returns a value of type 'b. In the case of effectful return with Fork f effect (lines 16–17), we enqueue the current continuation to the scheduler queue and spawn a new thread of control for evaluating f. In the case of Yield effect (lines 14–15), we enqueue the current continuation, and resume some other saved continuation from the scheduler queue.

Testing the scheduler

Lets write a simple concurrent program that utilises this scheduler, to create a binary tree of tasks. The sources for this test are available here. The program concurrent.ml:

let log = Printf.printf

let rec f id depth =
  log "Starting number %i\n%!" id;
  if depth > 0 then begin
    log "Forking number %i\n%!" (id * 2 + 1);
    Sched.fork (fun () -> f (id * 2 + 1) (depth - 1));
    log "Forking number %i\n%!" (id * 2 + 2);
    Sched.fork (fun () -> f (id * 2 + 2) (depth - 1))
  end else begin
    log "Yielding in number %i\n%!" id;
    Sched.yield ();
    log "Resumed number %i\n%!" id;
  end;
  log "Finishing number %i\n%!" id

let () = Sched.run (fun () -> f 0 2)

generates a binary tree of depth 2, where the tasks are numbered as shown below:

The program forks new tasks in a depth-first fashion and yields when it reaches maximum depth, logging the actions along the way. To run the program, first install multicore OCaml compiler, available from the OCaml Labs dev repo. Once the compiler is installed, the above test program can be compiled and run as follows:

$ git clone https://github.com/kayceesrk/ocaml-eff-example
$ cd ocaml-eff-example
$ make
$ ./concurrent
Starting number 0
Forking number 1
Starting number 1
Forking number 3
Starting number 3
Yielding in number 3
Forking number 2
Starting number 2
Forking number 5
Starting number 5
Yielding in number 5
Forking number 4
Starting number 4
Yielding in number 4
Resumed number 3
Finishing number 3
Finishing number 0
Forking number 6
Starting number 6
Yielding in number 6
Resumed number 5
Finishing number 5
Finishing number 1
Resumed number 4
Finishing number 4
Finishing number 2
Resumed number 6
Finishing number 6

The output illustrates how the tasks are forked and scheduled.

Implementation

Fibers for Concurrency

The main challenge in the implementation of algebraic effects is the efficient management of delimited continuations. In multicore OCaml³, the delimited continuations are implemented using fibers, which are small heap-allocated, dynamically resized stacks. Fibers represent the unit of concurrency in the runtime system.

Our continuations are linear (one-shot)⁴, in that once captured, they may be resumed at most once. Capturing a one-shot continuation is fast, since it involves only obtaining a pointer to the underlying fiber, and requires no allocation. OCaml uses a calling convention without callee-save registers, so capturing a one-shot continuation requires saving no more context than that necessary for a normal function call.

Since OCaml does not have linear types, we enforce the one-shot property at runtime by raising an exception the second time a continuation is invoked. For applications requiring true multi-shot continuations (such as probabilistic programming⁵), we envision providing an explicit operation to copy a continuation.

While continuation based concurrent functional programming runtimes such as Manticore and MultiMLton use undelimited continuations, our continuations are delimited. We believe delimited continuations enable complex nested and hierarchical schedulers to be expressed more naturally due to the fact that they introduce parent-child relationship between fibers similar to a function invocation.

Running on Multiple Cores

Multicore OCaml provides support for shared-memory parallel execution. The unit of parallelism is a domain, each running a separate system thread, with a relatively small local heap and a single shared heap shared among all of the domains. In order to distributed the fibers amongst the available domains, work sharing/stealing schedulers are initiated on each of the domains. The multicore runtime exposes to the programmer a small set of locking and signalling primitives for achieving mutual exclusion and inter-domain communication.

The multicore runtime has the invariant that there are no pointers between the domain local heaps. However, the programmer utilising the effect library to write schedulers need not be aware of this restriction as fibers are transparently promoted from local to shared heap on demand. We will have to save multicore-capable schedulers for another post.

Opam Switch to Multicore OCaml

2015-03-25T18:15:00+00:00

OPAM has a great compiler switch feature that lets you simultaneously host several OCaml installations, each with its own compiler version and a set of installed packages. I wanted to use the power of opam switch for working with the experimental multicore OCaml compiler. The key advantage of doing this is that it lets you easily install packages from the OPAM repository, while sandboxing it from other OCaml installations on your system. The post will show how to create OPAM compiler switch for multicore OCaml.

Install opam-compiler-conf

The first step is to install Gabriel Scherer’s opam-compiler-conf script which lets you do opam switches on local installations:

$ git clone https://github.com/gasche/opam-compiler-conf
$ cd opam-compiler-conf
$ mkdir -p ~/.local/bin
$ make BINDIR=~/.local/bin install

This installs the opam-compiler-conf script under ~/.local/bin. Make sure this directory is under your search path. Now, $opam compiler-conf should give you the list of available commands.

Build multicore OCaml locally

Typing opam switch should list the compilers currently installed in your system and those that are available. For instance, here is my setup:

$ opam switch
system  C system  System compiler (4.02.1)
4.02.1  I 4.02.1  Official 4.02.1 release
4.02.0  I 4.02.0  Official 4.02.0 release
4.01.0  I 4.01.0  Official 4.01.0 release
--     -- 3.11.2  Official 3.11.2 release
--     -- 3.12.1  Official 3.12.1 release
--     -- 4.00.0  Official 4.00.0 release
--     -- 4.00.1  Official 4.00.1 release
# 66 more patched or experimental compilers, use '--all' to show

You can easily switch between the installations using opam switch [system-name]. Let us now install multicore OCaml as a new switch:

$ git clone https://github.com/ocamllabs/ocaml-multicore
$ cd ocaml-multicore
$ opam compiler-conf configure
$ make world
$ opam compiler-conf install
$ eval `opam config env`

The multicore compiler is now installed and has been made the current compiler:

$ opam switch
system                      I system                      System compiler (4.02.1)
4.02.1+local-git-multicore  C 4.02.1+local-git-multicore  Local checkout of 4.02.1 at /Users/kc/ocaml-multicore
4.02.1                      I 4.02.1                      Official 4.02.1 release
4.02.0                      I 4.02.0                      Official 4.02.0 release
4.01.0                      I 4.01.0                      Official 4.01.0 release
--                         -- 3.11.2                      Official 3.11.2 release
--                         -- 3.12.1                      Official 3.12.1 release
--                         -- 4.00.0                      Official 4.00.0 release
--                         -- 4.00.1                      Official 4.00.1 release
# 66 more patched or experimental compilers, use '--all' to show

This can be confirmed by:

$ ocamlc -version
4.02.1+multicore-dev0

which shows the current OCaml bytecode compiler version.

Working with the local switch

Every time you change the compiler source, you need to rebuild the compiler and reinstall the switch:

# Changed compiler source...
$ make world
$ opam compiler-conf reinstall

The local installation can be removed by opam compiler-conf uninstall.

KC Sivaramakrishnan - Scalable Functional Programming

Shrinking the OxCaml js_of_ocaml bundle: 285 MB to 4 MB

Why bundle size matters

Why 285 MB?

The other recipe, and why it doesn’t compose

The patch

Wiring it through x-ocaml

Numbers

A few more snags

The cell

What next?

Capsules: compile-time lock discipline in OxCaml

Why atomics aren’t enough

Gensym in a capsule

The brand 'k: an existential tied to a fresh capsule

The bare ref is unreachable from outside the capsule

Cap.Data.t mode-crosses contention and portability

with_lock produces an access token, briefly

Brand mismatch is a type error

What each mode is doing

A note on the API

What’s left

Data race freedom in OxCaml

A quick aside: data races in OCaml are less scary

Hello, OxCaml

A racy gensym

Two new modes for data race freedom

Contention rejects mutable writes

Portability rejects captured refs

The trap, and the actual rule

Captured vs parameter, in code

A formal aside: defaults and submoding

Back to gensym: the fix

What’s left

From Convergence to Confidence: Push-button verification for RDTs

Convergence is not enough

A short history of how we tried to close that gap

What Sal actually is

The Peritext story

Beyond Peritext

The past three weeks

Closing

Foundations for hacking on OCaml

Testing x-ocaml, OCaml notebooks as a WebComponent

Reverse-mode AD using Effects

Using other libraries

What next?

Linearity and uniqueness

Capturing unique values

A linear ref

Why both linearity and uniqueness?

Uniqueness is more appropriate for safe refs

Past and the future

Conclusions

Acknowledgements

Uniqueness for Behavioural Types

Behavioural types and runtime overhead

Setting up OCaml with modes

An explicitly memory-managed reference

Modes.Aliased.t

Implementation

Refs that explain their work

Where next

Addendum

Footnotes

Joining my group

Internship positions

PhD/MS/MTech positions

Contributing to the OCaml community

Research Associate positions

Summary

Off-CPU-time analysis

Installation

Enabling frame pointers

OCaml

glibc

Running

Caveats

Oddities

Other links

Wiring it through `x-ocaml`

The brand `'k`: an existential tied to a fresh capsule

The bare `ref` is unreachable from outside the capsule

`Cap.Data.t` mode-crosses contention and portability

`with_lock` produces an `access` token, briefly

A racy `gensym`

Back to `gensym`: the fix