Notes from the Jane Street talk

Notes about the tech talk given by Jane Street (http://www.cs.brown.edu/events/talks/janestreet.2.html)

Basic idea: make asynchronous, event driven programming easier to write. Multi-stage computations, in a traditional event-driven style, requires the programmer to explicitly represent the stages as part of the program state. This can lead to an “inversion-of-control” issue in traditional event-driven programs. If I have a program that looks like:

  • do some work
  • call to the network
  • wait till the network call resolves
  • do some other work

then a traditional event driven model requires the “do some other work” entry point to have some kind of representation in the program. Something naturally expressed as a single sequence becomes a bunch of shorter, chunked functions, and the toplevel event loop needs to dispatch to them.

Jane Street’s approach is to use their Async library to handle this gruntwork. Their approach, in retrospect, is very similar to that of PLT Web server’s send/suspend: it provides a mechanism for linguistically representing the blocking chunks of computation, and let the event loop handle the instruction threading from one stage to the next.

In Jane Street’s case with their async library, they use a Deferred type, and write code in monadic style. Monads are their mechanism for representing what happens next, after a blocking call returns. The event loop handles the appropriate state-threading from one phase to the next.

The essence of the approach is that the programmer is writing in continuation-passing-style. The server manages these continuations and invokes them when each blocking call resumes. One of the things that the Async library does is to make the syntax for writing in CPS easy to express. And the CPS parts are only around blocking parts, so rest of the code doesn’t have to look unnatural.

This style seems to easily account for sequential, blocking computations. It doesn’t directly account for situations where you need branches of computations. e.g. I do ten requests to the network, but when five come back, I want to continue the computation. It doesn’t appear to be easy to naturally express this without some additional primitives, and what the Async library provides (choose, or choice?) sounds like it doesn’t handle this case with particular ease.

Leave a Reply

Your email address will not be published. Required fields are marked *