Async File I/O

Landing now—Hemlock’s async file I/O. We no longer use blocking system calls (open(2) , close(2), read(2) , write(2) ) for file operations. We do all of our I/O asynchronously via calls to io_uring_enter(2) .

New additions to our File API reflect the nature of asynchronous I/O.

module Open: sig
  type file = t
  type t
  (* An internally immutable token backed by an external I/O open completion
     data structure. *)

  val submit: ?flag:Flag.t -> ?mode:uns -> Bytes.Slice.t -> (t, Error.t) result
  (** [submit ~flag ~mode path] submits an open operation for a file at [path]
      with [flag] (default Flag.R_O) Unix file permissions and [mode] (default
      0o660) Unix file permissions. This operation does not block. Returns a
      [t] to the open submission or an [Error.t] if the open could not be
      submitted. *)

  val submit_hlt: ?flag:Flag.t -> ?mode:uns -> Bytes.Slice.t -> t
  (** [submit ~flag ~mode path] submits an open operation for a file at [path]
      with [flag] (default Flag.R_O) Unix file permissions and [mode] (default
      0o660) Unix file permissions. This operation does not block. Returns a
      [t] to the open submission or halts if the open could not be submitted.
  *)

  val complete: t -> (file, Error.t) result
  (** [complete t] blocks until given [t] is complete. Returns a [file] or an
      [Error.t] if the file could not be opened. *)

  val complete_hlt: t -> file
  (** [complete_hlt t] blocks until given [t] is complete. Returns a [file] or
      halts if the file could not be opened. *)
end

The underlying implementation of the old “blocking” API simply waits for completions without performing any intermediary work.

let of_path_hlt ?flag ?mode path =
  Open.(submit_hlt ?flag ?mode path |> complete_hlt)

Hemlock submit and complete signatures reflect that submissions must be completed on the same actor since they’re mutable.

Open = {
    type t: &t
    submit: ?flag:Flag.t -> ?mode:uns -> Bytes.Slice.t >{os}-> &result &t Error.t
    complete: !&t >{os}-> result file Error.t
}

In Hemlock, calling complete will idle the actor, yielding thread control back to the parent executor until the associated completion actually completes and the executor resumes the actor. In our bootstrap environment, we effectively have a single-actor, no-executor environment so this yielding behavior will come later when we build the Hemlock runtime.

Associated calls to submit and complete can be composed so that the composition remains immutable (File.of_path is a composition of File.Open.submit and File.Open.complete).

of_path: ?flag:Flag.t -> ?mode:uns -> Bytes.Slice.t >{os}-> result file Error.t

It remains valid to embed such composed calls in closures that reside in the global heap. This means that our File.Stream.t still works as expected, even though calls to Lazy.force create temporary mutable &File.Read.t values in the executing actor’s context.

Stream = {
    type t: t = stream Bytes.Slice.t os
    of_file: file -> t
}

So far, building asynchronous I/O on top of io_uring(7) has been a pleasant experience. Thoroughly reading through the source code for Facebook’s past implementation of Folly::IOThreadPoolExecutor had me reeling at the prospect of building our own asynchronous I/O libraries on top of epoll(7) due to relatively poor ergonomics and especially the dragons that stir when dealing with persistent storage (I recall scary mitigations for unavoidable blocking even with O_NONBLOCK enabled). We’ve been able to keep our usage of io_uring incredibly basic and boring, which is awesome. There is more to build with it (e.g. filesystem operations, socket I/O) and I’m optimistic that io_uring will continue to impress.