unikernel/duniverse/dune_/doc/dev/rpc-versioning.md

@@ -0,0 +1,180 @@
+# Runtime RPC versioning implementation notes
+
+This document describes the versioning protocol used to ensure two-way
+compatibility between different versions of the API.
+
+The approach is loosely inspired by the `Both_converts` model used by
+[Versioned_rpc](https://ocaml.janestreet.com/ocaml-core/latest/doc/async_rpc_kernel/Async_rpc_kernel/index.html#module-Versioned_rpc)
+in `Async`, in which both parties maintain a "menu" of supported RPC
+versions, which is used to negotiate a common protocol for each
+method.
+
+This is a working document and will be updated as the design evolves.
+
+## Terms
+
+- A **procedure** is a common term encompassing *notifications*
+    (one-way messages) and *requests* (a communication to which
+    a response is expected).
+
+- A **model** is the logical payload type for one direction of
+    a procedure. Note that this is a per-actor entity; the ultimate
+    goal of runtime RPC versioning is to allow clients and servers to
+    disagree on a model type without preventing them from interacting.
+
+- A **wire type** for a procedure is the logical type sent "over the
+    wire" for one direction of a procedure. The end result of
+    negotiating a version for a procedure is to select a wire type
+    known to both the client and the server. Typically, the older of
+    the two model types will be chosen as the wire type,
+
+- A **generation** of a procedure is the set of wire types
+    corresponding to each direction of a procedure, along with the
+    de/serialisation logic and upgrade and downgrade functions
+    transforming the wire types to the model types and vice versa.
+    Each generation is associated with a *version number*, which
+    should be unique within a procedure.
+
+- The **menu** is a mapping from method names to the particular
+    generations that will be used for each procedure for a particular
+    session. In the source code, this term is overloaded to also refer
+    to a mapping from method names to *all known* generations of
+    a procedure.
+
+- The **declaration** of a procedure lists its model types and all
+    known generations, along with its method name. Multiple
+    declarations of the same procedure is allowed, so long as they do
+    not overlap version numbers.
+
+- The **implementation** of a declaration is the actual behavior of
+    a procedure, which acts on the model types. Typically, this will
+    be on the server, but in the future there may also be a use for
+    server-to-client requests. This document is not concerned with the
+    internals of any given implementation, only whether such an
+    implementation exists at all.
+
+## Background
+
+Previously, there was no distinction between model and wire types.
+This meant that any change to a model type required both build servers
+and clients to upgrade in lockstep, as otherwise the receiver would be
+unable to deserialize the payload of a procedure.
+
+Unfortunately, most lighter-weight solutions (such as modifying the
+de/serialization logic to be resilient to, e.g., extra/missing fields
+or variants in types) are insufficient. Early designs of the
+diagnostic API, for example, reported targets as strings, but was
+changed to give structured information instead.
+
+Similarly, requirements like "the client must always be older than the
+server" (or the reverse) don't work in environments like Jane Street,
+where the same editor plugin must be able to interact seamlessly with
+multiple iterations of Dune (which may be older or newer than the
+editor plugin itself).
+
+The main goal of the system, then, is to ensure that both the server
+and client applications can be programmed against the current model
+types for each procedure, with all backwards- or forwards- conversions
+happening under the hood.
+
+## Protocol
+
+At session initialization time, the client will first send an
+initialization request to the server containing a single version
+number corresponding to the overall RPC version the client will use.
+If this number is determined to be versioning-compatible (see
+[Session versioning](#session-versioning)), the server will respond
+with a token instructing the client to initiate version negotiation.
+Otherwise, the server will respond with an error.
+
+Upon receiving this token, the client will initiate version
+negotiation by sending a list of `(method-name, generations)`
+pairs, where `method-name` is the name of each declared procedure, and
+`generations` is the list of version numbers for that procedure's
+generations.
+
+Upon receiving a list of supported versions from the client, the
+server will compare it to its list of *implemented* versions,
+selecting the greatest common generation for each procedure. If the
+client and server do not share any generations for a procedure, it is
+omitted entirely. If there is at least one method for which a common
+version exists, then the server responds with a list of `(method-name,
+selected-version)` pairs, where `selected-version` is the version
+number of the greatest common generation. This list is then used by
+both parties to construct the version menu. Otherwise, if there are no
+common versions for any methods, an error is returned to the client
+and the session is invalidated.
+
+Note that we do not currently require declared/implemented versions to
+span a contiguous range of version numbers. This can have a few uses,
+such as preventing clients from using a known-bugged generation of
+a procedure.
+
+When executing a procedure, the sender first looks up the correct
+generation in the menu (see [Error handling](#error-handling)), and
+downgrades the payload from the sender-side model type to the wire
+type. Upon receipt, the server performs the same lookup to deserialize,
+then upgrade the payload to the receiver-side model type, then the
+procedure implementation is performed, producing a response in the
+case of requests. If necessary, the same transformations are then
+performed in reverse, sending the value back to the sender, completing
+the procedure.
+
+Barring strange circumstances (such as a client declaring a generation
+with a newer version number than the type exposed in `dune_rpc.mli`), it
+is always the case that the transformation from wire to model types will
+be the identity function on the side that is older.
+
+## Miscellaneous implementation notes
+
+### Session versioning
+
+In addition to version numbers existing for each procedure version,
+there are two further version numbers associated with the session as
+a whole which are sent as part of session initialization.
+
+The first is the version of Dune each side purports to be as
+a `MAJOR.MINOR` number (serialized as an `int * int` pair). This is
+not currently checked.
+
+Next is a version of the initial handshake protocol to be used. This
+takes the form of a single `int`. In the future, if the initial
+negotiation protocol changes, this value can be adjusted and checked to
+account for this.
+
+### Error handling
+
+Handling of versioning errors has become more complex, as we need to
+distinguish between "no such method exists" and "the server and client
+do not share any common generations for this method". Secondly, this
+means that the initiation of a procedure can now fail, which
+complicates one-way communications (for example, the server must
+swallow errors and clients must be upgraded to handle version errors
+on notifications, which were previously infallible).
+
+Finally, the versioning protocol itself must be either versioned
+separately or stabilised (see [Session versioning](#session-versioning)).
+
+### Tweaks
+
+- We currently send the entire version menu from client to server and
+    back twice, once for the client to inform the server of all
+    supported versions, and again for the server to inform the client
+    of the common versions. This can lead to large messages being
+    passed at session initialization, which may become a performance
+    bottleneck.
+
+    - The size of the version negotiation messages is proportional to
+        the number of all known generations for all procedures, which
+        can be approximated by `number-of-procedures` times
+        `number-of-supported-generations`. In practice, I do not
+        expect this number to be large (I would be surprised if this
+        number is ever on the order of 100).
+
+    - One alternative is to perform per-procedure negotiation, where
+        the initiator of a procedure first sends its known version
+        ranges, the recipient sends the selected version (or an
+        error), then the procedure proceeds as before. This approach
+        trades startup and lookup overhead for a constant
+        per-communication overhead. It also makes distinguishing "no
+        such method exists" and "no common versions" simpler.