The Dune Mental Model ===================== It is not strictly necessary to understand Dune's underlying model to use it; but knowing how it works under the hood will help writing build rules, and also help understand some errors and what's possible with Dune. .. note:: This document is a simplification of the reality: the actual rules might be different, it does not touch rule loading and glosses over how caching works, but should be a useful tool to build an understanding of Dune. How Dune Works -------------- The building block of Dune is the *rule*: A *rule* reads *dependencies* and writes *targets* using an *action* (and it can be attached to *aliases*). When ``dune build`` is executed, it will first read the project's ``dune`` files to determine the rules that apply to the project. Once it has done this, it will determine what actions it needs to execute to build the required targets. An Example ---------- Let's take the following example. - there's a CLI tool written in OCaml. - it has some build-time configuration stored in ``config.json``. - it has an integration test, in which the tool is executed with ``testdata.txt`` as input. Configuration Generation ^^^^^^^^^^^^^^^^^^^^^^^^ To express the generation of the configuration module we could write: .. code:: dune (rule (deps convert/json2ml.exe config.json) (target config.ml) (action (run convert/json2ml.exe config.json -o config.ml))) This rule will: - read its dependencies: ``convert/json2ml.exe`` and ``config.json`` - and write its target: ``config.ml`` - using an action: ``(run convert/json2ml.exe config.json -o config.ml)`` This rule is very explicit: we write a stanza for a single Dune rule. Building the Executable ^^^^^^^^^^^^^^^^^^^^^^^ In contrast, to describe the compilation of the executable, we would write: .. code:: dune (executable (name tool) (modules main config)) Here, we use Dune's abstractions. Dune knows about the OCaml compilation model: the modules need to be compiled and linked together. So it will generate the following rules under the hood: - one rule to compile the ``Main`` module: - it will read its dependency: ``main.ml`` - and write its output: ``main.cmx`` - using an action: ``(run ocamlopt -c main.ml)`` - one rule to compile the ``Config`` module: - it will read its dependency: ``config.ml`` - and write its output: ``config.cmx`` - using an action: ``(run ocamlopt -c config.ml)`` - one rule to link the ``tool.exe`` executable: - it will read its dependencies: ``main.cmx`` and ``config.cmx`` - and write its output: ``tool.exe`` - using an action: ``(run ocamlopt -o tool.exe main.cmx config.cmx``) Note that in this example, some files are targets of a rule and dependencies of another (``.cmx`` files). We are unlikely to ever interact with them directly, so it can also be useful to think of the ``(executable)`` stanza as a group of rules with ``main.ml`` and ``config.ml`` as inputs and ``tool.exe`` as output. Running the Tests ^^^^^^^^^^^^^^^^^ Some rules do not produce any output file, but we're still interested in running their actions. A test is a good example: we want the build process to exit with an error code if the action fails. In that case, the rule does not have targets, but we "attach" it to an :term:`alias`, ``runtest`` in this case. This gives us a way of requesting this rule to be executed. As we are about to see, rules are executed lazily by asking for their targets to be built, so we would not be able to execute such rules. .. code:: dune (rule (deps tool.exe testdata.txt) (alias runtest) (action (run tool.exe testdata.txt))) This rule: - reads its dependencies: ``tool.exe`` and ``testdata.txt`` - writes no targets - using an action: ``(run tool.exe testdata.txt)`` - (and it is attached to ``runtest``) What to Build ------------- Dune can build *files* and *aliases*. These can be found on the command line: - ``dune build tool.exe`` will build the ``tool.exe`` file. - ``dune build @example`` will build the ``example`` alias. - ``dune build tool.exe @example`` will build both the file ``tool.exe`` and the ``example`` alias. - ``dune runtest`` is a shortcut for ``dune build @runtest``: it will build the ``runtest`` alias. Passing a directory will build all tests in that directory. Passing the path to a cram test will run that test individually. - ``dune build`` is a shortcut for ``dune build @@default``: it will build the default alias in the current directory (by default the ``all`` alias). In other words, each ``dune build`` or ``dune runtest`` command always corresponds to a list of files and aliases to build. .. seealso:: :doc:`Reference information on aliases` How Dune Interprets Rules ------------------------- We have now seen that Dune sets up rules for a project, and that every build command has a list of files and aliases that we are asking to build. Now let's see how this request is processed: - to build a file, Dune will first check if it is in the source tree. In that case, there is nothing to do. Otherwise, it will check if it is the target of a rule. In that case, it will execute this rule. (Dune will raise an error in other cases: if the file is both in the source tree and the target of a rule, or if it is neither) - to build an alias, Dune will execute all the rules that are attached to this alias. - to execute a rule, Dune will first build all the dependencies (files or aliases) of this rule. Then it will execute the action attached to the rule. When Dune is about to execute an action, it checks (in various caches) if it executed it before on the same set of dependencies, and, if yes, it can skip executing it and reuse the previous result. In the case of our example, if we call ``dune runtest``, Dune will consider all rules attached to the ``runtest`` alias. In this case it is just the integration test rule. It needs to build its dependencies, ``tool.exe`` and ``testdata.txt``. The latter is present in the source tree. However, ``tool.exe`` is the target of the linking rule defined by the ``(executable)`` stanza. This rule requires ``main.cmx`` and ``config.cmx``. ``main.cmx`` is the target of the compilation rule for the ``Main`` module, which depends on ``main.ml``. This file is in the source tree, so let's copy it under ``_build``. This rule has all its dependencies available, so we can run its action, which writes ``main.cmx``. Getting back to the dependencies of ``tool.exe``, ``config.cmx`` is the target of the linking rule of the ``Config`` module. This rule has ``config.ml`` has a dependency. This file is itself the target of the configuration module rule, which lists ``config.json`` and ``convert/json2ml.exe``. The first is available in the source tree and to simplify, let's assume that the second one has been built. This action has all its dependencies available, so we can execute its action to produce its target, ``config.ml``. Now the module compilation rule for ``Config`` can be executed, producing ``config.cmx``; and in turn the linking rule can be executed, producing ``tool.exe``. Finally, ``tool.exe`` can be executed with ``testdata.txt`` as its argument. In a nutshell: we recursively copied all the dependencies of the test rule, and executed the rules in the correct order. This is a "cold build", where there were no previous build artifacts. Note that if we change only part of the project (say the ``main.ml`` file), only a small number of rules will be evaluated, the ones that depend on ``main.ml``. Conclusion ---------- Dune's underlying model is based on rules. Stanzas are high-level constructs that can generate multiple rules, that are not always visible. To build a target, Dune looks for the rule that produces that target and makes its way back to source files. Rules define a directed acyclic graph which models dependency relations between files. Most of the rules in that graph may be executed for a cold build, but just the minimum will be executed for an incremental build.