unikernel/duniverse/dune_/doc/howto/bundle.rst

@@ -0,0 +1,41 @@
+How to Bundle Resources
+=======================
+
+This guide will show you how to configure Dune to generate modules with string resources
+from other files in your project.
+
+Folder Structure
+----------------
+
+.. code:: console
+
+    $ tree src
+    src
+    └── lib
+        └── my_lib
+            ├── dune
+            └── resources
+                └── site.css
+
+Dune Configuration
+------------------
+
+See :doc:`/reference/actions/progn` and
+:doc:`/reference/actions/with-outputs-to`.
+
+.. code:: dune
+
+    (rule
+     (with-stdout-to
+      css.ml
+      (progn
+       (echo "let css = {|")
+       (cat resources/site.css)
+       (echo "|}"))))
+
+Using the Bundled Resource
+--------------------------
+
+.. code:: ocaml
+
+   let () = Printf.printf "%s" Css.css

unikernel/duniverse/dune_/doc/howto/formatting.rst

@@ -0,0 +1,87 @@
+How to Set up Automatic Formatting
+==================================
+
+This guide will show you how to configure Dune so that it can check the formatting
+of your source code.
+
+Formatting is defined per project. This ensures that if a project is reused
+elsewhere, its formatting configuration will not interfere.
+
+Setting Up the Environment
+--------------------------
+
+First, let's open the ``dune-project`` file. Make sure that the version
+specified in ``(lang dune X.Y)`` is at least ``2.0``. Most formatting
+configuration happens in that file. If you want to format OCaml sources and
+``dune`` files, you don't have anything to add. Otherwise, refer to the
+:doc:`/reference/dune-project/formatting` stanza.
+
+Next we need to install some code formatting tools. For OCaml code, this means
+installing OCamlFormat_ with ``opam install ocamlformat``. Formatting ``dune``
+files is built into Dune and does not require any extra tools. For Reason code,
+this uses the ``refmt`` tool which is already installed if you are using Reason
+syntax in your project. If your project uses a :term:`dialect`, a specific tool
+might be required.
+
+.. _ocamlformat: https://github.com/ocaml-ppx/ocamlformat
+
+Using OCamlFormat requires some configuration. Take note of the version
+returned by ``ocamlformat --version`` (let's name that ``X.Y.Z``) and create an
+``.ocamlformat`` file in the same directory as ``dune-project`` with the
+following contents:
+
+.. code::
+
+   version=X.Y.Z
+   profile=default
+
+The ``version`` line is checked by OCamlFormat and ensures that everybody
+contributing to the project uses the same version.
+
+Note that you do not have to add ``ocamlformat`` to your opam files.
+
+Running the Formatters
+----------------------
+
+Run the ``dune build @fmt`` command. It will format the source files in the
+corresponding project and display the differences:
+
+.. code:: console
+
+    $ dune build @fmt
+    --- hello.ml
+    +++ hello.ml.formatted
+    @@ -1,3 +1 @@
+    -let () =
+    -  print_endline
+    -    "hello, world"
+    +let () = print_endline "hello, world"
+
+Then it's possible to accept the correction by calling ``dune promote`` to
+replace the source files with the corrected versions.
+
+.. code:: console
+
+    $ dune promote
+    Promoting _build/default/hello.ml.formatted to hello.ml.
+
+As usual with promotion, it's possible to combine these two steps by running
+``dune build @fmt --auto-promote``. This command can also be shortened to
+``dune fmt``. See :doc:`../concepts/promotion` for more details.
+
+Setting Up Your CI
+------------------
+
+To check formatting in CI, the precise set up depends on the CI system used,
+but in general it is easier to set up a dedicated job that just installs
+``dune`` and the formatting tools, rather than doing that as part of the jobs
+that run tests.
+
+If you use `ocaml-ci`_, you have nothing to do: a formatting job is set up
+automatically.
+
+If you use `setup-ocaml`_, you can use the `lint-fmt` extend listed in the
+README file.
+
+.. _ocaml-ci: https://ocaml.ci.dev/
+.. _setup-ocaml: https://github.com/ocaml/setup-ocaml

unikernel/duniverse/dune_/doc/howto/index.rst

@@ -0,0 +1,25 @@
+How-to Guides
+=============
+
+These guides will help you use Dune's features in your project.
+
+.. toctree::
+   :maxdepth: 1
+
+   install-dune
+   formatting
+   opam-file-generation
+   ../cross-compilation
+   ../foreign-code
+   ../documentation
+   ../sites
+   ../instrumentation
+   ../jsoo
+   ../wasmoo
+   ../melange
+   ../virtual-libraries
+   ../tests
+   bundle
+   toplevel
+   rule-generation
+   override-default-entrypoint

unikernel/duniverse/dune_/doc/howto/install-dune.rst

@@ -0,0 +1,39 @@
+How to Install Dune
+===================
+
+Dune is available as an Opam package. First, make sure that Opam is installed:
+
+  .. code:: console
+
+   $ opam --version
+   2.1.5
+
+Any version higher than 2.0.0 is supported, though preferably at least 2.1.0.
+
+If Opam is not available, follow `the official instructions on the Opam website
+<https://opam.ocaml.org/doc/Install.html>`_ to install it and then run its
+global setup with ``opam init``.
+
+.. note::
+
+   Opam requires a "shell hook" to work properly. Make sure to set it up
+   correctly during ``opam init``. Otherwise you will have to run ``eval $(opam
+   env)`` every time you create an Opam switch or change directory.
+
+Then, you can install Dune in an Opam switch using the following command:
+
+.. code:: console
+
+  $ opam install dune
+
+After the command completes, the following should display a version number:
+
+.. code:: console
+
+  $ dune --version
+  3.12.1
+
+.. note::
+
+   In most cases, when using Opam you will not need to install Dune by hand.
+   Installing the project's dependencies will install it in the Opam switch.

unikernel/duniverse/dune_/doc/howto/opam-file-generation.rst

@@ -0,0 +1,143 @@
+How to Generate Opam Files from ``dune-project``
+================================================
+
+.. highlight:: dune
+
+This guide will show you how to configure Dune so that it generates opam files.
+
+Declaring Package Dependencies
+------------------------------
+
+The goal of this first step is to add ``(package)`` stanzas in your
+``dune-project`` file. These stanzas declare the metadata that your package
+uses in the language of opam packages. See :ref:`declaring-a-package`.
+
+The next step depends on whether you are starting from a clean slate (new
+package) or adapting an existing opam file.
+
+For a New Package (No Existing Opam File)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If your project does not have any opam files, you will have to find your
+package dependencies. In the simple case, collect all the libraries
+that appear in the ``(libraries)`` fields of your project and put this list in
+the ``(depends)`` field of the corresponding ``(package)``. See
+:doc:`../explanation/ocaml-ecosystem` for the difference between libraries and
+packages.
+
+Example: you have a library that looks like::
+
+  (library
+   (public_name frobnitz)
+   (libraries lwt fmt))
+
+You can declare the package as::
+
+  (package
+   (name frobnitz)
+   (depends lwt fmt))
+
+Also add common metadata using ``(authors)``, ``(maintainers)``, ``(license)``,
+``(source)``, as well as a ``(synopsis)`` and a ``(description)`` for
+
+For an Existing Package
+^^^^^^^^^^^^^^^^^^^^^^^
+
+If you already have an opam file (or several of them), you can convert it by
+following the rules in :doc:`/reference/dune-project/package`.
+
+For example, if your opam file looks like:
+
+.. code:: opam
+
+   opam-version: 2.0
+   authors: ["Anil Madhavapeddy" "Rudi Grinberg"]
+   maintainer: ["team@mirage.org"]
+   name: "cohttp-async"
+   synopsis: "HTTP client and server for the Async library"
+   description: "A _really_ long description"
+   license: "ISC"
+   bug-reports: "https://github.com/mirage/ocaml-cohttp/issues"
+   homepage: "https://github.com/mirage/ocaml-cohttp/"
+   dev-repo: "git+https://github.com/mirage/ocaml-cohttp.git"
+   build: [
+     ["dune" "subst"] {dev}
+     [
+       "dune"
+       "build"
+       "-p"
+       name
+       "-j"
+       jobs
+       "@install"
+       "@runtest" {with-test}
+       "@doc" {with-doc}
+     ]
+   ]
+   depends: [
+     "dune" { >= "3.4" }
+     "odoc" { with-doc }
+     "cohttp" { >= "1.0.2" }
+     "conduit-async" { >= "1.0.3" }
+     "async" { >= "v0.10.0" }
+   ]
+   x-maintenance-intent: [ "(latest)" ]
+
+You can express this as::
+
+   (source (github mirage/ocaml-cohttp))
+   (license ISC)
+   (authors "Anil Madhavapeddy" "Rudi Grinberg")
+   (maintainers "team@mirage.org")
+   (maintenance_intent "(latest)")
+
+   (package
+    (name cohttp-async)
+    (synopsis "HTTP client and server for the Async library")
+    (description "A _really_ long description")
+    (depends
+     (cohttp (>= 1.0.2))
+     (conduit-async (>= 1.0.3))
+     (async (>= v0.10.0))))
+
+General Notes and Tips
+^^^^^^^^^^^^^^^^^^^^^^
+
+- Do not declare a dependency on the ``dune`` and ``odoc`` packages. Dune will
+  generate them with the right constraints.
+- For fields that are common between packages (like ``(authors)`` or
+  ``(license)``), you can use a global one rather than replicate it between
+  packages.
+- If you use a platform such as GitHub you can use ``(source)`` as a shorthand
+  instead of specifying ``(bug_reports)``, ``(homepage)``, etc.
+- ``(package)`` stanzas do not support all opam fields or complete syntax for
+  dependency specifications. If the package you are adapting requires this,
+  keep the corresponding opam fields in a ``pkg.opam.template`` file. See
+  :doc:`../reference/packages`.
+- It is not necessary to specify ``(version)``, this will be added at release
+  time if you use `dune-release <https://github.com/tarides/dune-release>`_.
+- To generate an opam variable such as ``version``, use a colon ``:`` followed
+  by the name of the variable. For example, to generate ``a { = version }`` in
+  the opam file, use ``(a (= :version))`` in ``dune-project``.
+
+Generating Opam Files
+---------------------
+
+If you have existing ``*.opam`` files, make a backup of them because the instructions in this section will overwrite them.
+
+Now that you have declared package metadata in ``dune-project``, you can add
+``(generate_opam_files)`` in ``(dune-project)``.
+
+From now on, commands like ``dune build`` and ``dune runtest`` are going to regenerate the contents of opam files from the metadata in ``(package)`` stanzas.
+If you only want to generate the opam file, run ``dune build <project_name>.opam``.
+
+Run ``dune build`` once and observe that the opam files have been created or
+updated. Make sure to add these changes to your version control system.
+
+.. seealso::
+
+   :token:`~pkg-dep:dep_specification`
+     How ``(depends)`` and similar fields are processed.
+
+   :doc:`/explanation/opam-integration`
+     How ``with-test`` and related variables are used by opam.

unikernel/duniverse/dune_/doc/howto/override-default-entrypoint.rst

@@ -0,0 +1,63 @@
+How to Override the Default C Entrypoint With C Stubs
+-----------------------------------------------------
+
+In some cases, it may be necessary to override the default C entry point of an
+OCaml program. For example, this is the case if you want to let your program
+handle argument wildcards expansion on Windows.
+
+Let's consider a trivial "Hello world" program contained in a ``hello.ml``
+file:
+
+.. code:: ocaml
+
+    let () = print_endline "Hello, world!"
+
+The default C entry point is a ``main`` function, originally defined in
+`runtime/main.c <https://github.com/ocaml/ocaml/blob/trunk/runtime/main.c>`_. It
+can be overridden by defining a ``main`` function that will at some point call
+the OCaml runtime. Let's write such a minimal example in a ``main.c`` file:
+
+.. code:: C
+
+    #include <stdio.h>
+
+    #define CAML_INTERNALS
+    #include "caml/misc.h"
+    #include "caml/mlvalues.h"
+    #include "caml/sys.h"
+    #include "caml/callback.h"
+
+    /* This is the new entry point */
+    int main(int argc, char_os **argv)
+    {
+        /* Here, we just print a statement */
+        printf("Doing stuff before calling the OCaml runtime\n");
+
+        /* Before calling the OCaml runtime */
+        caml_main(argv);
+        caml_do_exit(0);
+        return 0;
+    }
+
+The :doc:`foreign_stubs </reference/foreign-stubs>` stanza can be leveraged to
+compile and link our OCaml program with the new C entry point defined in
+``main.c``:
+
+.. code:: dune
+
+    (executable
+     (name hello)
+     (foreign_stubs
+      (language c)
+      (names main)))
+
+With this ``dune`` file, the whole program can be compiled by merely calling
+``dune build``. When run, the output shows that it calls the custom entry point
+we defined:
+
+.. code:: shell-session
+
+    $ dune build
+    $ _build/default/hello.exe
+    Doing stuff before calling the OCaml runtime
+    Hello, world!

unikernel/duniverse/dune_/doc/howto/rule-generation.rst

@@ -0,0 +1,266 @@
+Using Rule Generation
+=====================
+
+Sometimes it can be useful to generate Dune rules that depend on the file system
+layout or on the content of configuration files. This often happens for
+integration tests.
+
+In this document, we will see two ways to encode this behavior.
+
+We suppose that we are testing an executable named ``tool``. There are some
+input files named ``*.input``, output files named ``*.output``, and we want to
+ensure that when running ``tool`` on ``x.input``, the standard output
+corresponds to ``x.output``.
+
+The Generate-Include-Commit Pattern
+-----------------------------------
+
+.. note::
+
+   This is the most common way to do this. It has a couple drawbacks listed
+   below, but you should start with this pattern.
+
+What we are going to do is:
+
+- generate a ``dune.inc`` file;
+- include it in our main ``dune`` file;
+- commit the generated code in the source repository.
+
+This creates a loop: a program (the "generator") looks at the file system and
+creates a ``dune.inc`` file. Changes to the file system (for example, if a test
+is added) mean that a change in ``dune.inc`` will be promoted. These generated
+rules are included in the main ``dune`` file, so ``dune runtest`` will run the
+tests. Finally, the generated file is part of the source repository, so it is
+not necessary to run several commands to run the test suite.
+
+Let's expand a bit on how to achieve this.
+
+Generating a ``dune.inc`` File
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Create a ``gen`` subdirectory and create a ``gen.ml`` file in it:
+
+``gen/gen.ml``
+  .. code:: ocaml
+
+    let generate_rules base =
+      Printf.printf
+        {|
+            (rule
+             (with-stdout-to %s.gen
+              (run %%{bin:tool} %s.input)))
+        
+            (rule
+             (alias runtest)
+             (action
+              (diff %s.output %s.gen)))
+             |}
+        base base base base
+
+    let () =
+      Sys.readdir "."
+      |> Array.to_list
+      |> List.sort String.compare
+      |> List.filter_map (Filename.chop_suffix_opt ~suffix:".input")
+      |> List.iter generate_rules
+
+Create a ``dune`` file in that directory:
+
+``gen/dune``
+  .. code:: dune
+
+    (executable
+     (name gen))
+
+This defines an executable that lists ``*.input`` files in the current
+directory and outputs rules on its standard output.
+
+.. note::
+
+   It is important to sort the input files to ensure that the output is
+   independent from the order in which ``Sys.readdir`` returns the files .
+
+For each input file, we output two rules:
+
+- The first one creates a ``x.gen`` file that corresponds to the actual output.
+- The second uses a :doc:`/reference/actions/diff` action to compare the actual output to the expected output. If it is different, ``dune runtest`` will display the difference, which can be accepted by ``dune promote``.
+
+.. note::
+
+   It is possible to have more complicated logic here. For example, to pass
+   different arguments to ``tool`` depending on the presence of a ``*.args``
+   file. To do that, check if ``*.args`` exists in ``generate_rules`` and emit
+   a different ``(run ...)`` action.
+
+Including it in the Main ``dune`` File
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Our main test ``dune`` file contains the following:
+
+``dune``
+  .. code:: dune
+
+    (include dune.inc)
+
+    (rule
+     (deps (source_tree .))
+     (with-stdout-to
+      dune.inc.gen
+      (run gen/gen.exe)))
+
+    (rule
+     (alias runtest)
+     (action
+      (diff dune.inc dune.inc.gen)))
+
+In addition to including the contents of ``dune.inc``, we use the same pattern
+as before: ``dune.inc.gen`` is the actual output of the generator, and
+``dune.inc`` is the expected output. At runtime, the generator will read the
+contents of the current directory (where the ``*.input`` and ``*.output`` files
+are located), so we record ``(source_tree .)`` as a dependency to make it run
+again if a file is created, for example.
+
+Commit the Generated Code In The Source Repository
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To make this work, we have a final step to do. We have to add the generated
+file to our source tree. But since it is generated, we will have to first
+create an empty file, run the test, and promote the result.
+
+.. code:: console
+
+   $ touch dune.inc
+   $ dune runtest
+   + (rule
+   +  (with-stdout-to a.gen
+   +   (run %{bin:tool} a.input)))
+   +
+   + (rule
+   +  (alias runtest)
+   +  (action
+   +   (diff a.output a.gen)))
+   $ dune promote dune.inc
+   $ git add dune.inc
+
+Now, running ``dune runtest`` will run the test suite.
+
+Notes
+^^^^^
+
+This pattern is "correct": it will execute all tests and make sure the
+list of tests is up to date. But when adding a test, it is necessary to first
+run ``dune runtest``, promote the result, and then re-run ``dune runtest`` to
+actually run the test (and possibly promote the result of the test itself).
+
+There is a variant of this pattern which will promote the output automatically
+instead of using a manual promotion step. This variant can be used either for
+the test list or for the individual tests.
+
+To use it in the test list, replace the ``dune`` file by this version:
+
+``dune`` (alternative version)
+  .. code:: dune
+
+    (include dune.inc)
+
+    (rule
+     (mode promote)
+     (alias runtest)
+     (deps (source_tree .))
+     (with-stdout-to
+      dune.inc
+      (run gen/gen.exe)))
+
+Using this version, ``dune runtest`` will directly replace ``dune.inc`` with an
+updated version.
+
+Another caveat of this approach is that the generator needs to emit the same
+output on all systems. For example, if some tests should be skipped on Linux,
+the generator can not just filter the corresponding tests depending on
+``Sys.os_type``. It has to consistently emit a ``(enabled_if)`` field for the
+rules.
+
+Using ``(dynamic_include)``
+---------------------------
+
+.. versionadded:: 3.14
+
+This technique relies on :doc:`/reference/dune/dynamic_include`, which is
+more flexible than :doc:`/reference/dune/include`. The difference is that
+the intermediate ``dune.inc`` file does not need to be part of the source tree.
+It will only be generated by a rule and be present in the ``_build`` directory.
+
+At first it looks like it would be possible to reuse the same pattern as above:
+change ``include`` to ``dynamic_include`` and delete the ``dune.inc`` file.
+However, it is not possible. The reason is that rules are loaded per directory,
+and there needs to be a strict order (no cycles) between directories for this
+to work.
+
+So, instead we are going to:
+
+- generate ``dune.inc`` in a subdirectory named ``generate``, and
+- include these rules in a subdirectory named ``run``.
+
+These subdirectories do not need to be actual directories. They can be emulated
+through :doc:`/reference/dune/subdir`.
+
+To do this, we can create the following ``dune`` file in the same directory as
+the ``*.input`` and ``*.output`` files.
+
+``dune``
+  .. code:: dune
+
+    (executable
+     (name gen))
+    
+    (subdir run
+     (dynamic_include ../generate/dune.inc))
+    
+    (subdir generate
+     (rule
+      (deps (glob_files ../*.input))
+      (action
+       (with-stdout-to dune.inc
+        (run ../gen.exe)))))
+
+Then create the following ``gen.ml`` file. Note that here we can define it in
+the same directory.
+
+``gen.ml``
+  .. code:: ocaml
+
+    let generate_rules base =
+      Printf.printf
+        {|
+            (rule
+             (with-stdout-to %s.gen
+              (run %%{bin:tool} ../%s.input)))
+        
+            (rule
+             (alias runtest)
+             (action
+              (diff ../%s.output %s.gen)))
+             |}
+        base base base base
+    
+    let () =
+      Sys.readdir ".." |> Array.to_list |> List.sort String.compare
+      |> List.filter_map (Filename.chop_suffix_opt ~suffix:".input")
+      |> List.iter generate_rules
+
+There are a few differences from the generator above because this one
+is going to be invoked from subdirectories, so it is necessary to refer to the
+``..`` directory both in the input (which files to read) and in the output (how
+the rules are executed).
+
+These two files are enough. ``dune runtest`` is going to generate the rules and
+interpret them in a single command.
+
+Notes
+^^^^^
+
+This approach is shorter, but it might be more difficult to debug because changes
+to the generated rules will not be visible. Also, it works in that case, but it
+is not possible to generate all kinds of stanzas with that pattern. See
+:doc:`/reference/dune/dynamic_include` for more information about the
+limitations.

unikernel/duniverse/dune_/doc/howto/toplevel.rst

@@ -0,0 +1,55 @@
+How to Load a Project in a Toplevel
+===================================
+
+It is possible to use OCaml code in an interactive way, by typing an
+expression, which gets evaluated and its result printed. Such a program is
+called a `toplevel`, or REPL (Read-Eval-Print Loop).
+
+The compiler distribution comes with a small REPL called simply ``ocaml``, and
+the community has developed enhanced versions such as `UTop
+<https://github.com/ocaml-community/utop>`_.
+
+Building a Specialized UTop Executable
+--------------------------------------
+
+It is possible to generate a specialized version of UTop that embeds the
+current project. To do so, use the following command:
+
+.. code:: console
+
+  $ dune utop
+
+The interactive session will start with all the modules loaded.
+
+If some of the libraries are PPX rewriters, the phrases you type in the
+toplevel will be rewritten with these PPX rewriters. Similarly, PPX derivers
+defined in the project will be available.
+
+Loading the Project in a Toplevel
+---------------------------------
+
+It is also possible to load Dune projects in any toplevel. To do that, simply
+execute the following in your toplevel:
+
+.. code:: ocaml
+
+    # #use_output "dune ocaml top";;
+
+``dune ocaml top`` is a Dune command that builds all the libraries in the
+current directory and subdirectories and outputs the relevant toplevel
+directives (``#directory`` and ``#load``) to make the various modules
+available in the toplevel.
+
+Loading a Single Module in a Toplevel
+-------------------------------------
+
+It's also possible to load individual modules for interactive development. Use
+the following dune command:
+
+.. code:: ocaml
+
+  # #use_output "dune ocaml top-module foo.ml";;
+
+This will print directives that will load ``foo.ml`` without sealing it behind
+``foo.mli``. This is particularly useful for peeking and prodding at a module's
+internals.