swrup/mte
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Browse source
This commit is contained in:
swrup 2025-11-11 02:07:51 +01:00
parent aa2ff7b2f0
commit 2f3113f55d
11742 changed files with 1223940 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

19
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/conclusion.md Normal file
View file

@ -0,0 +1,19 @@
Conclusion
==========
This tutorial has covered the parts of Dune that users are the most likely to
interact with:
- the {doc}`/reference/dune/executable`, {doc}`/reference/dune/library`, and
{doc}`/reference/dune/test` stanzas;
- how to use {doc}`cram tests</reference/cram>` and the workflow of
{doc}`promotion </concepts/promotion>`;
- bindings to C code using {doc}`foreign stubs </reference/foreign-stubs>`;
- using a ppx deriver.
## Where to go from here
You might be interested in:
- our {doc}`how-to guides </howto/index>` to apply this to your projects;
- various {doc}`explanations </explanation/index>` about how Dune works.

688
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/development-cycle.md Normal file
View file

@ -0,0 +1,688 @@
The Development Cycle
=====================
Our calculator now has a better structure, but it still does not do much.
In this chapter, we are going to make several iterations where we add a cram
test, see it fail, implement the missing feature, and repeat. This is how many
OCaml projects are developed (including Dune itself).
:::{note}
There are many ways to do this: tests can be written first or last; some prefer
to `promote` only the fixed version.
In this tutorial, we're going to write tests first and `promote` the erroneous
version but as you become more familiar with Dune, you'll be able to explore
variations of this loop.
:::
## Display Errors
So far, our calculator is not doing any error handling.
We're going to create a test with an error, see how the calculator behaves, and
add a better error message.
### Create a Test
Add a new test in `test/calc.t` with the following content. Note that we do
not specify any output for the command. As before, make sure to include two
spaces before the `$` sign.
```{code-block} cram
:emphasize-lines: 4
$ calc -e '1+2'
3
$ calc -e '1+'
```
Next, run the tests using `dune runtest`.
Dune will display a diff with the actual output:
```diff
$ calc -e '1+'
+ calc: internal error, uncaught exception:
+ Calc.Parser.MenhirBasics.Error
+
+ [125]
```
Run `dune promote` and observe that the error message is part of the test.
At that point, running `dune runtest` will succeed.
Our goal for the rest of this section is to make this test display a nice error message.
### Handle the Exception
The message points to an uncaught exception.
Indeed, `Parser.main` can raise `Parser.Error`.
Let's catch this exception in `eval_lb` and display the location of the error
in the input file.
Edit `lib/cli.ml`:
```{code-block} ocaml
:emphasize-lines: 2,5-6
let eval_lb lb =
try
let e = Parser.main Lexer.token lb in
Printf.printf "%d\n" (eval e)
with Parser.Error ->
Printf.printf "parse error near character %d" lb.lex_curr_pos
```
:::{note}
This is just an excerpt from the file.
Edit the `eval_lb` to make it look like this.
The modified parts are highlighted.
:::
Now, run the tests again with `dune runtest`. It displays the following diff:
```diff
$ calc -e '1+'
- calc: internal error, uncaught exception:
- Calc.Parser.MenhirBasics.Error
-
- [125]
+ parse error near character 2
```
Run `dune promote` and note that `test/calc.t` has changed.
Run `dune runtest`. Nothing is displayed, indicated that the test has passed.
:::{note}
This is similar to what happened at the end of {doc}`the previous chapter
<structure>`.
The first `dune runtest` compares the *expected output*
(from `test/calc.t`, the uncaught exception message) to the *actual output*
(our new error message) and displays the diff. So, the uncaught exception
appears as deleted lines (prefixed with `-`) and the new error message appears
as added lines (prefixed with `+`).
Running `dune promote` will copy the last *actual output* to `calc/test.t`.
Running `dune runtest` a second time will compare the *expected output* (the
new message) with the *actual output* of the command (the new message) and find
no difference.
:::
## Add Floats
At this stage, our calculator only supports integers. In this section, we are
going to add support for floating-point numbers and operations.
### Add a Test
First, add a test at the end of `test/calc.t`:
```console
$ calc -e '1+2.5'
```
Run `dune runtest`: this displays an error.
```diff
$ calc -e '1+2.5'
+ calc: internal error, uncaught exception:
+ Failure("lexing: empty token")
+
+ [125]
```
Run `dune promote` to update the failing test.
Our goal for the rest of this section is to change that test to print `3.5`.
### Add a `Float` constructor
We need to add a new kind of expression. Let's extend the `exp` type in
`lib/ast.ml` to add a new `Float` constructor.
```{code-block} ocaml
:emphasize-lines: 4
type exp =
| Int of int
| Add of exp * exp
| Float of float
```
With this new constructor, we can represent the `2.5` part as `Float 2.5`.
### Lexing and Parsing
We also need to extend our lexer to produce a new token type for floats, and a
production rule in the grammar.
Let's first add a token type in `lib/parser.mly`:
```{code-block} ocaml
:emphasize-lines: 3
%token<int> Int
%token Plus
%token<float> Float
```
A new rule in `lib/lexer.mll`:
```{code-block} ocaml
:emphasize-lines: 7
rule token = parse
| eof { Parser.Eof }
| space { token lexbuf }
| '\n' { Parser.Eof }
| '+' { Parser.Plus }
| digit+ { Parser.Int (int_of_string (Lexing.lexeme lexbuf)) }
| digit+ '.' digit+ { Parser.Float (float_of_string (Lexing.lexeme lexbuf)) }
```
And a new rule in `lib/parser.mly`:
```{code-block} ocaml
:emphasize-lines: 4
expr:
| Int { Int $1 }
| expr Plus expr { Add ($1, $3) }
| Float { Float $1 }
```
### Evaluation
Let's run `dune build`.
With the new constructor, the compiler is now complaining that the pattern
matching in our `eval` function is incomplete.
```
File "lib/cli.ml", line 1, characters 15-70:
1 | let rec eval = function Ast.Int n -> n | Add (a, b) -> eval a + eval b
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error (warning 8 [partial-match]): this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
Float _
```
To fix this, we need to tweak `lib/cli.ml` a bit. Instead of returning an
`int`, let's introduce a `value` type that can represent either `int` or
`float`:
```ocaml
type value = VInt of int | VFloat of float
let value_to_string = function
| VInt n -> string_of_int n
| VFloat f -> Printf.sprintf "%.6g" f
```
And update the `eval_lb` function to use this printer:
```{code-block} ocaml
:emphasize-lines: 3-5
let eval_lb lb =
try
let expr = Parser.main Lexer.token lb in
let v = eval expr in
Printf.printf "%s\n" (value_to_string v)
with Parser.Error ->
Printf.printf "parse error near character %d" lb.lex_curr_pos
```
Finally, for `eval` itself, we'll use `(+)` or `(+.)` if both values have the
same type, or convert integers to floats if needed:
```ocaml
let rec eval = function
| Ast.Int n -> VInt n
| Float f -> VFloat f
| Add (a, b) -> (
match (eval a, eval b) with
| VInt na, VInt nb -> VInt (na + nb)
| VFloat fa, VFloat fb -> VFloat (fa +. fb)
| VInt na, VFloat fb -> VFloat (float na +. fb)
| VFloat fa, VInt nb -> VFloat (fa +. float nb))
```
With this implementation done, call `dune runtest` and notice that it changes
the output. Call `dune promote` to update the test.
## Pi
In this section, we're going to add named constants, and `pi` in particular.
### Create a test
Add a new test in `test/calc.t`:
```console
$ calc -e '1+pi'
```
Run `dune runtest` and see the failure.
Run `dune promote` to add the failure to the test file.
Our goal in the rest of the section is to change the output of this test.
### Add a Constructor
Let's add an new constructor in `lib/ast.ml`:
```{code-block} ocaml
:emphasize-lines: 5
type exp =
| Int of int
| Add of exp * exp
| Float of float
| Ident of string
```
That way, `pi` is going to be represented as `Ident "pi"`.
### Lexing and Parsing
We now need to parse these constants.
Let's add a new token in `lib/parser.mly`:
```{code-block} ocaml
:emphasize-lines: 5
%token Eof
%token<int> Int
%token<float> Float
%token Plus
%token<string> Ident
```
Produce it using a new lexing rule in `lib/lexer.mll`:
```{code-block} ocaml
:emphasize-lines: 3-5,14
let digit = ['0'-'9']
let letter = ['a'-'z']
let ident = letter+
rule token = parse
| eof { Parser.Eof }
| space { token lexbuf }
| '\n' { Parser.Eof }
| '+' { Parser.Plus }
| digit+ { Parser.Int (int_of_string (Lexing.lexeme lexbuf)) }
| digit+ '.' digit+ { Parser.Float (float_of_string (Lexing.lexeme lexbuf)) }
| ident { Parser.Ident (Lexing.lexeme lexbuf) }
```
And handle it as a new derivation in `lib/parser.mly`:
```{code-block} ocaml
:emphasize-lines: 5
expr:
| Int { Int $1 }
| Float { Float $1 }
| expr Plus expr { Add ($1, $3) }
| Ident { Ident $1 }
```
### Evaluation
Finally, we'll have to update our evaluation function to take the new
constructor into account.
OCaml does not have a value for `pi`, but it has a `Stdlib.acos` function.
Since {math}`\cos{\frac{\pi}{2}} = 0`, we can define `pi` as {math}`2 * \arccos
0` .
```{code-block} ocaml
:emphasize-lines: 10-11
let rec eval = function
| Ast.Int n -> VInt n
| Float f -> VFloat f
| Add (a, b) -> (
match (eval a, eval b) with
| VInt na, VInt nb -> VInt (na + nb)
| VFloat fa, VFloat fb -> VFloat (fa +. fb)
| VInt na, VFloat fb -> VFloat (float na +. fb)
| VFloat fa, VInt nb -> VFloat (fa +. float nb))
| Ident "pi" -> VFloat (2. *. Stdlib.acos 0.)
| Ident _ -> failwith "unknown ident"
```
Finally, let's run our test. `dune runtest` will display a diff with the new
value. Run `dune promote` to accept it.
## Multiplication
### Create a Test
Let's add a new test in `test/calc.t`:
```cram
$ calc -e '1+2*3'
```
Run the test with `dune runtest`. Notice the error message.
Let's run `dune promote` to add it to the file.
Now, our goal for the rest of this section is to change that to the expected
result.
### Add a Constructor
Let's update the AST `lib/ast.ml`: we're generalizing addition to binary
operations, and create a new `Mul` operation (notice that we remove the line corresponding to the `Add` expression).
```{code-block} ocaml
:emphasize-lines: 1-3,9
type op =
| Add
| Mul
type exp =
| Int of int
| Float of float
| Ident of string
| Op of op * exp * exp
```
### Lexing and Parsing
Let's add a new token for `*` in `lib/parser.mly`:
```{code-block} ocaml
:emphasize-lines: 3
%token<string> Ident
%token Plus
%token Star
```
Then, we'll produce it using a new rule in `lib/lexer.mll`:
```{code-block} ocaml
:emphasize-lines: 4
| space { token lexbuf }
| '\n' { Parser.Eof }
| '+' { Parser.Plus }
| '*' { Parser.Star }
```
And use that token in a new rule in `lib/parser.mly` (also modifying the `Add`
rule):
```{code-block} ocaml
:emphasize-lines: 4,5
| Int { Int $1 }
| Float { Float $1 }
| Ident { Ident $1 }
| expr Plus expr { Op (Add, $1, $3) }
| expr Star expr { Op (Mul, $1, $3) }
```
We have a last edit to do here, which is to add a precedence annotation:
```{code-block} ocaml
:emphasize-lines: 2
%left Plus
%left Star
```
### Evaluation
Now, we can update our evaluation function in `lib/cli.ml`:
```{code-block} ocaml
:emphasize-lines: 1-6,13-14
let eval_number_op f_int f_float va vb =
match (va, vb) with
| VInt na, VInt nb -> VInt (f_int na nb)
| VFloat fa, VFloat fb -> VFloat (f_float fa fb)
| VInt na, VFloat fb -> VFloat (f_float (float_of_int na) fb)
| VFloat fa, VInt nb -> VFloat (f_float fa (float_of_int nb))
let rec eval = function
| Ast.Int n -> VInt n
| Float f -> VFloat f
| Ident "pi" -> VFloat (2. *. Stdlib.acos 0.)
| Ident _ -> failwith "unknown ident"
| Op (Add, a, b) -> eval_number_op ( + ) ( +. ) (eval a) (eval b)
| Op (Mul, a, b) -> eval_number_op ( * ) ( *. ) (eval a) (eval b)
```
With these updates done, let's run `dune runtest`. It displays that the result
is `7`. Call `dune promote` to update the test.
## Division
This section is going to be very similar to the previous one. Instead we're
adding the division operator.
### Create a Test
Add a test in `test/calc.t`:
```cram
$ calc -e '4/2'
```
Call `dune runtest`, note the error, call `dune promote`.
### Add a Constructor
Add a constructor in `lib/ast.ml`:
```{code-block} ocaml
:emphasize-lines: 4
type op =
| Add
| Mul
| Div
```
### Lexing and Parsing
Update `lib/parser.mly`:
```{code-block} ocaml
:emphasize-lines: 4
%token<string> Ident;
%token Plus
%token Star
%token Slash
```
Then add the right precedence:
```{code-block} ocaml
:emphasize-lines: 2
%left Plus
+%left Star Slash
```
And the corresponding rule:
```{code-block} ocaml
:emphasize-lines: 4
| Ident { Ident $1 }
| expr Plus expr { Op (Add, $1, $3) }
| expr Star expr { Op (Mul, $1, $3) }
| expr Slash expr { Op (Div, $1, $3) }
```
Finally, add a lexing rule in `lib/lexer.mll`:
```{code-block} ocaml
:emphasize-lines: 3
| '+' { Parser.Plus }
| '*' { Parser.Star }
| '/' { Parser.Slash }
```
### Evaluation
Add a new case in `lib/cli.ml`:
```{code-block} ocaml
:emphasize-lines: 8
let rec eval = function
| Ast.Int n -> VInt n
| Float f -> VFloat f
| Ident "pi" -> VFloat (2. *. Stdlib.acos 0.)
| Ident _ -> failwith "unknown ident"
| Op (Add, a, b) -> eval_number_op ( + ) ( +. ) (eval a) (eval b)
| Op (Mul, a, b) -> eval_number_op ( * ) ( *. ) (eval a) (eval b)
| Op (Div, a, b) -> eval_number_op ( / ) ( /. ) (eval a) (eval b)
```
Now, running `dune runtest` should display the right result. Run `dune promote`
to accept it.
## Sine
### Create a Test
Create a new test in `test/calc.t`:
```cram
$ calc -e 'sin (pi / 6)'
```
Call `dune runtest`, note the error, call `dune promote`.
### Add a Constructor
Add a constructor in `lib/ast.ml`:
```{code-block} ocaml
:emphasize-lines: 6
type exp =
| Int of int
| Float of float
| Ident of string
| Op of op * exp * exp
| Call of string * exp
```
### Lexing and Parsing
Update `lib/parser.mly` by defining new tokens for parentheses:
```{code-block} ocaml
:emphasize-lines: 5
%token<string> Ident;
%token Plus
%token Star
%token Slash
%token Lpar Rpar
```
And a production for calls:
```{code-block} ocaml
:emphasize-lines: 5
| Ident { Ident $1 }
| expr Plus expr { Op (Add, $1, $3) }
| expr Star expr { Op (Mul, $1, $3) }
| expr Slash expr { Op (Div, $1, $3) }
| Ident Lpar expr Rpar { Call ($1, $3) }
```
Update `lib/lexer.mll`:
```{code-block} ocaml
:emphasize-lines: 4-5
| '+' { Parser.Plus }
| '*' { Parser.Star }
| '/' { Parser.Slash }
| '(' { Parser.Lpar }
| ')' { Parser.Rpar }
```
### Evaluation
We'll need a `float` to pass to `Stdlib.sin`, so let's introduce a conversion
function. We'll also add the corresponding cases in `lib/cli.ml`:
```{code-block} ocaml
:emphasize-lines: 1-3,13-14
let as_float = function
| VInt n -> float_of_int n
| VFloat f -> f
let rec eval = function
| Ast.Int n -> VInt n
| Float f -> VFloat f
| Ident "pi" -> VFloat (2. *. Stdlib.acos 0.)
| Ident _ -> failwith "unknown ident"
| Op (Add, a, b) -> eval_number_op ( + ) ( +. ) (eval a) (eval b)
| Op (Mul, a, b) -> eval_number_op ( * ) ( *. ) (eval a) (eval b)
| Op (Div, a, b) -> eval_number_op ( / ) ( /. ) (eval a) (eval b)
| Call ("sin", e) -> VFloat (Stdlib.sin (as_float (eval e)))
| Call _ -> failwith "unknown function"
```
Now, run the tests using `dune runtest`.
Accept the correction with `dune promote`.
## Conclusion
We've added several features to our calculator, and added tests in the meantime.
To do so, we've used `dune runtest` and `dune promote`, two of the most useful
Dune commands.
::::{dropdown} Checkpoint
:icon: location
This is how the project looks like at the end of this chapter.
:::{literalinclude} introduction/dune-project
:caption: dune-project (unchanged)
:language: dune
:::
:::{literalinclude} structure/bin/dune
:caption: bin/dune (unchanged)
:language: dune
:::
:::{literalinclude} structure/bin/calc.ml
:caption: bin/calc.ml (unchanged)
:language: ocaml
:::
:::{literalinclude} structure/lib/dune
:caption: lib/dune (unchanged)
:language: dune
:::
:::{literalinclude} development-cycle/lib/ast.ml
:caption: lib/ast.ml
:language: ocaml
:::
:::{literalinclude} development-cycle/lib/cli.ml
:caption: lib/cli.ml
:language: ocaml
:::
:::{literalinclude} development-cycle/lib/lexer.mll
:caption: lib/lexer.mll
:language: ocaml
:::
:::{literalinclude} development-cycle/lib/parser.mly
:caption: lib/parser.mly
:language: ocaml
:::
:::{literalinclude} structure/test/dune
:caption: test/dune (unchanged)
:language: dune
:::
:::{literalinclude} development-cycle/test/calc.t
:caption: test/calc.t
:language: cram
:::
::::

11
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/development-cycle/lib/ast.ml Normal file
View file

@ -0,0 +1,11 @@
type op =
| Add
| Mul
| Div
type exp =
| Int of int
| Float of float
| Ident of string
| Op of op * exp * exp
| Call of string * exp

57
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/development-cycle/lib/cli.ml Normal file
View file

@ -0,0 +1,57 @@
type value = VInt of int | VFloat of float
let value_to_string = function
| VInt n -> string_of_int n
| VFloat f -> Printf.sprintf "%.6g" f
let eval_number_op f_int f_float va vb =
match (va, vb) with
| VInt na, VInt nb -> VInt (f_int na nb)
| VFloat fa, VFloat fb -> VFloat (f_float fa fb)
| VInt na, VFloat fb -> VFloat (f_float (float_of_int na) fb)
| VFloat fa, VInt nb -> VFloat (f_float fa (float_of_int nb))
let as_float = function
| VInt n -> float_of_int n
| VFloat f -> f
let rec eval = function
| Ast.Int n -> VInt n
| Float f -> VFloat f
| Ident "pi" -> VFloat (2. *. Stdlib.acos 0.)
| Ident _ -> failwith "unknown ident"
| Op (Add, a, b) -> eval_number_op ( + ) ( +. ) (eval a) (eval b)
| Op (Mul, a, b) -> eval_number_op ( * ) ( *. ) (eval a) (eval b)
| Op (Div, a, b) -> eval_number_op ( / ) ( /. ) (eval a) (eval b)
| Call ("sin", e) -> VFloat (Stdlib.sin (as_float (eval e)))
| Call _ -> failwith "unknown function"
let info = Cmdliner.Cmd.info "calc"
let eval_lb lb =
try
let expr = Parser.main Lexer.token lb in
let v = eval expr in
Printf.printf "%s\n" (value_to_string v)
with Parser.Error ->
Printf.printf "parse error near character %d" lb.lex_curr_pos
let repl () =
while true do
Printf.printf ">> %!";
let lb = Lexing.from_channel Stdlib.stdin in
eval_lb lb
done
let term =
let open Cmdliner.Term.Syntax in
let+ expr_opt =
let open Cmdliner.Arg in
value & opt (some string) None & info [ "e" ]
in
match expr_opt with
| Some s -> eval_lb (Lexing.from_string s)
| None -> repl ()
let cmd = Cmdliner.Cmd.v info term
let main () = Cmdliner.Cmd.eval cmd |> Stdlib.exit

20
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/development-cycle/lib/lexer.mll Normal file
View file

@ -0,0 +1,20 @@
let space = [' ']+
let digit = ['0'-'9']
let letter = ['a'-'z']
let ident = letter+
rule token = parse
| eof { Parser.Eof }
| space { token lexbuf }
| '\n' { Parser.Eof }
| '+' { Parser.Plus }
| '*' { Parser.Star }
| '/' { Parser.Slash }
| '(' { Parser.Lpar }
| ')' { Parser.Rpar }
| digit+ { Parser.Int (int_of_string (Lexing.lexeme lexbuf)) }
| digit+ '.' digit+ { Parser.Float (float_of_string (Lexing.lexeme lexbuf)) }
| ident { Parser.Ident (Lexing.lexeme lexbuf) }

29
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/development-cycle/lib/parser.mly Normal file
View file

@ -0,0 +1,29 @@
%token Eof
%token<int> Int
%token Plus
%token Star
%token Slash
%token Lpar Rpar
%token<float> Float
%token<string> Ident
%start<Ast.exp> main
%left Plus
%left Star Slash
%{ open Ast %}
%%
main: expr Eof { $1 }
expr:
| Int { Int $1 }
| expr Plus expr { Op (Add, $1, $3) }
| expr Star expr { Op (Mul, $1, $3) }
| expr Slash expr { Op (Div, $1, $3) }
| Ident Lpar expr Rpar { Call ($1, $3) }
| Float { Float $1 }
| Ident { Ident $1 }
%%

20
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/development-cycle/test/calc.t Normal file
View file

@ -0,0 +1,20 @@
$ calc -e '1+2'
3
$ calc -e '1+'
parse error near character 2
$ calc -e '1+2.5'
3.5
$ calc -e '1+pi'
4.14159
$ calc -e '1+2*3'
7
$ calc -e '4/2'
2
$ calc -e 'sin (pi / 6)'
0.5

36
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/index.md Normal file
View file

@ -0,0 +1,36 @@
---
author: Etienne Millon
---
Developing with Dune
====================
:::{warning}
This tutorial is a work in progress.
:::
In this tutorial, will start with a small Dune project, and extend it by using
common features:
- the {doc}`/reference/dune/executable`, {doc}`/reference/dune/library`, and
{doc}`/reference/dune/test` {term}`stanzas <stanza>`;
- {doc}`cram </reference/cram>` tests;
- bindings to C code using {doc}`foreign stubs </reference/foreign-stubs>`;
- using a [ppx deriver](https://ocaml.org/docs/metaprogramming).
By doing so, you'll interact with `dune runtest` and `dune promote`, and will
use the most common {term}`stanzas <stanza>` in Dune files.
Start the tutorial with the {doc}`introduction`.
:::{toctree}
:hidden:
:maxdepth: 1
introduction
structure
development-cycle
interfacing-with-c
using-ppx
unit-tests
conclusion
:::

158
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/interfacing-with-c.md Normal file
View file

@ -0,0 +1,158 @@
Interfacing with C
==================
In this chapter, we're going to extend our calculator with a new function.
The difference with `sin` is that we're going to use a C implementation of the
function because it is not available in `Stdlib`. To do so, we're going to use
{doc}`(foreign_stubs) </reference/foreign-stubs>` to implement the function in
C.
## Create a test
Add a new test in `test/calc.t`:
```console
$ calc -e 'log10(123456)'
```
Run `dune runtest` and see the failure.
Run `dune promote` to add the failure to the test file.
Our goal in the rest of the chapter is to change the output of this test.
## Lexing
We have a tiny change to make in `lib/lexer.mll`: extend function names so that
they can contain numbers (but not at the beginning).
```{code-block} ocaml
let ident = letter (letter | digit)+
```
## Evaluation
Let's extend our `eval` function in `lib/cli.ml` to handle a `log10` function.
Instead of implementing the function in OCaml, we declare it as an `external`
(with its type).
```{code-block} ocaml
:emphasize-lines: 1,12
external log10_c : float -> float = "calc_log10"
let rec eval = function
| Ast.Int n -> VInt n
| Float f -> VFloat f
| Ident "pi" -> VFloat (2. *. Stdlib.acos 0.)
| Ident _ -> failwith "unknown ident"
| Op (Add, a, b) -> eval_number_op ( + ) ( +. ) (eval a) (eval b)
| Op (Mul, a, b) -> eval_number_op ( * ) ( *. ) (eval a) (eval b)
| Op (Div, a, b) -> eval_number_op ( / ) ( /. ) (eval a) (eval b)
| Call ("sin", e) -> VFloat (Stdlib.sin (as_float (eval e)))
| Call ("log10", e) -> VFloat (log10_c (as_float (eval e)))
| Call _ -> failwith "unknown function"
```
## Create Foreign Stubs
The final part is to implement `calc_log10` as a C function and link it with
our library.
Let's create a file `lib/calc_stubs.c`:
:::{literalinclude} interfacing-with-c/lib/calc_stubs.c
:language: c
:::
:::{note}
The interface between C and OCaml code uses a C type called `value`.
Values of this type can be converted from and to `double` using `Double_val`
and `caml_copy_double`.
They need to be registered with the garbage collector using the `CAMLparam1`
and `CAMLreturn` macros.
:::
And we finally we specify to Dune that this file is part of the library in
`lib/dune`:
```{code-block} dune
:emphasize-lines: 4-6
(library
(name calc)
(libraries cmdliner)
(foreign_stubs
(language c)
(names calc_stubs)))
```
Run the tests again with `dune runtest`.
At that point, the output should be correct.
Call `dune promote` to update the expected output.
## Conclusion
In this chapter, we've extended our library with some C code. The mechanism to
do so is called {doc}`foreign stubs </reference/foreign-stubs>`.
::::{dropdown} Checkpoint
:icon: location
This is how the project looks like at the end of this chapter.
:::{literalinclude} introduction/dune-project
:caption: dune-project (unchanged)
:language: dune
:::
:::{literalinclude} structure/bin/dune
:caption: bin/dune (unchanged)
:language: dune
:::
:::{literalinclude} structure/bin/calc.ml
:caption: bin/calc.ml (unchanged)
:language: ocaml
:::
:::{literalinclude} interfacing-with-c/lib/dune
:caption: lib/dune
:language: dune
:::
:::{literalinclude} development-cycle/lib/ast.ml
:caption: lib/ast.ml (unchanged)
:language: ocaml
:::
:::{literalinclude} interfacing-with-c/lib/calc_stubs.c
:caption: lib/calc_stubs.c
:language: c
:::
:::{literalinclude} interfacing-with-c/lib/cli.ml
:caption: lib/cli.ml
:language: ocaml
:::
:::{literalinclude} interfacing-with-c/lib/lexer.mll
:caption: lib/lexer.mll
:language: ocaml
:::
:::{literalinclude} development-cycle/lib/parser.mly
:caption: lib/parser.mly (unchanged)
:language: ocaml
:::
:::{literalinclude} structure/test/dune
:caption: test/dune (unchanged)
:language: dune
:::
:::{literalinclude} interfacing-with-c/test/calc.t
:caption: test/calc.t
:language: cram
:::
::::

11
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/interfacing-with-c/lib/calc_stubs.c Normal file
View file

@ -0,0 +1,11 @@
#include <caml/alloc.h>
#include <caml/memory.h>
#include <math.h>
value calc_log10 (value vx)
{
CAMLparam1(vx);
double x = Double_val(vx);
double r = log10(x);
CAMLreturn(caml_copy_double(r));
}

60
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/interfacing-with-c/lib/cli.ml Normal file
View file

@ -0,0 +1,60 @@
type value = VInt of int | VFloat of float
let value_to_string = function
| VInt n -> string_of_int n
| VFloat f -> Printf.sprintf "%.6g" f
let eval_number_op f_int f_float va vb =
match (va, vb) with
| VInt na, VInt nb -> VInt (f_int na nb)
| VFloat fa, VFloat fb -> VFloat (f_float fa fb)
| VInt na, VFloat fb -> VFloat (f_float (float_of_int na) fb)
| VFloat fa, VInt nb -> VFloat (f_float fa (float_of_int nb))
let as_float = function
| VInt n -> float_of_int n
| VFloat f -> f
external log10_c : float -> float = "calc_log10"
let rec eval = function
| Ast.Int n -> VInt n
| Float f -> VFloat f
| Ident "pi" -> VFloat (2. *. Stdlib.acos 0.)
| Ident _ -> failwith "unknown ident"
| Op (Add, a, b) -> eval_number_op ( + ) ( +. ) (eval a) (eval b)
| Op (Mul, a, b) -> eval_number_op ( * ) ( *. ) (eval a) (eval b)
| Op (Div, a, b) -> eval_number_op ( / ) ( /. ) (eval a) (eval b)
| Call ("sin", e) -> VFloat (Stdlib.sin (as_float (eval e)))
| Call ("log10", e) -> VFloat (log10_c (as_float (eval e)))
| Call _ -> failwith "unknown function"
let info = Cmdliner.Cmd.info "calc"
let eval_lb lb =
try
let expr = Parser.main Lexer.token lb in
let v = eval expr in
Printf.printf "%s\n" (value_to_string v)
with Parser.Error ->
Printf.printf "parse error near character %d" lb.lex_curr_pos
let repl () =
while true do
Printf.printf ">> %!";
let lb = Lexing.from_channel Stdlib.stdin in
eval_lb lb
done
let term =
let open Cmdliner.Term.Syntax in
let+ expr_opt =
let open Cmdliner.Arg in
value & opt (some string) None & info [ "e" ]
in
match expr_opt with
| Some s -> eval_lb (Lexing.from_string s)
| None -> repl ()
let cmd = Cmdliner.Cmd.v info term
let main () = Cmdliner.Cmd.eval cmd |> Stdlib.exit

11
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/interfacing-with-c/lib/dune Normal file
View file

@ -0,0 +1,11 @@
(library
(name calc)
(libraries cmdliner)
(foreign_stubs
(language c)
(names calc_stubs)))
(ocamllex lexer)
(menhir
(modules parser))

20
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/interfacing-with-c/lib/lexer.mll Normal file
View file

@ -0,0 +1,20 @@
let space = [' ']+
let digit = ['0'-'9']+
let letter = ['a'-'z']
let ident = letter (letter | digit)+
rule token = parse
| eof { Parser.Eof }
| space { token lexbuf }
| '\n' { Parser.Eof }
| '+' { Parser.Plus }
| '*' { Parser.Star }
| '/' { Parser.Slash }
| '(' { Parser.Lpar }
| ')' { Parser.Rpar }
| digit+ { Parser.Int (int_of_string (Lexing.lexeme lexbuf)) }
| digit+ '.' digit+ { Parser.Float (float_of_string (Lexing.lexeme lexbuf)) }
| ident { Parser.Ident (Lexing.lexeme lexbuf) }

23
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/interfacing-with-c/test/calc.t Normal file
View file

@ -0,0 +1,23 @@
$ calc -e '1+2'
3
$ calc -e '1+'
parse error near character 2
$ calc -e '1+2.5'
3.5
$ calc -e '1+pi'
4.14159
$ calc -e '1+2*3'
7
$ calc -e '4/2'
2
$ calc -e 'sin (pi / 6)'
0.5
$ calc -e 'log10(123456)'
5.09151

185
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/introduction.md Normal file
View file

@ -0,0 +1,185 @@
Introduction
============
The goal of this first chapter is to get to a place where you have a working
Dune project with a skeleton of a calculator.
This is is a [tutorial](https://diataxis.fr/tutorials/): it is meant to be
followed in order, but you can stop at any point. You can also restart from any
chapter, using these sections that are present at the end of the previous
chapter.
::::{dropdown} Checkpoint
:icon: location
:open:
This will contain the project at the end of each chapter.
::::
## Installing Packages
First, you'll need to have a working Opam installation. This is described in
{doc}`/howto/install-dune`.
Then, create an empty directory somewhere, say `~/dune-calc`. In this tutorial,
we will only create files in this directory.
Let's first make sure you have a working opam installation. Run this command:
```sh
opam --version
```
It should display something like "2.2.0". Anything greater than 2.0.0 is fine.
:::{important}
When asked to type a command, you can click {octicon}`copy` to copy the full
command to your clipboard.
In this tutorial, all commands will be typed at the root of your project, like
`~/dune-calc`.
:::
Let's create a local switch: `cd` to this directory and run the following command.
This can take a few minutes.
```sh
opam switch create ./ 5.3.0
```
This command has created a directory named `_opam` in the current directory.
Now, let's install some packages by running:
```sh
opam install dune.3.15.3 menhir.20231231
```
You can confirm that `opam` is correctly setup by typing `dune --version`,
which should display 3.15.3. Otherwise, please refer to
{doc}`/howto/install-dune`.
:::{note}
The instructions use precise version numbers in `opam install` command. This is
to ensure that the error messages will exactly map what you're seeing, but
it is very likely to work with any version.
:::
## The Calculator Skeleton
Now that we have an opam switch and some packages installed, let's create the
various files.
For each file, click {octicon}`chevron-down` to reveal the file contents and
click {octicon}`copy` to copy the contents to your clipboard. Open `file.txt`
in a text editor and paste the contents there.
::::{dropdown} `dune-project`
:icon: file-code
:::{literalinclude} introduction/dune-project
:language: dune
:::
This file contains metadata about the project:
- the version of the dune language we're using
- the extensions we're {doc}`using </reference/dune-project/using>`
- the {doc}`package </reference/dune-project/package>` we're defining
:::{seealso}
{doc}`/reference/dune-project/index`
Reference documentation about `dune-project` files
:::
::::
::::{dropdown} `dune`
:icon: file-code
:::{literalinclude} introduction/dune
:language: dune
:::
This file contains a description of what's in our project:
- an {doc}`/reference/dune/executable` stanza defining our calculator binary
- an {doc}`/reference/dune/ocamllex` stanza, setting up rules to compile
`lexer.mll` to a `Lexer` module
- a {doc}`/reference/dune/menhir` stanza, to similarly use `parser.mly` as a `Parser` module
::::
::::{dropdown} `parser.mly`
:icon: file-code
:::{literalinclude} introduction/parser.mly
:language: ocaml
:::
This contains definitions of our tokens and grammar rules, using
[Menhir](https://gallium.inria.fr/~fpottier/menhir/).
::::
::::{dropdown} `lexer.mll`
:icon: file-code
:::{literalinclude} introduction/lexer.mll
:language: ocaml
:::
This is our lexer, using [ocamllex](https://ocaml.org/manual/5.2/lexyacc.html).
::::
::::{dropdown} `ast.ml`
:icon: file-code
:::{literalinclude} introduction/ast.ml
:language: ocaml
:::
This contains a definition of the arithmetic expressions manipulated by the calculator.
:::{note}
This is in a separate file from `calc.ml` to avoid module cycles, since `Calc`
depends on `Parser`, which depends on the expression type.
:::
::::
:::{dropdown} `calc.ml`
:icon: file-code
:::{literalinclude} introduction/calc.ml
:language: ocaml
:::
This is the "business logic" of our app, in which we:
- display a prompt
- call the lexer and parser to get an expression
- evaluate the expression
- display the result
:::
At this stage, we have the skeleton of a calculator.
Run the following command to build and execute the calculator:
```sh
dune exec calc
```
You can enter additions, such as `1+2` followed by {kbd}`Enter`.
Exit with {kbd}`Ctrl+C`.
Initially, only addition is supported and anything else triggers an
exception terminating the execution.
Note that a `_build` directory is now present. This is where Dune will store
all compiled artifacts.
You can safely remove this directory - that's actually what the `dune clean`
command does. But that's not usually necessary since Dune will keep track of
dependencies and what is up to date.
The `_opam` directory is where your dependencies are located. It is managed by
opam. If it gets removed by accident or something is corrupted in there, it is
safe to remove it and recreate it by running `opam switch` and `opam install`
as described above.

3
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/introduction/ast.ml Normal file
View file

@ -0,0 +1,3 @@
type exp =
| Int of int
| Add of exp * exp

9
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/introduction/calc.ml Normal file
View file

@ -0,0 +1,9 @@
let rec eval = function Ast.Int n -> n | Add (a, b) -> eval a + eval b
let () =
while true do
Printf.printf ">> %!";
let lb = Lexing.from_channel Stdlib.stdin in
let e = Parser.main Lexer.token lb in
Printf.printf "%d\n" (eval e)
done

7
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/introduction/dune Normal file
View file

@ -0,0 +1,7 @@
(executable
(public_name calc))
(ocamllex lexer)
(menhir
(modules parser))

3
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/introduction/dune-project Normal file
View file

@ -0,0 +1,3 @@
(lang dune 3.18)
(using menhir 3.0)
(package (name calc))

10
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/introduction/lexer.mll Normal file
View file

@ -0,0 +1,10 @@
let space = [' ']+
let digit = ['0'-'9']
rule token = parse
| eof { Parser.Eof }
| space { token lexbuf }
| '\n' { Parser.Eof }
| '+' { Parser.Plus }
| digit+ { Parser.Int (int_of_string (Lexing.lexeme lexbuf)) }

18
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/introduction/parser.mly Normal file
View file

@ -0,0 +1,18 @@
%token Eof
%token<int> Int
%token Plus
%start<Ast.exp> main
%left Plus
%{ open Ast %}
%%
main: expr Eof { $1 }
expr:
| Int { Int $1 }
| expr Plus expr { Add ($1, $3) }
%%

201
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/structure.md Normal file
View file

@ -0,0 +1,201 @@
Improving Structure
===================
Our calculator is fairly monolithic at this stage.
Instead of a single executable, we're going to extract a library and create
some tests.
For now, this is just a cram test that will call the executable, but this
structure will later allow adding unit tests for the library.
## Extract a Library
Create folders `bin`, `lib` and `test`, for binary, library, and tests,
respectively.
Run the following command to install [cmdliner](https://ocaml.org/p/cmdliner/1.3.0/doc/index.html):
```sh
opam install cmdliner.1.3.0
```
Let's create a library. Create `lib/dune` with the `(ocamllex)` and `(menhir)` stanzas from the original `dune` file and a new `(library)` stanza:
:::{literalinclude} structure/lib/dune
:language: dune
:emphasize-lines: 1-3
:::
:::{note}
This is the whole contents of the file. The `(library)` part is highlighted to
show that it's the part that we've just added.
:::
:::{note}
We're defining a {doc}`library </reference/dune/library>` that depends on the
`cmdliner` library.
Libraries can either be defined in your project, or provided by an opam
package. In the case of `cmdliner`, this is the latter, since we've installed
it just before.
{doc}`The OCaml Ecosystem </explanation/ocaml-ecosystem>` covers the difference
between packages, libraries, and modules.
:::
Move `ast.ml`, `lexer.mll`, and `parser.mly` to the `lib` directory.
Now we're going to move `calc.ml` to `lib/cli.ml` and replace it by the following:
:::{literalinclude} structure/lib/cli.ml
:language: ocaml
:::
:::{note}
Two things are happening here.
We are adding a second code path to evaluate a `string` directly, so we extract
an `eval_lb` function that operates on a `lexbuf` (the "source" a lexer can
read from).
We are also moving to `cmdliner` for command-line parsing. This consists in:
- an `info` value (of type `Cmdliner.Cmd.info`) which contains metadata for the program (used in help, etc)
- a `term` value (of type `unit Cmdliner.Term.t`) which sets up arguments and calls `eval_lb` with the right `lexbuf`
- a `cmd` value (of type `unit Cmdliner.Cmd.t`) grouping `info` and `term` together
- a `main` function of type `unit -> 'a` to run `cmd`
:::
## Extract an Executable
Let's create an executable in `bin`. To do so, create a `bin/dune` file with the following contents:
:::{literalinclude} structure/bin/dune
:language: dune
:::
And `bin/calc.ml` with a single function call:
:::{literalinclude} structure/bin/calc.ml
:language: ocaml
:::
Delete the `dune` at the root.
## Create a Test
Create `test/calc.t` with the following contents.
:::{important}
In {doc}`cram tests </reference/cram>`, commands start with two spaces, a
dollar sign, and a space.
Make sure to include **two spaces** at the beginning of the line.
:::
:::{literalinclude} structure/test/calc.t
:language: cram
:lines: 1
:::
Now create `test/dune` to inform Dune that cram tests will use our `calc`
executable and need to be executed again when it changes:
:::{literalinclude} structure/test/dune
:language: dune
:::
At this stage, we're ready to run our test.
Let's do this with `dune runtest`.
It's displaying a diff:
```diff
$ calc -e '1+2'
+ 3
```
Now, run `dune promote`. The contents of `test/calc.t` have changed. Most
editors will pick this up automatically, but it might be necessary to reload
the file to see the change.
Finally, run `dune runtest`. Nothing happens.
Now, run the calculator by running `dune exec calc` to confirm that the
interactive mode still works.
:::{note}
What happened here? This Dune feature, where some tests can edit the source
file, is called {doc}`promotion </concepts/promotion>`.
{doc}`Cram tests </reference/cram>` contain both commands and their expected input.
We did not include any output in the initial cram test. When running `dune
runtest` for the first time, Dune executes the commands, and calls `diff`
between the *expected output* (in `test/calc.t`: no output at all) and the *actual
output* (from running the command: the line "3"), which will display added
lines with a `+` sign and deleted lines with a `-` sign.
Running `dune promote` replaces the input file (`test/calc.t`) with the last
*actual output*. So this includes the line with "3".
Running `dune runtest` again will execute the test again and compare the
*expected output* (`test/calc.t` with the "3" line in it) with the *actual
output* and finds no difference. This means that the test passes.
:::
::::{dropdown} Checkpoint
:icon: location
This is how the project looks like at the end of this chapter.
:::{literalinclude} introduction/dune-project
:caption: dune-project (unchanged)
:language: dune
:::
:::{literalinclude} structure/bin/dune
:caption: bin/dune
:language: dune
:::
:::{literalinclude} structure/bin/calc.ml
:caption: bin/calc.ml
:language: ocaml
:::
:::{literalinclude} structure/lib/dune
:caption: lib/dune
:language: dune
:::
:::{literalinclude} introduction/ast.ml
:caption: lib/ast.ml (unchanged)
:language: ocaml
:::
:::{literalinclude} structure/lib/cli.ml
:caption: lib/cli.ml
:language: ocaml
:::
:::{literalinclude} introduction/lexer.mll
:caption: lib/lexer.mll (unchanged)
:language: ocaml
:::
:::{literalinclude} introduction/parser.mly
:caption: lib/parser.mly (unchanged)
:language: ocaml
:::
:::{literalinclude} structure/test/dune
:caption: test/dune
:language: dune
:::
:::{literalinclude} structure/test/calc.t
:caption: test/calc.t
:language: cram
:::
::::

1
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/structure/bin/calc.ml Normal file
View file

@ -0,0 +1 @@
let () = Calc.Cli.main ()

3
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/structure/bin/dune Normal file
View file

@ -0,0 +1,3 @@
(executable
(public_name calc)
(libraries calc))

27
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/structure/lib/cli.ml Normal file
View file

@ -0,0 +1,27 @@
let rec eval = function Ast.Int n -> n | Add (a, b) -> eval a + eval b
let info = Cmdliner.Cmd.info "calc"
let eval_lb lb =
let e = Parser.main Lexer.token lb in
Printf.printf "%d\n" (eval e)
let repl () =
while true do
Printf.printf ">> %!";
let lb = Lexing.from_channel Stdlib.stdin in
eval_lb lb
done
let term =
let open Cmdliner.Term.Syntax in
let+ expr_opt =
let open Cmdliner.Arg in
value & opt (some string) None & info [ "e" ]
in
match expr_opt with
| Some s -> eval_lb (Lexing.from_string s)
| None -> repl ()
let cmd = Cmdliner.Cmd.v info term
let main () = Cmdliner.Cmd.eval cmd |> Stdlib.exit

8
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/structure/lib/dune Normal file
View file

@ -0,0 +1,8 @@
(library
(name calc)
(libraries cmdliner))
(ocamllex lexer)
(menhir
(modules parser))

2
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/structure/test/calc.t Normal file
View file

@ -0,0 +1,2 @@
$ calc -e '1+2'
3

2
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/structure/test/dune Normal file
View file

@ -0,0 +1,2 @@
(cram
(deps %{bin:calc}))

70
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/unit-tests.md Normal file
View file

@ -0,0 +1,70 @@
Unit Tests
==========
We're testing our calculator using cram tests, but in some cases, it can be more comfortable to use unit tests to test internal behaviors.
In this chapter, we're going to extract the function that deals with conversion
to floats and put it in a test harness.
## Install Dependencies
Run `opam install alcotest.1.7.0`.
## Refactor
Let's refactor our `eval_number_op` function in `lib/cli.ml` so that it uses the `as_float` function. Move `as_float` above it so that is can be used.
```{code-block} ocaml
let eval_number_op f_int f_float va vb =
match (va, vb) with
| VInt na, VInt nb -> VInt (f_int na nb)
| _ ->
let fa = as_float va in
let fb = as_float vb in
VFloat (f_float fa fb)
```
## Create a Test Suite
We're first going to move our cram tests so that they can be in `test/cram/`,
while our unit tests will be in `test/unit/`.
Move the contents of the folder `test` into a fresh folder `test/cram/`.
Create a folder `test/unit/`.
Create `test/unit/dune` with the following contents:
```dune
(test
(name test_calc)
(libraries alcotest calc))
```
And `test/unit/test_calc.ml`:
```ocaml
open Calc
let test_as_float =
let test ~name expression ~expected =
( Printf.sprintf "as_float(%s)" name,
`Quick,
fun () ->
let got = Cli.as_float expression in
Alcotest.check
(Alcotest.float Stdlib.epsilon_float)
__LOC__ expected got )
in
[
test ~name:"int" (VInt 2) ~expected:2.;
test ~name:"float" (VFloat 3.5) ~expected:3.5;
]
let suite = [ ("as_float", test_as_float) ]
let () = Alcotest.run __FILE__ suite
```
Now run `dune runtest`. In addition to the existing cram tests, this also runs
unit tests.

180
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/using-ppx.md Normal file
View file

@ -0,0 +1,180 @@
Using a PPX Preprocessor
========================
Our calculator is pretty much opaque: we feed it a string, and it displays a
result (on an error message), but we have now way to know what the internal expression looks like.
In this chapter, we're going to use a `ppx` deriver to generate a `pp_expr`
function that can display expressions.
## Prerequisites
Install [ppx_deriving](https://github.com/ocaml-ppx/ppx_deriving) by running:
```sh
opam install ppx_deriving.5.2.1
```
## Create a test
Add a new test in `test/calc.t`:
```console
$ calc --debug-ast -e '2 * sin (pi / 2)'
```
Run `dune runtest` and see the failure.
Run `dune promote` to add the failure to the test file.
Our goal in the rest of the chapter is to change the output of this test.
## Use `ppx_deriving.show`
Add an `[@@deriving show]` attribute on types in `lib/ast.ml`:
```{code-block} ocaml
:emphasize-lines: 5,13
type op =
| Add
| Mul
| Div
[@@deriving show]
type expr =
| Int of int
| Float of float
| Ident of string
| Op of op * expr * expr
| Call of string * expr
[@@deriving show]
```
This will generate `pp_op`, `show_op`, `pp_expr`, and `show_expr` functions.
To do do we need to instruct Dune to use `ppx_deriving.show` as a preprocessor by updating `lib/dune`:
```{code-block} dune
:emphasize-lines: 4-5
(library
(name calc)
(libraries cmdliner)
(preprocess
(pps ppx_deriving.show))
(foreign_stubs
(language c)
(names calc_stubs)))
```
## Add a `--debug-ast` Flag
We have a few edits to make to `lib/cli.ml`.
First, add a new `cmdliner` flag to parse the command-line and pass it to
`repl` and `eval_lb`:
```{code-block} ocaml
:emphasize-lines: 6-8,11-12
let term =
let open Cmdliner.Term.Syntax in
let+ expr_opt =
let open Cmdliner.Arg in
value & opt (some string) None & info [ "e" ]
and+ debug_ast =
let open Cmdliner.Arg in
value & flag & info [ "debug-ast" ]
in
match expr_opt with
| Some s -> eval_lb ~debug_ast (Lexing.from_string s)
| None -> repl ~debug_ast
```
Then, forward it from `repl` to `eval_lb`:
```{code-block} ocaml
:emphasize-lines: 1,5
let repl ~debug_ast =
while true do
Printf.printf ">> %!";
let lb = Lexing.from_channel Stdlib.stdin in
eval_lb ~debug_ast lb
done
```
Finally, update `eval_lb` to use it:
```{code-block} ocaml
:emphasize-lines: 1,4
let eval_lb ~debug_ast lb =
try
let expr = Parser.main Lexer.token lb in
if debug_ast then Format.eprintf "[debug] %a\n" Ast.pp_exp expr;
let v = eval expr in
Printf.printf "%s\n" (value_to_string v)
with Parser.Error ->
Printf.printf "parse error near character %d" lb.lex_curr_pos
```
Run the tests again with `dune runtest`.
At that point, the output should be correct.
Call `dune promote` to update the expected output.
::::{dropdown} Checkpoint
:icon: location
This is how the project looks like at the end of this chapter.
:::{literalinclude} introduction/dune-project
:caption: dune-project (unchanged)
:language: dune
:::
:::{literalinclude} structure/bin/dune
:caption: bin/dune (unchanged)
:language: dune
:::
:::{literalinclude} structure/bin/calc.ml
:caption: bin/calc.ml (unchanged)
:language: ocaml
:::
:::{literalinclude} using-ppx/lib/dune
:caption: lib/dune
:language: dune
:::
:::{literalinclude} using-ppx/lib/ast.ml
:caption: lib/ast.ml
:language: ocaml
:::
:::{literalinclude} interfacing-with-c/lib/calc_stubs.c
:caption: lib/calc_stubs.c (unchanged)
:language: c
:::
:::{literalinclude} using-ppx/lib/cli.ml
:caption: lib/cli.ml
:language: ocaml
:::
:::{literalinclude} interfacing-with-c/lib/lexer.mll
:caption: lib/lexer.mll
:language: ocaml
:::
:::{literalinclude} development-cycle/lib/parser.mly
:caption: lib/parser.mly (unchanged)
:language: ocaml
:::
:::{literalinclude} structure/test/dune
:caption: test/dune (unchanged)
:language: dune
:::
:::{literalinclude} using-ppx/test/calc.t
:caption: test/calc.t
:language: cram
:::
::::

13
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/using-ppx/lib/ast.ml Normal file
View file

@ -0,0 +1,13 @@
type op =
| Add
| Mul
| Div
[@@deriving show]
type exp =
| Int of int
| Float of float
| Ident of string
| Op of op * exp * exp
| Call of string * exp
[@@deriving show]

64
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/using-ppx/lib/cli.ml Normal file
View file

@ -0,0 +1,64 @@
type value = VInt of int | VFloat of float
let value_to_string = function
| VInt n -> string_of_int n
| VFloat f -> Printf.sprintf "%.6g" f
let eval_number_op f_int f_float va vb =
match (va, vb) with
| VInt na, VInt nb -> VInt (f_int na nb)
| VFloat fa, VFloat fb -> VFloat (f_float fa fb)
| VInt na, VFloat fb -> VFloat (f_float (float_of_int na) fb)
| VFloat fa, VInt nb -> VFloat (f_float fa (float_of_int nb))
let as_float = function
| VInt n -> float_of_int n
| VFloat f -> f
external log10_c : float -> float = "calc_log10"
let rec eval = function
| Ast.Int n -> VInt n
| Float f -> VFloat f
| Ident "pi" -> VFloat (2. *. Stdlib.acos 0.)
| Ident _ -> failwith "unknown ident"
| Op (Add, a, b) -> eval_number_op ( + ) ( +. ) (eval a) (eval b)
| Op (Mul, a, b) -> eval_number_op ( * ) ( *. ) (eval a) (eval b)
| Op (Div, a, b) -> eval_number_op ( / ) ( /. ) (eval a) (eval b)
| Call ("sin", e) -> VFloat (Stdlib.sin (as_float (eval e)))
| Call ("log10", e) -> VFloat (log10_c (as_float (eval e)))
| Call _ -> failwith "unknown function"
let info = Cmdliner.Cmd.info "calc"
let eval_lb ~debug_ast lb =
try
let expr = Parser.main Lexer.token lb in
if debug_ast then Format.eprintf "[debug] %a\n" Ast.pp_exp expr;
let v = eval expr in
Printf.printf "%s\n" (value_to_string v)
with Parser.Error ->
Printf.printf "parse error near character %d" lb.lex_curr_pos
let repl ~debug_ast =
while true do
Printf.printf ">> %!";
let lb = Lexing.from_channel Stdlib.stdin in
eval_lb ~debug_ast lb
done
let term =
let open Cmdliner.Term.Syntax in
let+ expr_opt =
let open Cmdliner.Arg in
value & opt (some string) None & info [ "e" ]
and+ debug_ast =
let open Cmdliner.Arg in
value & flag & info [ "debug-ast" ]
in
match expr_opt with
| Some s -> eval_lb ~debug_ast (Lexing.from_string s)
| None -> repl ~debug_ast
let cmd = Cmdliner.Cmd.v info term
let main () = Cmdliner.Cmd.eval cmd |> Stdlib.exit

13
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/using-ppx/lib/dune Normal file
View file

@ -0,0 +1,13 @@
(library
(name calc)
(libraries cmdliner)
(foreign_stubs
(language c)
(names calc_stubs))
(preprocess
(pps ppx_deriving.show)))
(ocamllex lexer)
(menhir
(modules parser))

30
unikernel/duniverse/dune_/doc/tutorials/developing-with-dune/using-ppx/test/calc.t Normal file
View file

@ -0,0 +1,30 @@
$ calc -e '1+2'
3
$ calc -e '1+'
parse error near character 2
$ calc -e '1+2.5'
3.5
$ calc -e '1+pi'
4.14159
$ calc -e '1+2*3'
7
$ calc -e '4/2'
2
$ calc -e 'sin (pi / 6)'
0.5
$ calc -e 'log10(123456)'
5.09151
$ calc --debug-ast -e '2 * sin (pi / 2)'
2
[debug] (Ast.Op (Ast.Mul, (Ast.Int 2),
(Ast.Call ("sin",
(Ast.Op (Ast.Div, (Ast.Ident "pi"), (Ast.Int 2)))))
))

140
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/dependencies.md Normal file
View file

@ -0,0 +1,140 @@
# Managing Dependencies
The OCaml ecosystem has a wealth of third-party packages that are available for
use. In this section we will look into how to use them with Dune.
## Adding Dependencies
Much like in regular projects, to add a library we need to add a dependency to
it. For simplicity we will use the popular `fmt` library as an example, but any
package from the [package repository](https://ocaml.org/packages) can be used.
First we update the `dune-project` file to add a dependency on the opam package.
::::{dropdown} `dune-project`
:icon: file-code
:::{literalinclude} dependencies/dune-project
:language: dune
:emphasize-lines: 8
:::
::::
After this change to our project dependencies, we need to relock dependencies
to update our lock directory with the new packages.
```
$ dune pkg lock
Solution for dune.lock:
- base-unix.base
- fmt.0.9.0
- ocaml.5.2.0
- ocaml-base-compiler.5.2.0
- ocaml-config.3
- ocamlbuild.0.15.0+dune
- ocamlfind.1.9.6+dune
- topkg.1.0.7
```
You can see a lot of new dependencies, among these `fmt`.
:::{note}
The list of packages being output includes all dependencies of your project,
including transitive dependencies.
:::
This will take care of installing the dependencies, but we still need to add it to
our build as a library as usual:
::::{dropdown} `dune`
:icon: file-code
:::{literalinclude} dependencies/dune
:language: dune
:emphasize-lines: 3
:::
Adding a library dependency to our `dune` file via the `libraries` stanza.
::::
This will allow us to use the `Fmt` module in our OCaml code.
::::{dropdown} `test.ml`
:icon: file-code
:::{literalinclude} dependencies/test.ml
:language: ocaml
:emphasize-lines: 4
:::
We update the code to define an `Fmt.t` pretty-printer for the list of strings
and then use it to print the value.
::::
To build it we just call `build` again.
```
$ dune build
```
which will download and install the new dependencies and build our project as
before.
As we see, the code works and uses `fmt` to do the pretty-printing:
```
$ dune exec ./test.exe
Hello, OCaml, Rust!
```
### Dependency Constraints
Packages are often only compatible with some versions of dependencies. To
specify a version range, use the regular Dune dependency syntax
used for opam dependencies in the `dune-project` file.
::::{dropdown} `dune-project`
:icon: file-code
:::{literalinclude} dependencies/constraints
:language: dune
:emphasize-lines: 7-8
:::
::::
This ensures the `fmt` package to install will be compatible with
our request. These constraints will be taken into account the next time the
package is locked:
```
$ dune pkg lock
Solution for dune.lock:
- base-unix.base
- fmt.0.9.0
- ocaml.5.2.0
- ocaml-base-compiler.5.2.0
- ocaml-config.3
- ocamlbuild.0.15.0+dune
- ocamlfind.1.9.6+dune
- topkg.1.0.7
```
The version of `fmt` picked is indeed between `0.6` and `1.0`.
## Removing Dependencies
Given all dependencies are defined in the `dune-project` file, removing a
dependency means to remove the dependency from the `depends` field of your
`dune-project` and relocking the project.
The new lock directory will not depend on the package anymore, and in future
builds, the package will not be accessible as `library` anymore.
:::{note}
The removed dependency might still be part of the lock directory if some other
dependency of your project depends on it.
:::

8
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/dependencies/constraints Normal file
View file

@ -0,0 +1,8 @@
(lang dune 3.17)
(name test)
(package
(name test)
(depends
(ocaml (>= 4.14))
(fmt (and (>= 0.6) (< 1.0)))))

3
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/dependencies/dune Normal file
View file

@ -0,0 +1,3 @@
(executable
(public_name test)
(libraries fmt))

8
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/dependencies/dune-project Normal file
View file

@ -0,0 +1,8 @@
(lang dune 3.17)
(name test)
(package
(name test)
(depends
(ocaml (>= 4.14))
fmt))

5
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/dependencies/test.ml Normal file
View file

@ -0,0 +1,5 @@
let langs = ["OCaml"; "Rust"]
let () =
let pp_langs = Fmt.(list ~sep:(any ", ") string) in
Format.printf "Hello, %a!\n" pp_langs langs

27
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/index.md Normal file
View file

@ -0,0 +1,27 @@
---
author: Marek Kubica
---
OCaml Package Management With Dune
==================================
:::{warning}
Dune Package Management is not final yet and details are still subject to
change.
:::
In this tutorial we will be looking at how to use Dune for managing project
dependencies. This enables users to install the compiler as well as third-party
dependencies using a single tool which takes care of building code and
dependencies.
To get started you only need Dune. Head to {doc}`setup` to begin the setup.
:::{toctree}
:hidden:
:maxdepth: 1
setup
dependencies
pinning
repos
:::

55
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/pinning.md Normal file
View file

@ -0,0 +1,55 @@
# Pinning Projects
When Dune is looking up packages to lock, it uses the (pre)configured OCaml
package repositories. However it is also possible to manually specify the
sources of packages; for example, if the package is not released in a package
repository. This is called "pinning".
## Installing Packages From a Pin
To pin a package, a new `pin` has to be declared in the `dune-project` file.
::::{dropdown} `dune-project`
:icon: file-code
:::{literalinclude} pinning/dune-project
:language: dune
:emphasize-lines: 4-6,12
:::
This will create a pin on the `fmt` package and use the specified Git
repository URL to retrieve the sources. For more information refer to {doc}`the
pin stanza reference </reference/dune-project/pin>`.
Don't forget to remove the version constraints from `fmt` in the list of
dependencies.
::::
The next time the package is locked, Dune will use this repository instead of
the information from the selected package repositories.
```
$ dune pkg lock
Solution for dune.lock:
- base-unix.base
- fmt.dev
- ocaml.5.0.0
- ocaml-base-compiler.5.0.0
- ocaml-config.3
- ocamlbuild.0.15.0+dune
- ocamlfind.1.9.6+dune
- topkg.1.0.7
```
Unlike previously, the version of the `fmt` library that is picked is `dev`, to
signify a development version.
The next time the project is built, the `fmt` package will be built from the
source in the specified Git repository rather than from the source tarball
released in the `opam-repository`.
```
$ dune exec ./test.exe
Hello, OCaml, Rust!
```

12
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/pinning/dune-project Normal file
View file

@ -0,0 +1,12 @@
(lang dune 3.17)
(name test)
(pin
(url "git+https://github.com/dbuenzli/fmt.git")
(package (name fmt)))
(package
(name test)
(depends
(ocaml (>= 4.14))
fmt))

56
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/repos.md Normal file
View file

@ -0,0 +1,56 @@
# Custom Repositories
By default when locking package versions, Dune looks up packages from two
sources:
1. The upstream, community maintained `opam-repository` at
[ocaml/opam-repository](https://github.com/ocaml/opam-repository) for most
packages
2. An overlay repository with patched software to allow usage in Dune at
[ocaml-dune/opam-overlays](https://github.com/ocaml-dune/opam-overlays)
To change the presets, the `dune-workspace` file has to be edited (and created
if it didn't exist):
::::{dropdown} `dune-workspace`
:icon: file-code
:::{literalinclude} repos/dune-workspace
:language: dune
:::
::::
In this case, we want to select a specific revision of the community repository
instead of always using the most recent one as it would do by default. We
define a new repository and configure the lock directory to use this
repository.
For more information about the stanzas refer to the {doc}`repositories stanza
</reference/dune-workspace/repository>` as well as the {doc}`lock_dir stanza
</reference/dune-workspace/lock_dir>`.
When relocking the dependencies, the list of packages that are found as
dependencies changes accordingly:
```
$ dune pkg lock
Solution for dune.lock:
- base-unix.base
- fmt.0.9.0
- ocaml.5.0.0
- ocaml-base-compiler.5.0.0
- ocaml-config.3
- ocamlbuild.0.15.0+dune
- ocamlfind.1.9.6+dune
- topkg.1.0.
```
Compared to before, the OCaml compiler version is older, which shows
that we did indeed pick an older version of the package repository for locking.
:::{note}
This feature can also be used to make sure the locked dependencies are
reproducible, as fixing all the package repository versions will lead to
equivalent locking results.
:::

8
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/repos/dune-workspace Normal file
View file

@ -0,0 +1,8 @@
(lang dune 3.17)
(lock_dir
(repositories overlay specific-upstream))
(repository
(name specific-upstream)
(url "git+https://github.com/ocaml/opam-repository.git#00ac3727bc4ac0eabd3c89e69c1660d6b63a3d48"))

120
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/setup.md Normal file
View file

@ -0,0 +1,120 @@
# Setting Up Package Management With Dune
The idea of package management with Dune has been as unobtrusive as
possible. Thus most projects can easily be built with just the minimum of
changes.
In this tutorial we will create a simple project to use the integrated package
management feature for the very first time.
## Declare Dependencies
The best way to work with the package management is to declare your
dependencies in the `dune-project` file.
::::{dropdown} `dune-project`
:icon: file-code
:::{literalinclude} setup/dune-project
:language: dune
:emphasize-lines: 6-7
:::
We define a project called `test` and declare that to build it we need an OCaml
compiler that is at least version 4.14.
This is exactly the same information that is used to generate opam files using
the `generate_opam_files` stanza as described in
{doc}`/howto/opam-file-generation`.
::::
::::{dropdown} `test.ml`
:icon: file-code
:::{literalinclude} setup/test.ml
:language: ocaml
:::
To show that the build works, this simple program will be built and executed.
::::
::::{dropdown} `dune`
:icon: file-code
:::{literalinclude} setup/dune
:language: dune
:::
To declare our module an executable we need a little bit of configuration, so we
just define the module as an executable.
::::
After our project skeleton is set up, we can proceed to the next step.
## Locking Dependencies
After declaring the dependencies, you will need to tell Dune which package
versions to use for your project. This is done by creating a lock directory.
This is easily done with a new Dune command:
```
$ dune pkg lock
Solution for dune.lock:
- ocaml.5.2.0
- ocaml-base-compiler.5.2.0
- ocaml-config.3
```
This will update all the required opam repositories, use the newest version of
each and try to find a set of packages and versions that satisfy the
constraints that your project dependencies declare.
:::{note}
The versions that get locked might be different from this tutorial, as we only
specified the lower bound of `ocaml`; barring any additional configuration, Dune
will pick the newest possible version for each dependency.
:::
## Build Project
To build the project, you can just use the regular Dune commands.
```sh
dune build
```
This will download, build, and install all your locked dependencies and then use
those to build your project. This means that the first time building it will take
longer than usual, as the dependencies need to be built first. Subsequent
builds where all dependencies have been built before will be just as fast as
before.
We can show that the package has been built successfully and works as expected:
```
$ dune exec ./test.exe
Hello, OCaml, Rust!
```
:::{note}
If you want to only build and fetch the project dependencies, you can use
the `@pkg-install` alias like so
```shell
$ dune build @pkg-install
```
See {doc}`/reference/aliases/pkg-install` for more information.
:::
## Conclusion
In this section we learned how to set up a Dune project that picks a compiler
and installs it without the need for any additional tooling.
In the next section {doc}`dependencies` we will look on how to add third party
dependencies.

2
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/setup/dune Normal file
View file

@ -0,0 +1,2 @@
(executable
(public_name test))

7
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/setup/dune-project Normal file
View file

@ -0,0 +1,7 @@
(lang dune 3.17)
(name test)
(package
(name test)
(depends
(ocaml (>= 4.14))))

5
unikernel/duniverse/dune_/doc/tutorials/dune-package-management/setup/test.ml Normal file
View file

@ -0,0 +1,5 @@
let langs = ["OCaml"; "Rust"]
let () =
let s = String.concat ", " langs in
Format.printf "Hello, %s!\n" s

11
unikernel/duniverse/dune_/doc/tutorials/index.md Normal file
View file

@ -0,0 +1,11 @@
Tutorials
=========
These tutorials are hands-on lessons to learn about Dune.
:::{toctree}
:maxdepth: 1
developing-with-dune/index
dune-package-management/index
:::