This commit is contained in:
swrup 2025-11-11 02:07:51 +01:00
parent aa2ff7b2f0
commit 2f3113f55d
11742 changed files with 1223940 additions and 0 deletions

View file

@ -0,0 +1,231 @@
Test for --action-stdxxx-on-success
====================================
$ export BUILD_PATH_PREFIX_MAP="sh=$(command -v sh):$BUILD_PATH_PREFIX_MAP"
$ echo '(lang dune 3.0)' > dune-project
$ cat > dune <<EOF
> (rule
> (alias default)
> (action (system "echo 'Hello, world!'")))
>
> (rule
> (alias default)
> (action (system "echo 'Something went wrong!' >&2")))
>
> (rule
> (alias both-stdout-and-stderr-output)
> (action (system "echo stdout; echo stderr >&2")))
> EOF
By default, stdout and stderr are always printed:
$ dune build
Hello, world!
Something went wrong!
swallow tests
-------------
$ dune clean
$ dune build --action-stdout-on-success=swallow --action-stderr-on-success=swallow
must-be-empty tests
----------------------
In the two above tests, we ask Dune to enforce that the the stdout
(resp. stderr) of actions is empty via the must-be-empty setting.
Since the first rule has a non-empty stdout and the second has a
non-empty stderr, we observe that in each case the build fails
printing the output of the action that had a non-empty output.
$ dune clean
$ dune build --action-stdout-on-success=must-be-empty
Something went wrong!
File "dune", lines 1-3, characters 0-65:
1 | (rule
2 | (alias default)
3 | (action (system "echo 'Hello, world!'")))
Hello, world!
[1]
$ dune clean
$ dune build --action-stderr-on-success=must-be-empty
Hello, world!
File "dune", lines 5-7, characters 0-77:
5 | (rule
6 | (alias default)
7 | (action (system "echo 'Something went wrong!' >&2")))
Something went wrong!
[1]
Same but with output on both stdout and stderr:
$ dune clean
$ dune build @both-stdout-and-stderr-output \
> --action-stdout-on-success=must-be-empty \
> --action-stderr-on-success=must-be-empty
File "dune", lines 9-11, characters 0-95:
9 | (rule
10 | (alias both-stdout-and-stderr-output)
11 | (action (system "echo stdout; echo stderr >&2")))
stdout
stderr
[1]
Incremental builds
------------------
Dune handles --action-stdxxx-on-success in such a way that if
changing the status of one of the two option changes what is printed
to the terminal, then the action is re-executed.
$ dune clean
$ dune build \
> --action-stdout-on-success=swallow \
> --action-stderr-on-success=swallow
For instance, if we previously swallowed stdout/stderr and stop doing
it, actions that printed something to stdout or stderr are
re-executed:
$ dune build
Hello, world!
Something went wrong!
However, we currently re-execute too much. In particular, we
re-execute actions whose outcome is not affected by the change:
$ cat > dune <<EOF
> (rule
> (alias default)
> (action (system "echo a.stdout; echo a.stderr >&2")))
>
> (rule
> (alias default)
> (action (system "echo b.stderr >&2")))
> EOF
$ dune clean
$ dune build --action-stdout-on-success=swallow
a.stderr
b.stderr
You can observe in the below call that both actions are being
re-executed:
$ dune build
a.stdout
a.stderr
b.stderr
However, re-executing the second action was not necessary given that
its stdout was empty. Dune could have recorded the fact that the
second action had an empty stdout and so was unaffected by the status
of --action-stdout-on-success. Dune could also cache the
stdout/stderr of actions across builds and only re-print them rather
than re-execute actions entirely.
In case of errors
-----------------
In case of errors, we print everything no matter what.
$ cat > dune <<EOF
> (rule
> (alias default)
> (action (system "echo 'Hello, world!'; exit 1")))
> EOF
$ dune build
File "dune", lines 1-3, characters 0-73:
1 | (rule
2 | (alias default)
3 | (action (system "echo 'Hello, world!'; exit 1")))
Hello, world!
[1]
$ dune clean
$ dune build --action-stdout=swallow
File "dune", lines 1-3, characters 0-73:
1 | (rule
2 | (alias default)
3 | (action (system "echo 'Hello, world!'; exit 1")))
Hello, world!
[1]
$ dune clean
$ dune build --action-stdout=must-be-empty
File "dune", lines 1-3, characters 0-73:
1 | (rule
2 | (alias default)
3 | (action (system "echo 'Hello, world!'; exit 1")))
Hello, world!
[1]
With compound actions
---------------------
At the moment, the behavior is a bit odd. We swallow the stdout of the
first command but not the second:
$ cat > dune <<EOF
> (rule
> (alias default)
> (action
> (progn
> (system "echo 1")
> (system "echo 2; exit 1"))))
> EOF
$ dune build --action-stdout-on-success=swallow
File "dune", lines 1-6, characters 0-93:
1 | (rule
2 | (alias default)
3 | (action
4 | (progn
5 | (system "echo 1")
6 | (system "echo 2; exit 1"))))
2
[1]
For must-be-empty, if two programs print something without failing we
stop at the first program. That's not terrible, but it would seem
better if we stop at the end of the whole action.
$ cat > dune <<EOF
> (rule
> (alias default)
> (action
> (progn
> (system "echo 1")
> (system "echo 2"))))
> EOF
$ dune build --action-stdout-on-success=must-be-empty
File "dune", lines 1-6, characters 0-85:
1 | (rule
2 | (alias default)
3 | (action
4 | (progn
5 | (system "echo 1")
6 | (system "echo 2"))))
1
[1]
With builtin actions
--------------------
We currently never swallow the output of builtin actions such as
`echo`, which is odd:
$ cat > dune <<EOF
> (rule
> (alias default)
> (action (echo "Hello, world!\n")))
> EOF
$ dune build --action-stdout-on-success=swallow
Hello, world!

View file

@ -0,0 +1,3 @@
(executable
(name sub_process)
(libraries unix))

View file

@ -0,0 +1,10 @@
let () =
match Sys.argv with
| [| _ |] -> exit (Sys.command (Filename.quote_command Sys.executable_name [ "sub" ]))
| [| _; "sub" |] ->
let oc = open_out (Sys.getenv "BEACON_FILE") in
Printf.fprintf oc "%d" (Unix.getpid ());
close_out oc;
Unix.sleep max_int
| _ -> assert false
;;

View file

@ -0,0 +1,60 @@
$ cat > dune-project << EOF
> (lang dune 3.4)
> EOF
It is an error to pass an empty list:
$ cat > dune << EOF
> (rule
> (alias runtest)
> (action
> (cat result)))
>
> (rule
> (with-stdout-to result
> (cat)))
> EOF
$ dune runtest
File "dune", line 8, characters 2-7:
8 | (cat)))
^^^^^
Error: Not enough arguments for "cat"
[1]
The cat action supports several files.
$ echo "file a" > a
$ echo "file b" > b
$ echo "file c" > c
$ cat > dune << EOF
> (rule
> (alias runtest)
> (action
> (cat result)))
>
> (rule
> (with-stdout-to result
> (cat a b c)))
> EOF
$ dune runtest
file a
file b
file c
This requires 3.4.
$ cat > dune-project << EOF
> (lang dune 3.3)
> EOF
$ dune runtest
File "dune", line 8, characters 2-13:
8 | (cat a b c)))
^^^^^^^^^^^
Error: Passing several arguments to 'cat' is only available since version 3.4
of the dune language. Please update your dune-project file to have (lang dune
3.4).
[1]

View file

@ -0,0 +1,84 @@
The use of (concurrent ) illustrated in the context of diffing multiple files.
Say we want to diff 3 files.
$ cat A
I am file A.
$ cat B
I am file B.
$ cat C
I am file C.
We set up a (progn ) rule to diff all of them against their generated versions.
$ cat > dune << EOF
> (rule
> (action
> (progn
> (with-outputs-to A.diff (echo "I am file A.\n"))
> (with-outputs-to B.diff (echo "I am certainly file B.\n"))
> (with-outputs-to C.diff (echo "I am most certainly file C.\n")))))
>
> (rule
> (action
> (progn
> (with-outputs-to some-target (echo a))
> (diff A A.diff)
> (diff B B.diff)
> (diff C C.diff))))
> EOF
We can now run the rule and see that we fail before diffing C.
$ dune build
File "B", line 1, characters 0-0:
Error: Files _build/default/B and _build/default/B.diff differ.
[1]
We can check which diffs were run by asking Dune to promote the files.
$ dune promotion apply
Promoting _build/default/B.diff to B.
Since we failed early, only B was promoted.
Let's reset B to its original state.
$ rm B
$ cat > B << EOF
> I am file B.
> EOF
If we implement the rule using (concurrent ) instead.
$ cat > dune << EOF
> (rule
> (action
> (progn
> (with-outputs-to A.diff (echo "I am file A.\n"))
> (with-outputs-to B.diff (echo "I am certainly file B.\n"))
> (with-outputs-to C.diff (echo "I am most certainly file C.\n")))))
>
> (rule
> (action
> (concurrent
> (with-outputs-to some-target (echo a))
> (diff A A.diff)
> (diff B B.diff)
> (diff C C.diff))))
> EOF
We see that all the files get diffed.
$ dune build
File "B", line 1, characters 0-0:
Error: Files _build/default/B and _build/default/B.diff differ.
File "C", line 1, characters 0-0:
Error: Files _build/default/C and _build/default/C.diff differ.
[1]
And we have promotions for the two that failed.
$ dune promote
Promoting _build/default/B.diff to B.
Promoting _build/default/C.diff to C.

View file

@ -0,0 +1,58 @@
Specification of the concurrency action:
$ cat > dune-project << EOF
> (lang dune 3.7)
> EOF
$ cat > dune << EOF
> (rule
> (action
> (concurrent )))
> EOF
$ dune build
File "dune", line 3, characters 2-15:
3 | (concurrent )))
^^^^^^^^^^^^^
Error: 'concurrent' is only available since version 3.8 of the dune language.
Please update your dune-project file to have (lang dune 3.8).
[1]
Requires Dune 3.8.
$ cat > dune-project << EOF
> (lang dune 3.8)
> EOF
(concurrent ...) runs actions concurrently. Here we mock up an example where two
subactions rely on eachother to also be running in order to terminate.
We write a shell script that will simultaneously read and write to two named
pipes. (This has to be similtaneious otherwise the read will block the write).
They will block on the read however, which means if called with the same two
pipes but swapped, they will only terminate when both scripts are running at the
same time.
$ cat > run.sh << EOF
> echo foo>\$1 & read line<\$2
> EOF
We create an action that will create named pipes a and b and then run our script
on both of them, but importantly inside the concurrent action. This will
demonstrate that subactions are indeed being run concurrently.
$ cat > dune << EOF
> (rule
> (deps run.sh)
> (alias my-rule)
> (action
> (progn
> (run mkfifo a b)
> (concurrent
> (run sh run.sh a b)
> (run sh run.sh b a)))))
> EOF
When we run the rule, we see that the two actions are indeed run concurrently.
$ dune build -j2 @my-rule --force
Notice the need for a -j2. If Dune was configured with -j1 then the action would
never terminate.

View file

@ -0,0 +1,6 @@
(cram
(deps ../watching/helpers.sh bin/sub_process.exe))
(cram
(applies_to concurrent)
(enabled_if false))

View file

@ -0,0 +1,35 @@
Shows what happens when Dune tries to kill an action that has sub-processes.
$ . ../watching/helpers.sh
$ export PATH=$PWD/bin:$PATH
$ echo '(lang dune 3.0)' > dune-project
$ cat >dune <<"EOF"
> (rule
> (action
> (progn
> (run sub_process.exe)
> (with-stdout-to x (echo "")))))
> EOF
$ export BEACON_FILE=$PWD/test-started
$ start_dune
$ build x >/dev/null 2>&1 &
sub_process.exe spawns a sub-process that creates $BEACON_FILE. We
wait for the beacon to be notified that the sub-process has started:
$ with_timeout dune_cmd wait-for-file-to-appear $BEACON_FILE
$ CHILD_PID=`cat $BEACON_FILE`
Now we stop Dune, which should normally kill all sub-processes:
$ stop_dune
$ if kill -s 0 $CHILD_PID 2> /dev/null; then
> echo "FAILURE: child is still running"
> else
> echo "SUCCESS: child has exited"
> fi
SUCCESS: child has exited

View file

@ -0,0 +1,71 @@
$ cat > dune-project << EOF
> (lang dune 2.0)
> (using action-plugin 0.1)
> EOF
$ cat > dune <<EOF
> (executable
> (name exit)
> (modules exit))
> (rule (with-stdout-to exit.ml (echo "let () = exit (int_of_string Sys.argv.(1))")))
> EOF
$ cat >> dune <<EOF
> (rule
> (alias a)
> (action (with-accepted-exit-codes 0 (run ./exit.exe 1))))
> EOF
$ dune build --display=short --root . @a
ocamlc .exit.eobjs/byte/dune__exe__Exit.{cmi,cmo,cmt}
ocamlopt .exit.eobjs/native/dune__exe__Exit.{cmx,o}
ocamlopt exit.exe
File "dune", lines 5-7, characters 0-75:
5 | (rule
6 | (alias a)
7 | (action (with-accepted-exit-codes 0 (run ./exit.exe 1))))
exit alias a (exit 1)
[1]
$ cat >> dune <<EOF
> (rule
> (alias b)
> (action (with-accepted-exit-codes (not 0) (run ./exit.exe 1))))
> EOF
$ dune build --display=short --root . @b
exit alias b
$ cat >> dune <<EOF
> (rule
> (alias c)
> (action (with-accepted-exit-codes (or 1 2 3) (run ./exit.exe 2))))
> (rule
> (alias d)
> (action (with-accepted-exit-codes (or 4 5 6) (run ./exit.exe 7))))
> EOF
$ dune build --display=short --root . @c
exit alias c
$ dune build --display=short --root . @d
File "dune", lines 14-16, characters 0-84:
14 | (rule
15 | (alias d)
16 | (action (with-accepted-exit-codes (or 4 5 6) (run ./exit.exe 7))))
exit alias d (exit 7)
[1]
$ cat >> dune <<EOF
> (rule
> (alias e)
> (action (with-accepted-exit-codes (not 0) (dynamic-run ./exit.exe 1))))
> EOF
$ dune build --display=short --root . @e
File "dune", line 19, characters 43-69:
19 | (action (with-accepted-exit-codes (not 0) (dynamic-run ./exit.exe 1))))
^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: with-accepted-exit-codes can only be used with "run", "bash" or
"system"
[1]

View file

@ -0,0 +1,116 @@
$ cat > dune-project << EOF
> (lang dune 2.2)
> (using action-plugin 0.1)
> EOF
$ cat > dune <<EOF
> (executable
> (name exit)
> (modules exit))
> (rule (with-stdout-to exit.ml (echo "let () = exit (int_of_string Sys.argv.(1))")))
> EOF
$ cat >> dune <<EOF
> (rule
> (alias f)
> (action (with-accepted-exit-codes
> 1
> (with-stdout-to out.txt
> (run ./exit.exe 1)))))
> EOF
$ dune build --display=short --root . @f
ocamlc .exit.eobjs/byte/dune__exe__Exit.{cmi,cmo,cmt}
ocamlopt .exit.eobjs/native/dune__exe__Exit.{cmx,o}
ocamlopt exit.exe
exit out.txt
$ cat >> dune <<EOF
> (rule
> (alias f2)
> (action (with-accepted-exit-codes
> 1
> (with-stdin-from input
> (chdir .
> (run ./exit.exe 1))))))
> EOF
$ echo "Hello, Dune!" > input
$ dune build --display=short --root . @f2
exit alias f2
$ cat >> dune <<EOF
> (rule
> (alias f3)
> (action (with-accepted-exit-codes
> 0
> (setenv VAR myvar
> (chdir .
> (system "echo \$VAR"))))))
> EOF
$ dune build --display=short --root . @f3
sh alias f3
myvar
$ cat >> dune <<EOF
> (rule
> (alias f4)
> (action (with-accepted-exit-codes
> 0
> (setenv VAR myvar
> (ignore-stdout
> (bash "echo \$VAR"))))))
> EOF
$ dune build --display=short --root . @f4
bash alias f4
$ cat >> dune <<EOF
> (rule
> (alias f5)
> (action (with-accepted-exit-codes
> 0
> (setenv VAR myvar
> (with-stdin-from input
> (chdir .
> (with-stdout-to out2.txt
> (run ./exit.exe 1))))))))
> EOF
$ echo "Hello, Dune!" > input
$ dune build --display=short --root . @f5
File "dune", lines 32-40, characters 0-225:
32 | (rule
33 | (alias f5)
34 | (action (with-accepted-exit-codes
35 | 0
36 | (setenv VAR myvar
37 | (with-stdin-from input
38 | (chdir .
39 | (with-stdout-to out2.txt
40 | (run ./exit.exe 1))))))))
exit out2.txt (exit 1)
[1]
$ cat >> dune <<EOF
> (rule
> (alias g)
> (action
> (with-accepted-exit-codes
> (not 0)
> (setenv VAR myvar
> (chdir .
> (with-stdout-to out.txt
> (dynamic-run ./exit.exe 1)))))))
> EOF
$ dune build --display=short --root . @g
File "dune", lines 46-49, characters 3-98:
46 | (setenv VAR myvar
47 | (chdir .
48 | (with-stdout-to out.txt
49 | (dynamic-run ./exit.exe 1)))))))
Error: Only "run", "bash", "system", "chdir", "setenv", "ignore-<outputs>",
"with-stdin-from", "with-<outputs>-to" and "no-infer" can be nested under
"with-accepted-exit-codes"
[1]

View file

@ -0,0 +1,13 @@
$ cat > dune-project <<EOF
> (lang dune 2.0)
> EOF
$ cat > dune << EOF
> (rule (with-stdin-from input (with-stdout-to output (run cat))))
> EOF
$ echo "Hello, Dune!" > input
$ dune build --root . @all
$ cat _build/default/output
Hello, Dune!

View file

@ -0,0 +1,29 @@
$ cat > dune-project << EOF
> (lang dune 2.0)
> (using action-plugin 0.1)
> EOF
$ cat > dune <<EOF
> (executable
> (name exit)
> (modules exit))
> (rule (with-stdout-to exit.ml (echo "let () = exit (int_of_string Sys.argv.(1))")))
> EOF
$ cat >> dune <<EOF
> (rule
> (alias a)
> (action (with-accepted-exit-codes
> 1
> (with-stdout-to out.txt
> (run ./exit.exe 1)))))
> EOF
$ dune build --display=short --root . @a
File "dune", lines 9-10, characters 10-64:
9 | (with-stdout-to out.txt
10 | (run ./exit.exe 1)))))
Error: nesting modifiers under 'with-accepted-exit-codes' is only available
since version 2.2 of the dune language. Please update your dune-project file
to have (lang dune 2.2).
[1]