This commit is contained in:
swrup 2025-11-11 02:07:51 +01:00
parent aa2ff7b2f0
commit 2f3113f55d
11742 changed files with 1223940 additions and 0 deletions

View file

@ -0,0 +1,51 @@
This checks what happens when a file available in the cache is used in a directory target.
$ export DUNE_CACHE_ROOT=$PWD/.cache
$ export DUNE_CACHE=enabled
$ . ./helpers.sh
$ cat > dune-project << EOF
> (lang dune 3.11)
> (using directory-targets 0.1)
> EOF
$ cat > dune << EOF
> (rule
> (with-stdout-to file_out (run ./gen.sh)))
>
> (rule
> (target (dir dir_out))
> (deps gen.sh)
> (action
> (no-infer
> (progn
> (run mkdir dir_out)
> (with-stdout-to dir_out/a (run ./gen.sh))
> (write-file dir_out/b contents_b)))))
> EOF
$ cat > gen.sh << EOF
> #!/usr/bin/env bash
> >&2 echo running command
> echo contents
> EOF
$ chmod +x gen.sh
We will check whether an entry is linked from the cache. This corresponds to a
file with more than one link.
We prime the cache with the file:
$ dune build file_out
running command
$ is_linked _build/default/file_out
linked
Then use it in the directory target. We expect the command to run (because we
can not know in advance that it is going to generate that file); but to link
the resulting file from the cache.
$ dune build dir_out/
running command
$ is_linked _build/default/dir_out/a
linked

View file

@ -0,0 +1,52 @@
We expect a clean restore when some directory targets have modes like 755.
See #11533.
This test relies on a particular umask.
$ umask 002
$ export DUNE_CACHE_ROOT=$PWD/.cache
$ export DUNE_CACHE=enabled
$ cat > dune-project <<EOF
> (lang dune 3.0)
> (using directory-targets 0.1)
> EOF
$ cat > dune <<EOF
> (rule
> (targets (dir d))
> (action
> (progn
> (bash "echo building d")
> (run mkdir d)
> (run chmod 755 d)
> (run touch d/x))))
>
> (rule
> (deps d)
> (action
> (progn
> (echo building other)
> (write-file other data))))
> EOF
First build: `d` (with mode 755) and `other` are stored in cache
$ dune build other
building d
building other
The chmod command was run, so this is expected
$ dune_cmd stat permissions _build/default/d
755
Second build: `d` is restored and `other` can use it, so no rebuild happens.
$ dune clean
$ dune build other
We'll note that the permissions are still set to the umask
$ dune_cmd stat permissions _build/default/d
775

View file

@ -0,0 +1,50 @@
We create 2 directory targets which share a whole subdirectory.
$ export DUNE_CACHE_ROOT=$PWD/.cache
$ export DUNE_CACHE=enabled
$ . ./helpers.sh
$ cat > dune-project << EOF
> (lang dune 3.11)
> (using directory-targets 0.1)
> EOF
$ cat > dune << EOF
> (rule
> (target (dir d1))
> (action
> (progn
> (run ./gen.sh d1/shared1)
> (no-infer
> (write-file d1/x contents_x)))))
>
> (rule
> (target (dir d2))
> (action
> (progn
> (run ./gen.sh d2/shared2)
> (no-infer
> (write-file d2/y contents_y)))))
> EOF
$ cat > gen.sh << 'EOF'
> #!/usr/bin/env sh
> out=$1
> mkdir -p $out
> echo contents_a > $out/a
> echo contents_b > $out/b
> EOF
$ chmod +x gen.sh
$ dune build d1/ d2/
We expect the targets to be linked in the shared cache.
$ is_linked _build/default/d1/shared1/a
linked
$ is_linked _build/default/d1/shared1/b
linked
$ is_linked _build/default/d2/shared2/a
linked
$ is_linked _build/default/d2/shared2/b
linked

View file

@ -0,0 +1,32 @@
Symlinks in directory targets should go in the shared cache.
See #11523.
$ export DUNE_CACHE_ROOT=$PWD/.cache
$ export DUNE_CACHE=enabled
$ cat > dune-project << EOF
> (lang dune 3.0)
> (using directory-targets 0.1)
> EOF
$ cat > dune << EOF
> (rule
> (target (dir d))
> (action
> (progn
> (run mkdir -p d)
> (chdir d
> (progn
> (echo building)
> (run touch target)
> (run ln -s target symlink))))))
> EOF
$ dune build d
building
$ dune clean
$ dune build d
building

View file

@ -0,0 +1,65 @@
We test that directory targets can go in the shared cache. See #8067.
$ export DUNE_CACHE_ROOT=$PWD/.cache
$ export DUNE_CACHE=enabled
In project a, we create a rule with a directory target. The script that creates
the target displays a message.
$ mkdir a
$ cd a
$ cat > dune-project << EOF
> (lang dune 3.0)
> (using directory-targets 0.1)
> EOF
$ cat > create.sh << EOF
> #!/usr/bin/env sh
> echo "Running create.sh"
> mkdir out/
> echo contents_a > out/a
> echo contents_b > out/b
> EOF
$ chmod +x create.sh
$ cat > dune << EOF
> (rule
> (target (dir out))
> (action
> (run ./create.sh)))
> EOF
We run, and expect the error message to be displayed, the targets to be
created, and the cache to be populated.
$ dune build out
Running create.sh
$ dune_cmd exists _build/default/out/a
true
$ dune_cmd exists _build/default/out/b
true
$ dune cache size
0B
Now we create another project with the same contents.
$ cd ..
$ mkdir b
$ cp a/dune-project a/dune a/create.sh b/
$ cd b
When building, we expect to restore from cache. So the message should not be
displayed, but the files should be created.
$ dune build out
$ dune_cmd exists _build/default/out/a
true
$ dune_cmd exists _build/default/out/b
true
If part of a directory target is removed, we expect it to be rebuilt using the
cache.
$ dune_cmd wait-for-fs-clock-to-advance
$ rm _build/default/out/a
$ dune build out
$ dune_cmd exists _build/default/out/a
true

View file

@ -0,0 +1,57 @@
Copy files from inside a directory target
$ cat >dune-project <<EOF
> (lang dune 3.0)
> (using directory-targets 0.1)
> EOF
Copy from a generated sub-directory
-----------------------------------
This test just documents that copying from a generated sub-directory
causes a cycle. In theory, it would be possible to avoid but it would
requires deep changes in Dune. The cycle exists at the moment because
Dune loads all the rules of a directory at once.
$ cat >dune <<EOF
> (rule
> (target (dir foo))
> (deps (sandbox always))
> (action (system "mkdir foo && touch foo/x foo/y foo/z")))
> (copy_files foo/*)
> EOF
$ dune build
Error: Dependency cycle between:
Computing directory contents of _build/default
-> { dir = In_build_dir "default/foo"
; predicate = Glob (Glob "*")
; only_generated_files = false
}
-> Computing directory contents of _build/default
[1]
$ ls _build/default/
Copy from a generated directory somewhere else
----------------------------------------------
$ rm -f dune
$ mkdir a b
$ cat >a/dune <<EOF
> (rule
> (target (dir foo))
> (deps (sandbox always))
> (action (system "mkdir foo && touch foo/x foo/y foo/z")))
> EOF
$ cat >b/dune <<EOF
> (copy_files ../a/foo/*)
> EOF
$ dune build b
$ ls _build/default/b
x
y
z

View file

@ -0,0 +1,128 @@
Sandbox a rule that depends on a directory target using the copying sandbox
mode:
$ umask 022
$ cat >dune-project <<EOF
> (lang dune 3.8)
> (using directory-targets 0.1)
> EOF
$ cat >print-permissions.sh <<EOF
> /usr/bin/env sh
> echo permissions of output/
> dune_cmd stat permissions output
> echo permissions of output/y
> dune_cmd stat permissions output/y
> echo permissions of output/x
> dune_cmd stat permissions output/x
> echo permissions of output/subdir
> dune_cmd stat permissions output/subdir
> echo permissions of output/subdir/z
> dune_cmd stat permissions output/subdir/z
> echo ""
> EOF
$ chmod +x print-permissions.sh
$ cat >dune <<EOF
> (rule
> (deps print-permissions.sh)
> (targets (dir output))
> (action (system "\| echo building output/
> "\| mkdir -p output/subdir
> "\| touch output/x output/y output/subdir/z
> "\| chmod 0777 output/
> "\| chmod 0444 output/y
> "\| chmod 0776 output/subdir
> "\| chmod 0666 output/subdir/z
> "\| . ./print-permissions.sh
> )))
> (rule
> (target foo)
> (deps print-permissions.sh (sandbox always) output/)
> (action (system "\| . ./print-permissions.sh
> "\| touch foo;
> )))
> EOF
$ dune build foo --sandbox=copy
building output/
permissions of output/
777
permissions of output/y
444
permissions of output/x
644
permissions of output/subdir
776
permissions of output/subdir/z
666
permissions of output/
755
permissions of output/y
644
permissions of output/x
644
permissions of output/subdir
754
permissions of output/subdir/z
644
$ ( cd _build/default && ../../print-permissions.sh )
permissions of output/
777
permissions of output/y
444
permissions of output/x
444
permissions of output/subdir
776
permissions of output/subdir/z
444
Now we demonstrate that symlinks aren't supported:
$ mkdir sub
$ cat >dune <<EOF
> (rule
> (alias foo)
> (action (echo test))
> (deps sub/targetdir))
> EOF
$ runtest() {
> cat >sub/dune <<EOF
> (rule
> (target (dir targetdir))
> (action (system "mkdir targetdir && cd targetdir && $1")))
> EOF
> dune build @foo --sandbox=copy
> }
$ runtest "touch foo && ln -s foo bar"
test
$ runtest "mkdir bar && touch bar/somefileinbar && ln -s bar symlinktobar"
File "sub/dune", lines 1-3, characters 0-151:
1 | (rule
2 | (target (dir targetdir))
3 | (action (system "mkdir targetdir && cd targetdir && mkdir bar && touch bar/somefileinbar && ln -s bar symlinktobar")))
Error: Error trying to read targets after a rule was run:
- sub/targetdir/symlinktobar: Unexpected file kind "S_DIR" (directory)
[1]
Now we try a broken symlink:
$ runtest "ln -s foo bar"
File "sub/dune", lines 1-3, characters 0-102:
1 | (rule
2 | (target (dir targetdir))
3 | (action (system "mkdir targetdir && cd targetdir && ln -s foo bar")))
Error: Error trying to read targets after a rule was run:
- sub/targetdir/bar: Broken symbolic link
[1]

View file

@ -0,0 +1,29 @@
Test creating directory targets by symlinking:
$ cat > dune-project <<EOF
> (lang dune 3.3)
> (using directory-targets 0.1)
> EOF
$ cat >dune <<EOF
> (rule
> (targets (dir symlinked))
> ;; not exactly correctly, but it's just a test
> (deps bar/foo (sandbox always))
> (action (system "ln -s ./bar symlinked")))
> EOF
$ mkdir bar && touch bar/foo
$ dune build ./symlinked
$ {
> path=_build/default/symlinked
> if [ -e $path ]
> then
> printf "symlink exists and points to: %s" "$(readlink $path)"
> else
> echo symlink does not exist
> fi
> }
symlink exists and points to: ./bar

View file

@ -0,0 +1,25 @@
Attempt to create a directory with chdir + with-stdout-to:
$ cat > dune-project <<EOF
> (lang dune 3.1)
> (using directory-targets 0.1)
> EOF
$ cat > dune <<EOF
> (rule
> (targets (dir output))
> (deps (sandbox always))
> (action
> (progn
> (chdir output
> (with-stdout-to x (echo foobar))))))
> EOF
$ dune build foobar/
File "dune", line 7, characters 20-21:
7 | (with-stdout-to x (echo foobar))))))
^
Error: This action has targets in a different directory than the current one,
this is not allowed by dune at the moment:
- output/x
[1]

View file

@ -0,0 +1,31 @@
Directory targets and ocaml/coq/etc sources
$ cat >dune-project <<EOF
> (lang dune 3.2)
> (using directory-targets 0.1)
> EOF
$ cat >produce.sh <<EOF
> mkdir sources
> cd sources
> echo "print_endline Foo.bar;;" > main.ml
> echo "let bar = "hello world" > main.ml
> EOF
$ chmod +x produce.sh
$ cat >dune <<EOF
> (rule
> (targets (dir sources))
> (action (run ./produce.sh)))
> (include_subdirs unqualified)
> (executable (name main))
> EOF
$ dune exec ./main.exe
File "dune", line 5, characters 18-22:
5 | (executable (name main))
^^^^
Error: Module "Main" doesn't exist.
[1]

View file

@ -0,0 +1,7 @@
(cram
(deps helpers.sh)
(applies_to :whole_subtree))
(cram
(applies_to remove-write-permissions)
(deps %{bin:bash}))

View file

@ -0,0 +1,7 @@
(rule
(targets (dir foo))
(action (run mkdir foo)))
(rule
(targets (dir foo))
(action (run mkdir foo)))

View file

@ -0,0 +1,2 @@
(lang dune 3.5)
(using directory-targets 0.1)

View file

@ -0,0 +1,12 @@
Duplicate directory targets
$ dune build
File "dune", lines 1-3, characters 0-53:
1 | (rule
2 | (targets (dir foo))
3 | (action (run mkdir foo)))
Error: The following both define the same directory target:
_build/default/foo
- dune:1
- dune:5
[1]

View file

@ -0,0 +1,11 @@
(executable
(name fakenpm)
(modules fakenpm)
(libraries unix))
(env
(_
(binaries fakenpm.exe)))
(cram
(deps fakenpm.exe))

View file

@ -0,0 +1,6 @@
let () =
Sys.mkdir "fakenode_modules" 0o777;
Sys.mkdir "fakenode_modules/foo" 0o777;
open_out "fakenode_modules/foo/file" |> close_out;
Unix.symlink "file" "./fakenode_modules/foo/bar"
;;

View file

@ -0,0 +1,18 @@
Testing a bug with npm creating a directory target with a symlink inside but
Dune not recognizing it
$ cat > dune-project << EOF
> (lang dune 3.5)
> (using directory-targets 0.1)
> EOF
$ cat > dune << EOF
> (rule
> (alias fakenode)
> (targets
> (dir fakenode_modules))
> (action
> (run ./fakenpm.exe)))
> EOF
$ dune build @fakenode

View file

@ -0,0 +1,19 @@
This test makes sure that whenever we evaluate a glob inside a directory target
that matches nothing, we still copy the directory and make it empty.
$ cat > dune-project <<EOF
> (lang dune 3.0)
> (using directory-targets 0.1)
> EOF
$ cat > dune <<EOF
> (rule
> (targets (dir output))
> (action (system "mkdir output && touch output/foo.txt")))
> (rule
> (targets x)
> (deps (glob_files output/*.baz))
> (action (system "ls output/ && touch x")))
> EOF
$ DUNE_SANDBOX=copy dune build x

View file

@ -0,0 +1,4 @@
is_linked() {
nlinks=$(dune_cmd stat hardlinks "$1")
[ "$nlinks" -gt 1 ] && echo linked || echo not linked
}

View file

@ -0,0 +1,67 @@
Allow directories to be installable
$ cat >dune-project <<EOF
> (lang dune 3.5)
> (package (name foo))
> (using directory-targets 0.1)
> EOF
$ cat >dune <<EOF
> (install
> (dirs rules/bar)
> (section share))
> (install
> (dirs (rules/another-dir as renamed) (rules/nested as some/nesting/here))
> (section lib))
> EOF
$ mkdir rules
$ cat >rules/dune <<EOF
> (rule
> (target (dir bar))
> (deps (sandbox always))
> (action (bash "mkdir -p %{target}/baz && touch %{target}/{x,y,z} && touch %{target}/baz/{a,b}")))
> (rule
> (target (dir another-dir))
> (deps (sandbox always))
> (action (chdir %{target} (run touch x))))
> (rule
> (target (dir nested))
> (deps (sandbox always))
> (action (chdir %{target} (run touch x))))
> EOF
$ dune build foo.install
$ cat _build/default/foo.install
lib: [
"_build/install/default/lib/foo/META"
"_build/install/default/lib/foo/dune-package"
"_build/install/default/lib/foo/renamed/x" {"renamed/x"}
"_build/install/default/lib/foo/some/nesting/here/x" {"some/nesting/here/x"}
]
share: [
"_build/install/default/share/foo/bar/baz/a" {"bar/baz/a"}
"_build/install/default/share/foo/bar/baz/b" {"bar/baz/b"}
"_build/install/default/share/foo/bar/x" {"bar/x"}
"_build/install/default/share/foo/bar/y" {"bar/y"}
"_build/install/default/share/foo/bar/z" {"bar/z"}
]
$ mkdir ./installation
$ dune install --prefix ./installation --display short
Installing installation/lib/foo/META
Installing installation/lib/foo/dune-package
Installing installation/lib/foo/renamed/x
Installing installation/lib/foo/some/nesting/here/x
Installing installation/share/foo/bar/baz/a
Installing installation/share/foo/bar/baz/b
Installing installation/share/foo/bar/x
Installing installation/share/foo/bar/y
Installing installation/share/foo/bar/z
$ ls ./installation/lib/foo
META
dune-package
renamed
some
$ ls ./installation/lib/foo/some/nesting/here
x

View file

@ -0,0 +1,52 @@
Allow directories to be installable
$ mkdir a b prefix
$ cat >a/dune-project <<EOF
> (lang dune 3.5)
> (package (name foo))
> (using directory-targets 0.1)
> EOF
$ touch a/x a/y
$ cat >a/dune <<EOF
> (install
> (dirs rules/bar)
> (files x y)
> (section share))
> EOF
$ mkdir a/rules
$ cat >a/rules/dune <<EOF
> (rule
> (target (dir bar))
> (deps (sandbox always))
> (action (bash "mkdir -p %{target}/baz && touch %{target}/{x,y,z} && touch %{target}/baz/{a,b}")))
> EOF
$ dune build --root=a foo.install
Entering directory 'a'
Leaving directory 'a'
$ sed -E 's/lang dune [0-9.]+/lang dune XXX/' a/_build/install/default/lib/foo/dune-package
(lang dune XXX)
(name foo)
(sections (lib .) (share ../../share/foo))
(files (lib (META dune-package)) (share ((dir bar) x y)))
$ dune install --root a --prefix $PWD/prefix --display short
Installing $TESTCASE_ROOT/prefix/lib/foo/META
Installing $TESTCASE_ROOT/prefix/lib/foo/dune-package
Installing $TESTCASE_ROOT/prefix/share/foo/bar/baz/a
Installing $TESTCASE_ROOT/prefix/share/foo/bar/baz/b
Installing $TESTCASE_ROOT/prefix/share/foo/bar/x
Installing $TESTCASE_ROOT/prefix/share/foo/bar/y
Installing $TESTCASE_ROOT/prefix/share/foo/bar/z
Installing $TESTCASE_ROOT/prefix/share/foo/x
Installing $TESTCASE_ROOT/prefix/share/foo/y
$ cat > b/dune-project <<EOF
> (lang dune 3.5)
> EOF
$ cat > b/dune <<EOF
> (alias (name foo) (deps (package foo)))
> EOF
$ OCAMLPATH=$PWD/prefix/lib/:$OCAMLPATH dune build --root=b @foo --display=short
Entering directory 'b'
Leaving directory 'b'

View file

@ -0,0 +1,57 @@
This test tries to load the rules in a directory that is a target of another
rule.
$ cat > dune-project <<EOF
> (lang dune 3.4)
> (using directory-targets 0.1)
> EOF
$ cat >dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "echo creating output dir && mkdir -p output/a && touch output/a/b")))
> EOF
$ dune build --debug-load-dir output/
Loading build directory _build/default
Loading build directory _build/default/.dune
Loading build directory _build
creating output dir
$ find _build/default/output
_build/default/output
_build/default/output/a
_build/default/output/a/b
We are loading the rules in output/a and while making sure that we don't delete
and re-create output/b. The following should not re-run the rule that recreates
output/
$ dune build --debug-load-dir output/a/b
Loading build directory _build/default/output/a
Loading build directory _build/default
Loading build directory _build/default/.dune
Loading build directory _build
$ find _build/default/output
_build/default/output
_build/default/output/a
_build/default/output/a/b
Now we try loading the rules in output/a and make sure that nothing is deleted:
$ dune rules --debug-load-dir output/a/
Loading build directory _build/default/output
Loading build directory _build/default
Loading build directory _build/default/.dune
Loading build directory _build
((deps ())
(targets ((files ()) (directories (_build/default/output))))
(context default)
(action
(chdir
_build/default
(bash "echo creating output dir && mkdir -p output/a && touch output/a/b"))))
$ find _build/default/output
_build/default/output
_build/default/output/a
_build/default/output/a/b

View file

@ -0,0 +1,482 @@
Tests for directory targets.
$ cat > dune-project <<EOF
> (lang dune 3.0)
> EOF
Directory targets require an extension.
$ cat > dune <<EOF
> (rule
> (targets (dir output))
> (action (bash "true")))
> EOF
$ dune build output/x
File "dune", line 2, characters 16-22:
2 | (targets (dir output))
^^^^^^
Error: Directory targets require the 'directory-targets' extension
[1]
$ cat > dune-project <<EOF
> (lang dune 3.0)
> (using directory-targets 0.1)
> EOF
Ensure directory targets are produced.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "true")))
> EOF
$ dune build output/x
File "dune", lines 1-4, characters 0-82:
1 | (rule
2 | (deps (sandbox always))
3 | (targets (dir output))
4 | (action (bash "true")))
Error: Rule failed to produce directory "output"
[1]
Error message when the matching directory target is empty.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "mkdir output")))
> EOF
$ dune build output/x
File "dune", lines 1-4, characters 0-90:
1 | (rule
2 | (deps (sandbox always))
3 | (targets (dir output))
4 | (action (bash "mkdir output")))
Error: Rule produced directory "output" that contains no files nor non-empty
subdirectories
[1]
Error message when the matching directory target doesn't contain a requested path.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "mkdir output && touch output/y")))
> EOF
$ dune build output/x
File "dune", lines 1-4, characters 0-108:
1 | (rule
2 | (deps (sandbox always))
3 | (targets (dir output))
4 | (action (bash "mkdir output && touch output/y")))
Error: This rule defines a directory target "output" that matches the
requested path "output/x" but the rule's action didn't produce it
[1]
Build directory target from the command line.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
> EOF
$ dune build output/x
$ cat _build/default/output/x
x
$ cat _build/default/output/y
y
Test that workspace-local cache works for directory targets.
$ dune build output/x --debug-cache=workspace-local
Requesting the directory target directly works too.
$ cat > dune <<EOF
> (rule
> (deps src_x (sandbox always))
> (targets (dir output))
> (action (bash "mkdir output; cat src_x > output/x; echo y > output/y")))
> EOF
$ rm -rf _build
$ echo x > src_x
$ dune build output
$ cat _build/default/output/x
x
$ cat _build/default/output/y
y
Rebuilding works correctly.
$ echo new-x > src_x
$ dune build output
$ cat _build/default/output/x
new-x
Hints for directory targets.
$ dune build outputs
Error: Don't know how to build outputs
Hint: did you mean output?
[1]
Print rules:
$ dune rules -m output | tr '\t' ' '
_build/default/output: _build/default/src_x
mkdir -p _build/default; \
mkdir -p _build/default; \
cd _build/default; \
bash -e -u -o pipefail -c \
'mkdir output; cat src_x > output/x; echo y > output/y'
$ dune rules output
((deps ((File (In_build_dir _build/default/src_x))))
(targets ((files ()) (directories (_build/default/output))))
(context default)
(action
(chdir
_build/default
(bash "mkdir output; cat src_x > output/x; echo y > output/y"))))
Error when requesting a missing subdirectory of a directory target.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
> EOF
$ dune build output/subdir
File "dune", lines 1-4, characters 0-128:
1 | (rule
2 | (deps (sandbox always))
3 | (targets (dir output))
4 | (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
Error: This rule defines a directory target "output" that matches the
requested path "output/subdir" but the rule's action didn't produce it
[1]
Error message when depending on a file that isn't produced by the matching
directory target.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "\| mkdir -p output/subdir;
> "\| echo a > output/a;
> "\| echo b > output/subdir/b
> )))
> (rule
> (deps output/subdir/c)
> (target main)
> (action (bash "cat output/subdir/c > main")))
> EOF
$ dune build main
File "dune", lines 1-7, characters 0-188:
1 | (rule
2 | (deps (sandbox always))
3 | (targets (dir output))
4 | (action (bash "\| mkdir -p output/subdir;
5 | "\| echo a > output/a;
6 | "\| echo b > output/subdir/b
7 | )))
Error: This rule defines a directory target "output" that matches the
requested path "output/subdir/c" but the rule's action didn't produce it
-> required by _build/default/main
[1]
Depend on a file from a directory target.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "\| mkdir -p output/subdir;
> "\| echo a > output/a;
> "\| echo b > output/subdir/b
> )))
> (rule
> (deps output/subdir/b)
> (target main)
> (action (bash "cat output/subdir/b > main; echo 2 >> main")))
> EOF
$ dune build main
$ cat _build/default/main
b
2
$ cat _build/default/output/a
a
$ cat _build/default/output/subdir/b
b
Interaction of globs and directory targets.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "\| mkdir -p output/subdir;
> "\| echo a > output/a.txt;
> "\| echo b > output/b.txt;
> "\| echo c > output/c;
> "\| echo d > output/subdir/d.txt;
> "\| echo e > output/subdir/e
> )))
> (rule
> (deps (glob_files output/*.txt))
> (target level1)
> (action (bash "echo %{deps}; ls output > level1")))
> (rule
> (deps (glob_files output/subdir/*))
> (target level2)
> (action (bash "echo %{deps}; ls output/subdir > level2")))
> EOF
Note: %{deps} expands to the set of generated files that match the glob [*.txt],
however, the action currently has access to all of the paths, along with any of
the subdirectories included into the directory target.
# CR-someday amokhov: Remove the files that action didn't depend on.
$ dune build level1
output/a.txt output/b.txt
$ cat _build/default/level1
a.txt
b.txt
c
subdir
Depending on a glob in a subdirectory of a directory target works too.
$ dune build level2
output/subdir/d.txt output/subdir/e
$ cat _build/default/level2
d.txt
e
Depending on a directory target directly (rather than on individual files) works
too. Note that this can be achieved in two ways:
(1) By depending on the recursively computed digest of the directory's contents;
(2) By depending on the mtime of the directory.
Currently Dune implements (2) but we'd like to switch to (1) because it supports
the early cutoff optimisation and is also more reliable.
The [src_c] dependency is unused in the rule's action but we use it to force the
rule to rerun when needed.
# CR-someday amokhov: Right now we accept simply "output" as a dependency
# specification, which is inconsistent with the target specification. This
# should be fixed, i.e. we should require "(dir output)" instead.
$ cat > dune <<EOF
> (rule
> (deps src_a src_b src_c (sandbox always))
> (targets (dir output))
> (action (bash "\| echo running;
> "\| mkdir -p output/subdir;
> "\| cat src_a > output/a;
> "\| cat src_b > output/subdir/b
> )))
> (rule
> (deps output)
> (target contents)
> (action (bash "echo running; echo 'a:' > contents; cat output/a >> contents; echo 'b:' >> contents; cat output/subdir/b >> contents")))
> EOF
$ echo a > src_a
$ echo b > src_b
$ echo c > src_c
$ dune build contents
running
running
$ cat _build/default/contents
a:
a
b:
b
We wait for the file system's clock to advance to make sure the directory's
mtime changes when the rule reruns. We can delete this when switching to (1).
$ dune_cmd wait-for-fs-clock-to-advance
$ echo new-b > src_b
$ dune build contents
running
running
$ cat _build/default/contents
a:
a
b:
new-b
Dune supports early cutoff for directory targets: it skips the second action
since the produced directory has the same contents.
$ dune_cmd wait-for-fs-clock-to-advance
$ echo new-cc > src_c
$ dune build contents
running
$ cat _build/default/contents
a:
a
b:
new-b
Check that Dune clears stale files from directory targets.
$ cat > dune <<EOF
> (rule
> (deps src_a src_b src_c (sandbox always))
> (targets (dir output))
> (action (bash "\| echo running;
> "\| mkdir -p output/subdir;
> "\| cat src_a > output/new-a;
> "\| cat src_b > output/subdir/b
> )))
> (rule
> (deps output)
> (target contents)
> (action (bash "echo running; echo 'new-a:' > contents; cat output/new-a >> contents; echo 'b:' >> contents; cat output/subdir/b >> contents")))
> EOF
$ dune build contents
running
running
Note that the stale "output/a" file got removed.
$ ls _build/default/output | sort
new-a
subdir
Directory target whose name conflicts with an internal directory used by Dune.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir .dune))
> (action (bash "mkdir .dune; echo hello > .dune/hello")))
> EOF
$ dune build .dune/hello
File "dune", lines 1-4, characters 0-114:
1 | (rule
2 | (deps (sandbox always))
3 | (targets (dir .dune))
4 | (action (bash "mkdir .dune; echo hello > .dune/hello")))
Error: This rule defines a target ".dune" whose name conflicts with an
internal directory used by Dune. Please use a different name.
[1]
Multi-component target directories are not allowed.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output/subdir))
> (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
> EOF
$ dune build output/x
File "dune", line 3, characters 16-29:
3 | (targets (dir output/subdir))
^^^^^^^^^^^^^
Error: Directory targets must have exactly one path component.
[1]
File and directory target with the same name.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets output (dir output))
> (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
> EOF
$ dune build output/x
File "dune", lines 1-4, characters 0-135:
1 | (rule
2 | (deps (sandbox always))
3 | (targets output (dir output))
4 | (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
Error: "output" is declared as both a file and a directory target.
[1]
File and directory target with the same name, defined in different rules.
$ cat > dune <<EOF
> (rule
> (targets output)
> (action (bash "echo hi > output")))
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
> EOF
$ dune build output/x
Error: Multiple rules generated for _build/default/output:
- dune:1
- dune:4
[1]
Conflict between a target directory and a source directory.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
> EOF
$ mkdir output
$ dune build output/x
File "dune", lines 1-4, characters 0-128:
1 | (rule
2 | (deps (sandbox always))
3 | (targets (dir output))
4 | (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
Error: This rule defines a target "output" whose name conflicts with a source
directory in the same directory.
Hint: If you want Dune to generate and replace "output", add (mode promote)
to the rule stanza. Alternatively, you can delete "output" from the source
tree or change the rule to generate a different target.
[1]
Conflict between a target directory and a source file.
$ cat > dune <<EOF
> (rule
> (deps (sandbox always))
> (targets (dir output))
> (action (bash "mkdir output; echo x > output/x; echo y > output/y")))
> EOF
$ rm -rf output
$ touch output
$ dune build output/x
Error: Multiple rules generated for _build/default/output:
- file present in source tree
- dune:1
Hint: rm -f output
[1]

View file

@ -0,0 +1,88 @@
Tests for directory targets that are produced by unsandboxed rule
$ cat > dune-project <<EOF
> (lang dune 3.4)
> (using directory-targets 0.1)
> EOF
Build directory target from the command line without sandboxing
$ cat > dune <<EOF
> (rule
> (targets (dir output))
> (action (system "mkdir output; echo x > output/x; echo y > output/y")))
> EOF
$ dune build output/x
$ cat _build/default/output/x
x
$ cat _build/default/output/y
y
We ask to build a file that doesn't exist inside the directory:
$ dune build output/fake
File "dune", lines 1-3, characters 0-102:
1 | (rule
2 | (targets (dir output))
3 | (action (system "mkdir output; echo x > output/x; echo y > output/y")))
Error: This rule defines a directory target "output" that matches the
requested path "output/fake" but the rule's action didn't produce it
[1]
When we fail to create the directory, dune complains:
$ cat > dune <<EOF
> (rule
> (targets (dir output))
> (action (system "true")))
> EOF
$ dune build output/
File "dune", lines 1-3, characters 0-56:
1 | (rule
2 | (targets (dir output))
3 | (action (system "true")))
Error: Rule failed to produce directory "output"
[1]
Check that Dune clears stale files from directory targets.
$ cat >dune <<EOF
> (rule
> (deps src_a src_b src_c)
> (targets (dir output))
> (action (bash "\| echo running;
> "\| mkdir -p output/subdir;
> "\| cat src_a > output/new-a;
> "\| cat src_b > output/subdir/b
> )))
> (rule
> (deps output)
> (target contents)
> (action (bash "echo running; echo 'new-a:' > contents; cat output/new-a >> contents; echo 'b:' >> contents; cat output/subdir/b >> contents")))
> EOF
$ echo a > src_a
$ echo b > src_b
$ echo c > src_c
$ dune build contents
running
running
Directory target whose name conflicts with an internal directory used by Dune.
$ cat > dune <<EOF
> (rule
> (targets (dir .dune))
> (action (bash "mkdir .dune; echo hello > .dune/hello")))
> EOF
$ dune build .dune/hello
File "dune", lines 1-3, characters 0-88:
1 | (rule
2 | (targets (dir .dune))
3 | (action (bash "mkdir .dune; echo hello > .dune/hello")))
Error: This rule defines a target ".dune" whose name conflicts with an
internal directory used by Dune. Please use a different name.
[1]

View file

@ -0,0 +1,4 @@
(alias
(name default)
(deps dir)
(action (bash "cat %{deps}/*")))

View file

@ -0,0 +1,4 @@
$ dune build
Error: No rule found for dir
-> required by alias default in dune:1
[1]

View file

@ -0,0 +1,3 @@
(rule
(targets dir)
(action (bash "mkdir %{targets}")))

View file

@ -0,0 +1,14 @@
File targets overlapping with directories
$ dune build @all
File "dune", lines 1-3, characters 0-57:
1 | (rule
2 | (targets dir)
3 | (action (bash "mkdir %{targets}")))
Error: This rule defines a target "dir" whose name conflicts with a source
directory in the same directory.
Hint: If you want Dune to generate and replace "dir", add (mode promote) to
the rule stanza. Alternatively, you can delete "dir" from the source tree or
change the rule to generate a different target.
[1]

View file

@ -0,0 +1,4 @@
(rule
(targets dir)
(mode promote)
(action (bash "mkdir %{targets}")))

View file

@ -0,0 +1,5 @@
directory target and (mode promote)
$ dune build @all 2>&1 | head -n2
Error: Is a directory
-> required by _build/default/dir

View file

@ -0,0 +1,2 @@
(rule
(with-stdout-to foo (echo "foobar\n")))

View file

@ -0,0 +1,11 @@
(executable
(name foo)
(libraries unix))
(rule
(targets dir)
(action (run ./foo.exe dir)))
(alias
(name default)
(deps dir dir/foo))

View file

@ -0,0 +1,14 @@
let () = Printexc.record_backtrace false
let dir = Sys.argv.(1)
let write f =
let path = Filename.concat dir f in
let out = open_out path in
output_string out (f ^ " contents\n");
close_out_noerr out
let () =
Unix.mkdir dir 0o777;
write "foo";
write "bar"

View file

@ -0,0 +1,14 @@
We should not be able to produce a directory in a rule that already exists
$ dune build
File "dune", lines 5-7, characters 0-51:
5 | (rule
6 | (targets dir)
7 | (action (run ./foo.exe dir)))
Error: This rule defines a target "dir" whose name conflicts with a source
directory in the same directory.
Hint: If you want Dune to generate and replace "dir", add (mode promote) to
the rule stanza. Alternatively, you can delete "dir" from the source tree or
change the rule to generate a different target.
[1]

View file

@ -0,0 +1,19 @@
let dir = Sys.argv.(1)
let input_all t =
let buffer = Buffer.create 0 in
let rec loop () =
Buffer.add_channel buffer t 65536;
loop ()
in
try loop () with
| End_of_file -> Buffer.contents buffer
let () =
let files =
Sys.readdir dir |> Array.to_list |> ListLabels.sort ~cmp:String.compare
in
ListLabels.iter files ~f:(fun file ->
let inp = open_in (Filename.concat dir file) in
Printf.printf "%s:\n%s\n" file (input_all inp)
)

View file

@ -0,0 +1,22 @@
(executable
(name foo)
(modules foo)
(libraries unix))
(executable
(name cat_dir)
(modules cat_dir)
(libraries unix))
(rule
(targets dir)
(action (run ./foo.exe dir)))
(alias
(name default)
(deps dir))
(alias
(name cat_dir)
(deps dir)
(action (run ./cat_dir.exe dir)))

View file

@ -0,0 +1,12 @@
let dir = Sys.argv.(1)
let write f =
let path = Filename.concat dir f in
let out = open_out path in
output_string out (f ^ " contents\n");
close_out_noerr out
let () =
Unix.mkdir dir 0o777;
write "foo";
write "bar"

View file

@ -0,0 +1,12 @@
$ dune build && cat _build/default/dir/*
bar contents
foo contents
$ dune build @cat_dir
bar:
bar contents
foo:
foo contents

View file

@ -0,0 +1,122 @@
Promotion of directory targets.
$ mkdir test; cd test
$ cat > dune-project <<EOF
> (lang dune 3.0)
> (using directory-targets 0.1)
> EOF
$ cat > dune <<EOF
> (rule
> (mode promote)
> (deps (sandbox always))
> (targets a (dir dir))
> (action (bash "\| echo a > a;
> "\| mkdir -p dir/subdir;
> "\| echo b > dir/b;
> "\| echo c > dir/c;
> "\| echo d > dir/subdir/d
> )))
> EOF
$ dune build a
$ cat a dir/b dir/c dir/subdir/d
a
b
c
d
If a destination directory is taken up by a file, Dune deletes it.
$ rm -rf dir
$ mkdir dir
$ touch dir/subdir
$ dune build a
$ cat a dir/b dir/c dir/subdir/d
a
b
c
d
If a destination file is taken up by a directory, Dune deletes it.
$ rm dir/b
$ mkdir -p dir/b
$ touch dir/b
$ dune build a
$ cat a dir/b dir/c dir/subdir/d
a
b
c
d
Promoting a badly specified directory target gives a weird error:
$ cat > dune-project <<EOF
> (lang dune 3.2)
> (using directory-targets 0.1)
> EOF
$ cat > dune <<EOF
> (rule
> (targets blah-blah)
> (deps (sandbox always))
> (mode promote)
> (action (bash "mkdir %{targets}")))
> EOF
$ dune build
Error: Is a directory
-> required by _build/default/blah-blah
-> required by alias all
-> required by alias default
[1]
Test error message for (promote (into <dir>)) if <dir> is missing.
$ cat > dune <<EOF
> (rule
> (mode (promote (into another_dir)))
> (deps (sandbox always))
> (targets a (dir dir))
> (action (bash "\| echo a > a;
> "\| mkdir -p dir/subdir;
> "\| echo b > dir/b;
> "\| echo c > dir/c;
> "\| echo d > dir/subdir/d
> )))
> EOF
$ dune build a
File "dune", line 2, characters 22-33:
2 | (mode (promote (into another_dir)))
^^^^^^^^^^^
Error: Directory "another_dir" does not exist. Please create it manually.
-> required by _build/default/a
[1]
Test cleaning up unexpected files and directories in directory targets.
$ cat > dune <<EOF
> (rule
> (mode (promote))
> (deps (sandbox always))
> (targets a (dir dir))
> (action (bash "\| echo a > a;
> "\| mkdir -p dir/subdir;
> "\| echo b > dir/b;
> "\| echo c > dir/c;
> "\| echo d > dir/subdir/d
> )))
> EOF
$ mkdir -p dir/unexpected-dir-1
$ mkdir -p dir/subdir/unexpected-dir-2
$ touch dir/unexpected-file-1
$ touch dir/unexpected-dir-1/unexpected-file-2
$ touch dir/subdir/unexpected-file-3
$ dune build a
$ ls dir | grep unexpected
[1]
$ ls dir/subdir | grep unexpected
[1]

View file

@ -0,0 +1,23 @@
Write permissions on directory targets.
$ umask 022
$ cat >dune-project <<EOF
> (lang dune 3.4)
> (using directory-targets 0.1)
> EOF
$ cat >dune <<EOF
> (rule
> (action (bash "mkdir -p foo/foo2 && touch foo/foo2/bar"))
> (targets (dir foo)))
> EOF
$ dune build ./foo
$ dir=_build/default/foo
$ dune_cmd stat permissions $dir
755
$ dune_cmd stat permissions $dir/foo2
755
$ dune_cmd stat permissions $dir/foo2/bar
444

View file

@ -0,0 +1,28 @@
Depend on a source directory.
Currently, this feature isn't working. It's only possible to depend on
directories that are a target of a rule.
$ cat >dune-project <<EOF
> (lang dune 3.0)
> (using directory-targets 0.1)
> EOF
$ mkdir foo
$ touch foo/{x,y,z}
$ cat >dune <<EOF
> (rule
> (deps foo)
> (target bar)
> (action (bash "ls -f %{deps} > %{target}")))
> EOF
$ dune build ./bar
File "dune", lines 1-4, characters 0-77:
1 | (rule
2 | (deps foo)
3 | (target bar)
4 | (action (bash "ls -f %{deps} > %{target}")))
Error: No rule found for foo
[1]

View file

@ -0,0 +1,23 @@
We test that a directory target with only other subdirs can be
properly promoted.
$ cat > dune-project <<EOF
> (lang dune 3.16)
> (using directory-targets 0.1)
> EOF
$ cat > dune <<EOF
> (rule
> (targets (dir foo))
> (mode (promote (until-clean)))
> (action
> (progn
> (run mkdir -p foo/bar)
> (run touch foo/bar/file1)
> (run mkdir -p foo/bar/baz/qux)
> (run touch foo/bar/baz/qux/file2))))
> EOF
$ dune build foo
$ ls foo/bar/baz/qux
file2

View file

@ -0,0 +1,38 @@
This demonstrates what happens when a directory target contains a symlink that
points to a directory.
See #9873.
$ cat > dune-project << EOF
> (lang dune 3.0)
> (using directory-targets 0.1)
> EOF
$ cat > dune << EOF
> (rule
> (target (dir d))
> (action
> (progn
> (run mkdir -p d)
> (chdir d
> (progn
> (run mkdir a)
> (run touch a/x.txt)
> (run ln -s a b))))))
> EOF
$ dune build d
File "dune", lines 1-10, characters 0-154:
1 | (rule
2 | (target (dir d))
3 | (action
4 | (progn
5 | (run mkdir -p d)
6 | (chdir d
7 | (progn
8 | (run mkdir a)
9 | (run touch a/x.txt)
10 | (run ln -s a b))))))
Error: Error trying to read targets after a rule was run:
- d/b: Unexpected file kind "S_DIR" (directory)
[1]