This commit is contained in:
parent
aa2ff7b2f0
commit
2f3113f55d
11742 changed files with 1223940 additions and 0 deletions
|
|
@ -0,0 +1,133 @@
|
|||
Here we observe the documentation for the dune cache commands.
|
||||
|
||||
$ dune cache --help=plain
|
||||
NAME
|
||||
dune-cache - Manage Dune's shared cache of build artifacts.
|
||||
|
||||
SYNOPSIS
|
||||
dune cache COMMAND …
|
||||
|
||||
DESCRIPTION
|
||||
Dune can share build artifacts between workspaces. We currently only
|
||||
support a few subcommands; however, we plan to provide more
|
||||
functionality soon.
|
||||
|
||||
COMMANDS
|
||||
clear [OPTION]…
|
||||
Clear the Dune cache.
|
||||
|
||||
size [--machine-readable] [OPTION]…
|
||||
Query the size of the Dune cache.
|
||||
|
||||
trim [--size=BYTES] [--trimmed-size=BYTES] [OPTION]…
|
||||
Trim the Dune cache.
|
||||
|
||||
COMMON OPTIONS
|
||||
--help[=FMT] (default=auto)
|
||||
Show this help in format FMT. The value FMT must be one of auto,
|
||||
pager, groff or plain. With auto, the format is pager or plain
|
||||
whenever the TERM env var is dumb or undefined.
|
||||
|
||||
--version
|
||||
Show version information.
|
||||
|
||||
EXIT STATUS
|
||||
dune cache exits with:
|
||||
|
||||
0 on success.
|
||||
|
||||
1 if an error happened.
|
||||
|
||||
130 if it was interrupted by a signal.
|
||||
|
||||
SEE ALSO
|
||||
dune(1)
|
||||
|
||||
Testing the output of `dune cache size --machine-readable`
|
||||
|
||||
$ dune cache size --help=plain
|
||||
NAME
|
||||
dune-cache-size - Query the size of the Dune cache.
|
||||
|
||||
SYNOPSIS
|
||||
dune cache size [--machine-readable] [OPTION]…
|
||||
|
||||
Compute the total size of files in the Dune cache which are not
|
||||
hardlinked from any build directory and output it in a human-readable
|
||||
form.
|
||||
|
||||
OPTIONS
|
||||
--machine-readable
|
||||
Outputs size as a plain number of bytes.
|
||||
|
||||
COMMON OPTIONS
|
||||
--help[=FMT] (default=auto)
|
||||
Show this help in format FMT. The value FMT must be one of auto,
|
||||
pager, groff or plain. With auto, the format is pager or plain
|
||||
whenever the TERM env var is dumb or undefined.
|
||||
|
||||
--version
|
||||
Show version information.
|
||||
|
||||
EXIT STATUS
|
||||
dune cache size exits with:
|
||||
|
||||
0 on success.
|
||||
|
||||
1 if an error happened.
|
||||
|
||||
130 if it was interrupted by a signal.
|
||||
|
||||
SEE ALSO
|
||||
dune(1)
|
||||
|
||||
Testing the output of dune cache trim.
|
||||
|
||||
$ dune cache trim --help=plain
|
||||
NAME
|
||||
dune-cache-trim - Trim the Dune cache.
|
||||
|
||||
SYNOPSIS
|
||||
dune cache trim [--size=BYTES] [--trimmed-size=BYTES] [OPTION]…
|
||||
|
||||
Trim the Dune cache to a specified size or by a specified amount.
|
||||
|
||||
OPTIONS
|
||||
--size=BYTES
|
||||
Size to trim the cache to. BYTES is the number of bytes followed
|
||||
by a unit. Byte units can be one of B, kB, KiB, MB, MiB, GB, GiB,
|
||||
TB or TiB.
|
||||
|
||||
--trimmed-size=BYTES
|
||||
Size to trim from the cache. BYTES is the same as for --size.
|
||||
|
||||
COMMON OPTIONS
|
||||
--help[=FMT] (default=auto)
|
||||
Show this help in format FMT. The value FMT must be one of auto,
|
||||
pager, groff or plain. With auto, the format is pager or plain
|
||||
whenever the TERM env var is dumb or undefined.
|
||||
|
||||
--version
|
||||
Show version information.
|
||||
|
||||
EXIT STATUS
|
||||
dune cache trim exits with:
|
||||
|
||||
0 on success.
|
||||
|
||||
1 if an error happened.
|
||||
|
||||
130 if it was interrupted by a signal.
|
||||
|
||||
EXAMPLES
|
||||
Trimming the Dune cache to 1 GB.
|
||||
|
||||
$ dune cache trim --size=1GB
|
||||
|
||||
Trimming 500 MB from the Dune cache.
|
||||
|
||||
$ dune cache trim --trimmed-size=500MB
|
||||
|
||||
SEE ALSO
|
||||
dune(1)
|
||||
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
Test for the "dune cache clear" command.
|
||||
|
||||
$ export DUNE_CACHE=enabled
|
||||
$ export DUNE_CACHE_ROOT=$PWD/dune-cache
|
||||
|
||||
$ cat >dune-project <<EOF
|
||||
> (lang dune 3.10)
|
||||
> EOF
|
||||
|
||||
$ cat >dune <<EOF
|
||||
> (rule (with-stdout-to foo (progn)))
|
||||
> EOF
|
||||
|
||||
$ dune build
|
||||
|
||||
$ ls $DUNE_CACHE_ROOT | sort -u
|
||||
files
|
||||
meta
|
||||
temp
|
||||
values
|
||||
|
||||
$ dune cache clear
|
||||
|
||||
$ ! test -d $DUNE_CACHE_ROOT
|
||||
|
||||
Next let us add some extra directories/files and check that they are not deleted
|
||||
by mistake.
|
||||
|
||||
$ dune build
|
||||
|
||||
$ mkdir -p $DUNE_CACHE_ROOT/extra; touch $DUNE_CACHE_ROOT/extra1 $DUNE_CACHE_ROOT/extra/extra2
|
||||
|
||||
$ dune cache clear
|
||||
Error:
|
||||
rmdir($TESTCASE_ROOT/dune-cache): Directory not empty
|
||||
[1]
|
||||
|
||||
$ find $DUNE_CACHE_ROOT -type f | sort -u
|
||||
$TESTCASE_ROOT/dune-cache/extra/extra2
|
||||
$TESTCASE_ROOT/dune-cache/extra1
|
||||
|
|
@ -0,0 +1,220 @@
|
|||
Test cache configuration.
|
||||
|
||||
Check that old cache configuration format works fine with an old language
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 2.1)
|
||||
> (cache enabled)
|
||||
> (cache-transport direct)
|
||||
> (cache-duplication copy)
|
||||
> (cache-trim-period 1h)
|
||||
> (cache-trim-size 1GB)
|
||||
> EOF
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 2.1)
|
||||
> EOF
|
||||
$ cat > dune <<EOF
|
||||
> (rule
|
||||
> (deps source)
|
||||
> (targets target)
|
||||
> (action (copy source target)))
|
||||
> EOF
|
||||
$ cat > source <<EOF
|
||||
> \_o< COIN
|
||||
> EOF
|
||||
|
||||
Test that DUNE_CACHE_ROOT can be used to control the cache location
|
||||
|
||||
$ export DUNE_CACHE_ROOT=$PWD/.cache
|
||||
|
||||
Build succeeds and the 'copy' mode is respected
|
||||
|
||||
$ dune build --config-file config target
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
1
|
||||
|
||||
Switch to the 'hardlink' mode now
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 2.1)
|
||||
> (cache enabled)
|
||||
> (cache-transport direct)
|
||||
> (cache-duplication hardlink)
|
||||
> (cache-trim-period 1h)
|
||||
> (cache-trim-size 1GB)
|
||||
> EOF
|
||||
|
||||
Build succeeds and the 'hardlink' mode is respected
|
||||
|
||||
$ rm -rf _build/default
|
||||
$ dune build --config-file config target
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
3
|
||||
|
||||
Now repeat the tests with the old configuration format but 3.0 language
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-transport direct)
|
||||
> (cache-duplication copy)
|
||||
> (cache-trim-period 1h)
|
||||
> (cache-trim-size 1GB)
|
||||
> EOF
|
||||
|
||||
Build fails because 'cache-transport' was deleted
|
||||
|
||||
$ dune build --config-file config target
|
||||
File "$TESTCASE_ROOT/config", line 3, characters 0-24:
|
||||
3 | (cache-transport direct)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Error: 'cache-transport' was deleted in version 3.0 of the dune language.
|
||||
Dune cache now uses only the direct transport mode.
|
||||
[1]
|
||||
|
||||
So, we comply and delete it
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-duplication copy)
|
||||
> (cache-trim-period 1h)
|
||||
> (cache-trim-size 1GB)
|
||||
> EOF
|
||||
|
||||
Build fails because 'cache-duplication' was renamed
|
||||
|
||||
$ dune build --config-file config target
|
||||
File "$TESTCASE_ROOT/config", line 3, characters 0-24:
|
||||
3 | (cache-duplication copy)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Error: 'cache-duplication' was renamed to 'cache-storage-mode' in the 3.0
|
||||
version of the dune language
|
||||
[1]
|
||||
|
||||
So, we comply and rename it.
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-storage-mode copy)
|
||||
> (cache-trim-period 1h)
|
||||
> (cache-trim-size 1GB)
|
||||
> EOF
|
||||
|
||||
Build fails because 'cache-trim-period' was deleted
|
||||
|
||||
$ dune build --config-file config target
|
||||
File "$TESTCASE_ROOT/config", line 4, characters 0-22:
|
||||
4 | (cache-trim-period 1h)
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
Error: 'cache-trim-period' was deleted in version 3.0 of the dune language.
|
||||
To trim the cache, use the 'dune cache trim' command.
|
||||
[1]
|
||||
|
||||
So, we comply and delete it
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-storage-mode copy)
|
||||
> (cache-trim-size 1GB)
|
||||
> EOF
|
||||
|
||||
Build fails because 'cache-trim-size' was deleted
|
||||
|
||||
$ dune build --config-file config target
|
||||
File "$TESTCASE_ROOT/config", line 4, characters 0-21:
|
||||
4 | (cache-trim-size 1GB)
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
Error: 'cache-trim-size' was deleted in version 3.0 of the dune language. To
|
||||
trim the cache, use the 'dune cache trim' command.
|
||||
[1]
|
||||
|
||||
So, we comply and delete it
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-storage-mode copy)
|
||||
> EOF
|
||||
|
||||
Build succeeds and the 'copy' mode is respected
|
||||
|
||||
$ rm -rf _build/default
|
||||
$ dune build --config-file config target
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
1
|
||||
|
||||
Switch to the 'hardlink' mode now
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-storage-mode hardlink)
|
||||
> EOF
|
||||
|
||||
Build succeeds and the 'hardlink' mode is respected
|
||||
|
||||
$ rm -rf _build/default
|
||||
$ dune build --config-file config target
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
3
|
||||
|
||||
Let's disable the cache
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache disabled)
|
||||
> (cache-storage-mode hardlink)
|
||||
> EOF
|
||||
|
||||
Test that in this mode the shared cache directory is not created
|
||||
|
||||
$ rm -rf $DUNE_CACHE_ROOT
|
||||
$ rm -rf _build/default
|
||||
$ dune build --config-file config target
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
1
|
||||
$ dune_cmd exists $DUNE_CACHE_ROOT
|
||||
false
|
||||
|
||||
Test that the cache can be enabled via the environment variable
|
||||
|
||||
$ rm -rf _build/default
|
||||
$ DUNE_CACHE=enabled dune build --config-file config target
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
3
|
||||
$ dune_cmd exists $DUNE_CACHE_ROOT
|
||||
true
|
||||
|
||||
Test that we can override the environment variable from the command line
|
||||
|
||||
$ rm -rf $DUNE_CACHE_ROOT
|
||||
$ rm -rf _build/default
|
||||
$ DUNE_CACHE=enabled dune build --config-file config target --cache=disabled
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
1
|
||||
$ dune_cmd exists $DUNE_CACHE_ROOT
|
||||
false
|
||||
|
||||
Test that we can override the storage mode via the environment variable
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-storage-mode hardlink)
|
||||
> EOF
|
||||
|
||||
$ rm -rf _build/default
|
||||
$ DUNE_CACHE_STORAGE_MODE=copy dune build --config-file config target
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
1
|
||||
|
||||
Test that we can override the environment variable from the command line
|
||||
|
||||
$ rm -rf $DUNE_CACHE_ROOT
|
||||
$ rm -rf _build/default
|
||||
$ DUNE_CACHE_STORAGE_MODE=copy dune build --config-file config target --cache-storage-mode=hardlink
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
3
|
||||
|
|
@ -0,0 +1,63 @@
|
|||
Test deduplication of build artifacts when using Dune cache with hard links.
|
||||
|
||||
$ export DUNE_CACHE=enabled
|
||||
$ export DUNE_CACHE_ROOT=$PWD/.cache
|
||||
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 2.1)
|
||||
> EOF
|
||||
$ cat > dune <<EOF
|
||||
> (rule
|
||||
> (deps source)
|
||||
> (targets target)
|
||||
> (action (copy source target)))
|
||||
> EOF
|
||||
$ cat > source <<EOF
|
||||
> \_o< COIN
|
||||
> EOF
|
||||
|
||||
Here we build [target], which is a copy of [source]. After the build, the same
|
||||
file will appear in the build directory twice: (i) as [_build/default/source],
|
||||
because all sources are first copied to the build directory, and (ii) as
|
||||
[_build/default/target], as the build result. Furthermore, the same file will
|
||||
also be stored to the build cache, somewhere in the [files/v4] directory. Dune
|
||||
cache will recognise that all three files are identical and will use hard links
|
||||
to share them on disk, hence the hard link counts of 3.
|
||||
|
||||
$ dune build target
|
||||
$ cat _build/default/target
|
||||
\_o< COIN
|
||||
$ dune_cmd stat hardlinks _build/default/source
|
||||
3
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
3
|
||||
|
||||
Demonstrate that sharing of build artifacts can be abused to corrupt the cache.
|
||||
|
||||
$ chmod 777 _build/default/target
|
||||
$ echo "\_o< MEOW" > _build/default/target
|
||||
$ rm -rf _build
|
||||
$ dune build target
|
||||
$ cat _build/default/target
|
||||
\_o< MEOW
|
||||
|
||||
Test that by using the [copy] mode we can disable the sharing.
|
||||
|
||||
$ rm -rf $DUNE_CACHE_ROOT
|
||||
$ rm -rf _build
|
||||
$ dune build target --cache-storage-mode=copy
|
||||
$ cat _build/default/target
|
||||
\_o< COIN
|
||||
$ dune_cmd stat hardlinks _build/default/source
|
||||
1
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
1
|
||||
|
||||
In the [copy] mode, we can't corrupt the cache so easily.
|
||||
|
||||
$ chmod 777 _build/default/target
|
||||
$ echo "\_o< MEOW" > _build/default/target
|
||||
$ rm -rf _build
|
||||
$ dune build target
|
||||
$ cat _build/default/target
|
||||
\_o< COIN
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
The dune cache should be enabled by default
|
||||
|
||||
$ echo "(lang dune 3.17)" > dune-project
|
||||
|
||||
$ cat > dune << EOF
|
||||
> (library
|
||||
> (name foo))
|
||||
> EOF
|
||||
|
||||
$ cat > foo.ml << EOF
|
||||
> let f x y = x + y
|
||||
> EOF
|
||||
|
||||
Set up cache directory
|
||||
|
||||
$ export DUNE_CACHE_ROOT=$(pwd)/dune_test_cache
|
||||
$ mkdir $DUNE_CACHE_ROOT
|
||||
$ DUNE_CACHE=disabled dune build
|
||||
$ ls $DUNE_CACHE_ROOT
|
||||
|
||||
We have not written anything to the cache yet.
|
||||
|
||||
Change source files to force a recompilation
|
||||
|
||||
$ cat > foo.ml << EOF
|
||||
> let f x y = x - y
|
||||
> EOF
|
||||
$ dune build
|
||||
$ ls $DUNE_CACHE_ROOT | sort
|
||||
files
|
||||
meta
|
||||
temp
|
||||
values
|
||||
|
||||
Cache has been written to!
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
Check the cache restores empty directories
|
||||
|
||||
$ export DUNE_CACHE=enabled
|
||||
$ export DUNE_CACHE_ROOT=$PWD/dune-cache
|
||||
$ cat >dune-project <<EOF
|
||||
> (lang dune 3.10)
|
||||
> (using directory-targets 0.1)
|
||||
> EOF
|
||||
|
||||
$ cat >dune <<EOF
|
||||
> (rule
|
||||
> (target (dir output))
|
||||
> (action
|
||||
> (progn
|
||||
> (run mkdir output)
|
||||
> (run mkdir output/child)
|
||||
> (run touch output/file))))
|
||||
> EOF
|
||||
|
||||
Build an empty directory.
|
||||
|
||||
$ dune build output
|
||||
$ find _build/default/output | sort
|
||||
_build/default/output
|
||||
_build/default/output/child
|
||||
_build/default/output/file
|
||||
|
||||
Restore it from cache.
|
||||
|
||||
$ rm -rf _build
|
||||
$ dune build output
|
||||
$ find _build/default/output | sort
|
||||
_build/default/output
|
||||
_build/default/output/child
|
||||
_build/default/output/file
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
Check that Dune cache can cope with missing file/metadata entries.
|
||||
|
||||
$ export DUNE_CACHE_ROOT=$PWD/.cache
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 2.1)
|
||||
> (sandboxing_preference none)
|
||||
> (cache enabled)
|
||||
> (cache-duplication copy)
|
||||
> (cache-transport direct)
|
||||
> EOF
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 2.1)
|
||||
> EOF
|
||||
$ cat > dune <<EOF
|
||||
> (rule
|
||||
> (deps source)
|
||||
> (targets target)
|
||||
> (action (bash "cat source source > target")))
|
||||
> (rule
|
||||
> (deps source)
|
||||
> (targets twin-a twin-b)
|
||||
> (action (bash "echo twin-a-contents >> twin-a; echo twin-b-contents >> twin-b")))
|
||||
> EOF
|
||||
$ cat > source <<EOF
|
||||
> \_o< COIN
|
||||
> EOF
|
||||
|
||||
$ dune build --config-file=config target twin-a twin-b
|
||||
$ cat _build/default/target
|
||||
\_o< COIN
|
||||
\_o< COIN
|
||||
$ cat _build/default/twin-a _build/default/twin-b
|
||||
twin-a-contents
|
||||
twin-b-contents
|
||||
|
||||
Delete the [files/v4] storage and test that Dune can cope with this.
|
||||
|
||||
$ rm -rf _build "$DUNE_CACHE_ROOT"/files/v4/
|
||||
$ dune build --config-file=config target
|
||||
$ cat _build/default/target
|
||||
\_o< COIN
|
||||
\_o< COIN
|
||||
|
||||
Same but for the [meta/v5] storage.
|
||||
|
||||
$ rm -rf _build "$DUNE_CACHE_ROOT"/meta/v5/
|
||||
$ dune build --config-file=config target
|
||||
$ cat _build/default/target
|
||||
\_o< COIN
|
||||
\_o< COIN
|
||||
|
||||
Now nuke the whole cache directory.
|
||||
|
||||
$ rm -rf "$DUNE_CACHE_ROOT"
|
||||
$ rm -rf _build
|
||||
$ dune build --config-file=config target twin-a twin-b
|
||||
$ cat _build/default/target
|
||||
\_o< COIN
|
||||
\_o< COIN
|
||||
|
||||
Selectively delete just one of the set of targets.
|
||||
|
||||
$ twin_b_entry=$(dune_cmd find-file-by-contents-regexp "$DUNE_CACHE_ROOT" "twin-b-contents")
|
||||
$ rm "$twin_b_entry"
|
||||
$ rm -r _build
|
||||
|
||||
$ dune build --config-file=config twin-a
|
||||
$ cat _build/default/twin-a
|
||||
twin-a-contents
|
||||
$ cat _build/default/twin-b
|
||||
twin-b-contents
|
||||
|
|
@ -0,0 +1,132 @@
|
|||
Test basic cache store/restore functionality in the [copy] mode.
|
||||
|
||||
Dune supports setting the cache directory in two ways, via the [XDG_CACHE_HOME]
|
||||
variable, and via the [DUNE_CACHE_ROOT] variable. Here we test the former.
|
||||
|
||||
$ export XDG_RUNTIME_DIR=$PWD/.xdg-runtime
|
||||
$ export XDG_CACHE_HOME=$PWD/.xdg-cache
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-storage-mode copy)
|
||||
> EOF
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 3.5)
|
||||
> EOF
|
||||
$ cat > dune <<EOF
|
||||
> (rule
|
||||
> (deps source)
|
||||
> (targets target1 target2)
|
||||
> (action
|
||||
> (progn
|
||||
> (no-infer (with-stdout-to beacon (echo "")))
|
||||
> (with-stdout-to target1 (cat source))
|
||||
> (with-stdout-to target2 (cat source source)))))
|
||||
> EOF
|
||||
|
||||
It's a duck. It quacks. (Yes, the author of this comment didn't get it.)
|
||||
|
||||
$ cat > source <<EOF
|
||||
> \_o< COIN
|
||||
> EOF
|
||||
|
||||
Test that after the build, the files in the build directory have the hard link
|
||||
count of 1, because they are not shared with the corresponding cache entries.
|
||||
|
||||
We expect to see both workspace-local and shared cache misses, because we've
|
||||
never built [target1] before.
|
||||
|
||||
$ dune build --config-file=config target1 --debug-cache=shared,workspace-local \
|
||||
> 2>&1 | grep '_build/default/source\|_build/default/target'
|
||||
Workspace-local cache miss: _build/default/source: never seen this target before
|
||||
Shared cache miss [2eb3625df15d127fd24d71cb12a0c110] (_build/default/source): not found in cache
|
||||
Workspace-local cache miss: _build/default/target1: never seen this target before
|
||||
Shared cache miss [5bd79a21d9a54fa9f8d991ad45109d1e] (_build/default/target1): not found in cache
|
||||
|
||||
$ dune_cmd stat hardlinks _build/default/source
|
||||
1
|
||||
$ dune_cmd stat hardlinks _build/default/target1
|
||||
1
|
||||
$ dune_cmd stat hardlinks _build/default/target2
|
||||
1
|
||||
$ dune_cmd exists _build/default/beacon
|
||||
true
|
||||
|
||||
Test that rebuilding works.
|
||||
|
||||
Now we expect to see only workspace-local cache misses, because we've cleaned
|
||||
[_build/default] but not the shared cache.
|
||||
|
||||
$ rm -rf _build/default
|
||||
$ dune build --config-file=config target1 --debug-cache=shared,workspace-local \
|
||||
> 2>&1 | grep '_build/default/source\|_build/default/target'
|
||||
Workspace-local cache miss: _build/default/source: target missing from build dir
|
||||
Workspace-local cache miss: _build/default/target1: target missing from build dir
|
||||
|
||||
$ dune_cmd stat hardlinks _build/default/source
|
||||
1
|
||||
$ dune_cmd stat hardlinks _build/default/target1
|
||||
1
|
||||
$ dune_cmd stat hardlinks _build/default/target2
|
||||
1
|
||||
$ dune_cmd exists _build/default/beacon
|
||||
false
|
||||
$ cat _build/default/source
|
||||
\_o< COIN
|
||||
$ cat _build/default/target1
|
||||
\_o< COIN
|
||||
$ cat _build/default/target2
|
||||
\_o< COIN
|
||||
\_o< COIN
|
||||
|
||||
Test how zero the zero build is. We do not expect to see any cache misses.
|
||||
|
||||
$ dune build --config-file=config target1 --debug-cache=shared,workspace-local \
|
||||
> 2>&1 | grep '_build/default/source\|_build/default/target'
|
||||
[1]
|
||||
|
||||
Test that the cache stores all historical build results.
|
||||
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 2.1)
|
||||
> EOF
|
||||
$ cat > dune-v1 <<EOF
|
||||
> (rule
|
||||
> (targets t1)
|
||||
> (action (bash "echo running; echo v1 > t1")))
|
||||
> (rule
|
||||
> (deps t1)
|
||||
> (targets t2)
|
||||
> (action (bash "echo running; cat t1 t1 > t2")))
|
||||
> EOF
|
||||
$ cat > dune-v2 <<EOF
|
||||
> (rule
|
||||
> (targets t1)
|
||||
> (action (bash "echo running; echo v2 > t1")))
|
||||
> (rule
|
||||
> (deps t1)
|
||||
> (targets t2)
|
||||
> (action (bash "echo running; cat t1 t1 > t2")))
|
||||
> EOF
|
||||
$ cp dune-v1 dune
|
||||
$ dune build --config-file=config t2
|
||||
running
|
||||
running
|
||||
$ cat _build/default/t2
|
||||
v1
|
||||
v1
|
||||
$ cp dune-v2 dune
|
||||
$ dune build --config-file=config t2
|
||||
running
|
||||
running
|
||||
$ cat _build/default/t2
|
||||
v2
|
||||
v2
|
||||
$ cp dune-v1 dune
|
||||
$ dune build --config-file=config t2
|
||||
$ cat _build/default/t1
|
||||
v1
|
||||
$ cat _build/default/t2
|
||||
v1
|
||||
v1
|
||||
|
|
@ -0,0 +1,127 @@
|
|||
Test basic cache store/restore functionality in the default [hardlink] mode.
|
||||
|
||||
Dune supports setting the cache directory in two ways, via the [XDG_CACHE_HOME]
|
||||
variable, and via the [DUNE_CACHE_ROOT] variable. Here we test the former.
|
||||
|
||||
$ export XDG_RUNTIME_DIR=$PWD/.xdg-runtime
|
||||
$ export XDG_CACHE_HOME=$PWD/.xdg-cache
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 2.1)
|
||||
> (cache enabled)
|
||||
> EOF
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 2.1)
|
||||
> EOF
|
||||
$ cat > dune <<EOF
|
||||
> (rule
|
||||
> (deps source)
|
||||
> (targets target1 target2)
|
||||
> (action (bash "touch beacon; cat source > target1; cat source source > target2")))
|
||||
> EOF
|
||||
|
||||
It's a duck. It quacks. (Yes, the author of this comment didn't get it.)
|
||||
|
||||
$ cat > source <<EOF
|
||||
> \_o< COIN
|
||||
> EOF
|
||||
|
||||
Test that after the build, the files in the build directory have the hard link
|
||||
counts greater than 1, because they are shared with the corresponding cache entries.
|
||||
|
||||
We expect to see both workspace-local and shared cache misses, because we've
|
||||
never built [target1] before.
|
||||
|
||||
$ dune build --config-file=config target1 --debug-cache=shared,workspace-local \
|
||||
> 2>&1 | grep '_build/default/source\|_build/default/target'
|
||||
Workspace-local cache miss: _build/default/source: never seen this target before
|
||||
Shared cache miss [055e127f555b3fafe95f560290795689] (_build/default/source): not found in cache
|
||||
Workspace-local cache miss: _build/default/target1: never seen this target before
|
||||
Shared cache miss [03f7dfdb12ee66311712e0aa5c65de08] (_build/default/target1): not found in cache
|
||||
|
||||
$ dune_cmd stat hardlinks _build/default/source
|
||||
3
|
||||
$ dune_cmd stat hardlinks _build/default/target1
|
||||
3
|
||||
$ dune_cmd stat hardlinks _build/default/target2
|
||||
2
|
||||
$ dune_cmd exists _build/default/beacon
|
||||
true
|
||||
|
||||
Test that rebuilding works.
|
||||
|
||||
Now we expect to see only workspace-local cache misses, because we've cleaned
|
||||
[_build/default] but not the shared cache.
|
||||
|
||||
$ rm -rf _build/default
|
||||
$ dune build --config-file=config target1 --debug-cache=shared,workspace-local \
|
||||
> 2>&1 | grep '_build/default/source\|_build/default/target'
|
||||
Workspace-local cache miss: _build/default/source: target missing from build dir
|
||||
Workspace-local cache miss: _build/default/target1: target missing from build dir
|
||||
|
||||
$ dune_cmd stat hardlinks _build/default/source
|
||||
3
|
||||
$ dune_cmd stat hardlinks _build/default/target1
|
||||
3
|
||||
$ dune_cmd stat hardlinks _build/default/target2
|
||||
2
|
||||
$ dune_cmd exists _build/default/beacon
|
||||
false
|
||||
$ cat _build/default/source
|
||||
\_o< COIN
|
||||
$ cat _build/default/target1
|
||||
\_o< COIN
|
||||
$ cat _build/default/target2
|
||||
\_o< COIN
|
||||
\_o< COIN
|
||||
|
||||
Test how zero the zero build is. We do not expect to see any cache misses.
|
||||
|
||||
$ dune build --config-file=config target1 --debug-cache=shared,workspace-local \
|
||||
> 2>&1 | grep '_build/default/source\|_build/default/target'
|
||||
[1]
|
||||
|
||||
Test that the cache stores all historical build results.
|
||||
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 2.1)
|
||||
> EOF
|
||||
$ cat > dune-v1 <<EOF
|
||||
> (rule
|
||||
> (targets t1)
|
||||
> (action (bash "echo running; echo v1 > t1")))
|
||||
> (rule
|
||||
> (deps t1)
|
||||
> (targets t2)
|
||||
> (action (bash "echo running; cat t1 t1 > t2")))
|
||||
> EOF
|
||||
$ cat > dune-v2 <<EOF
|
||||
> (rule
|
||||
> (targets t1)
|
||||
> (action (bash "echo running; echo v2 > t1")))
|
||||
> (rule
|
||||
> (deps t1)
|
||||
> (targets t2)
|
||||
> (action (bash "echo running; cat t1 t1 > t2")))
|
||||
> EOF
|
||||
$ cp dune-v1 dune
|
||||
$ dune build --config-file=config t2
|
||||
running
|
||||
running
|
||||
$ cat _build/default/t2
|
||||
v1
|
||||
v1
|
||||
$ cp dune-v2 dune
|
||||
$ dune build --config-file=config t2
|
||||
running
|
||||
running
|
||||
$ cat _build/default/t2
|
||||
v2
|
||||
v2
|
||||
$ cp dune-v1 dune
|
||||
$ dune build --config-file=config t2
|
||||
$ cat _build/default/t1
|
||||
v1
|
||||
$ cat _build/default/t2
|
||||
v1
|
||||
v1
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
Test that files promoted to the source tree are writable by the user
|
||||
|
||||
Reproduction case for #3026
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 2.1)
|
||||
> (cache enabled)
|
||||
> EOF
|
||||
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 2.1)
|
||||
> EOF
|
||||
|
||||
$ cat > dune <<EOF
|
||||
> (rule
|
||||
> (mode promote)
|
||||
> (action (with-stdout-to file (echo "Hello, world!\n"))))
|
||||
> EOF
|
||||
|
||||
Run Dune a first time to fill the cache, then delete the promoted file and run
|
||||
Dune again. At the end of the first run, the file [_build/default/file] should
|
||||
get deduplicated and so become read-only. As a result, on the second run the
|
||||
promoted file will be copied from a read-only file. However, we still want the
|
||||
user to be able to edit this file given that it is in the source tree, so Dune
|
||||
should change the permission of this file.
|
||||
|
||||
We check that Dune does change the permission by echoing something into the file
|
||||
after the second run.
|
||||
|
||||
$ env XDG_RUNTIME_DIR=$PWD/.xdg-runtime \
|
||||
> XDG_CACHE_HOME=$PWD/.xdg-cache \
|
||||
> dune build --config-file=config file
|
||||
|
||||
$ rm -f file
|
||||
|
||||
$ env XDG_RUNTIME_DIR=$PWD/.xdg-runtime \
|
||||
> XDG_CACHE_HOME=$PWD/.xdg-cache \
|
||||
> dune build --config-file=config file
|
||||
|
||||
$ cat file
|
||||
Hello, world!
|
||||
|
||||
$ echo plop > file
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
The cache can't be written if the location to where it is supposed to be
|
||||
written can't be written to.
|
||||
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 3.17)
|
||||
> EOF
|
||||
$ cat >dune <<EOF
|
||||
> (rule (with-stdout-to foo (progn)))
|
||||
> EOF
|
||||
|
||||
Create a directory in which we can't write to and use this as the location
|
||||
where Dune is supposed to store the cache:
|
||||
|
||||
$ mkdir readonly
|
||||
$ chmod a-w readonly
|
||||
$ export DUNE_CACHE=enabled
|
||||
$ export DUNE_CACHE_ROOT=$(pwd)/readonly/cache-dir
|
||||
|
||||
$ dune build
|
||||
Warning: Cache directories could not be created: Permission denied; disabling
|
||||
cache
|
||||
Hint: Make sure the directory
|
||||
$TESTCASE_ROOT/readonly/cache-dir/temp
|
||||
can be created
|
||||
|
||||
Likewise, this should also happen if the location is set via XDG variables.
|
||||
|
||||
$ unset DUNE_CACHE_ROOT
|
||||
$ export XDG_CACHE_HOME=$(pwd)/readonly/xdg-cache-dir
|
||||
$ export DUNE_CONFIG__SKIP_LINE_BREAK=enabled
|
||||
|
||||
$ dune build 2>&1 | sed 's/created: .*;/created: $REASON:/'
|
||||
Warning: Cache directories could not be created: $REASON: disabling cache
|
||||
Hint: Make sure the directory $TESTCASE_ROOT/readonly/xdg-cache-dir/dune/db/temp can be created
|
||||
|
||||
$ export HOME=/homeless-shelter
|
||||
$ unset XDG_CACHE_HOME
|
||||
$ dune build 2>&1 | sed 's/created: .*;/created: $REASON:/'
|
||||
Warning: Cache directories could not be created: $REASON: disabling cache
|
||||
Hint: Make sure the directory /homeless-shelter/.cache/dune/db/temp can be created
|
||||
|
|
@ -0,0 +1,144 @@
|
|||
Test reproducibility check
|
||||
|
||||
$ export DUNE_CACHE_ROOT=$PWD/.cache
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> EOF
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 3.0)
|
||||
> EOF
|
||||
$ cat > dune <<EOF
|
||||
> (rule
|
||||
> (deps dep)
|
||||
> (targets reproducible)
|
||||
> (action (progn (echo "build reproducible\n")
|
||||
> (copy dep reproducible))))
|
||||
> (rule
|
||||
> (targets non-reproducible)
|
||||
> (action (no-infer (progn (echo "build non-reproducible\n")
|
||||
> (copy dep non-reproducible)))))
|
||||
> EOF
|
||||
|
||||
Both rules read [dep] but only the reproducible rule declares it as a dependency
|
||||
|
||||
$ echo old-content > dep
|
||||
|
||||
Build both, which will store the results to the cache
|
||||
|
||||
$ dune build --config-file config reproducible non-reproducible
|
||||
build reproducible
|
||||
build non-reproducible
|
||||
|
||||
Update the content and rebuild; only the reproducible rule will rerun
|
||||
|
||||
$ echo new-content > dep
|
||||
$ dune build --config-file config reproducible non-reproducible
|
||||
build reproducible
|
||||
|
||||
When 'cache-check-probability' is unset, we skip reproducibility check
|
||||
|
||||
$ rm -rf _build
|
||||
$ dune build --config-file config reproducible non-reproducible
|
||||
|
||||
Note that we didn't rerun the rules, since the results are in the cache
|
||||
|
||||
$ dune_cmd cat _build/default/reproducible
|
||||
new-content
|
||||
$ dune_cmd cat _build/default/non-reproducible
|
||||
old-content
|
||||
|
||||
Set 'cache-check-probability' to 0.0, which should also skip the check
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-check-probability 0.0)
|
||||
> EOF
|
||||
$ rm -rf _build
|
||||
$ dune build --config-file config reproducible non-reproducible
|
||||
|
||||
Set 'cache-check-probability' to 1.0, which should trigger the check
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-check-probability 1.0)
|
||||
> EOF
|
||||
$ rm -rf _build
|
||||
$ dune build --config-file config reproducible non-reproducible
|
||||
Warning: cache store error [78f2c7a1ca2fb3c739b6adc7c2310f3d]: ((in_cache
|
||||
((non-reproducible 7378fb2d7d80dc4468d6558d864f0897))) (computed
|
||||
((non-reproducible 074ebdc1c3853f27c68566d8d183032c)))) after executing
|
||||
(echo 'build non-reproducible';cp dep non-reproducible)
|
||||
build reproducible
|
||||
build non-reproducible
|
||||
|
||||
Check that the reported digests make sense
|
||||
|
||||
$ dune_cmd cat $DUNE_CACHE_ROOT/files/v4/73/7378fb2d7d80dc4468d6558d864f0897
|
||||
old-content
|
||||
$ dune_cmd cat $DUNE_CACHE_ROOT/files/v4/074/074ebdc1c3853f27c68566d8d183032c
|
||||
Fatal error: exception Unix.Unix_error(Unix.ENOENT, "open", "$TESTCASE_ROOT/.cache/files/v4/074/074ebdc1c3853f27c68566d8d183032c")
|
||||
[2]
|
||||
|
||||
Check that probability values less than zero and greater than one are rejected
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-check-probability -0.1)
|
||||
> EOF
|
||||
$ dune build --config-file config reproducible non-reproducible
|
||||
File "$TESTCASE_ROOT/config", line 3, characters 0-30:
|
||||
3 | (cache-check-probability -0.1)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Error: The reproducibility check probability must be in the range [0, 1].
|
||||
[1]
|
||||
|
||||
$ cat > config <<EOF
|
||||
> (lang dune 3.0)
|
||||
> (cache enabled)
|
||||
> (cache-check-probability 3.14)
|
||||
> EOF
|
||||
$ dune build --config-file config reproducible non-reproducible
|
||||
File "$TESTCASE_ROOT/config", line 3, characters 0-30:
|
||||
3 | (cache-check-probability 3.14)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Error: The reproducibility check probability must be in the range [0, 1].
|
||||
[1]
|
||||
|
||||
Test that the environment variable and the command line flag work too
|
||||
|
||||
$ rm -rf _build
|
||||
$ DUNE_CACHE_CHECK_PROBABILITY=-1 dune build --cache=enabled reproducible non-reproducible
|
||||
Error: The reproducibility check probability must be in the range [0, 1].
|
||||
[1]
|
||||
|
||||
$ rm -rf _build
|
||||
$ DUNE_CACHE_CHECK_PROBABILITY=0.0 dune build --cache=enabled reproducible non-reproducible
|
||||
|
||||
$ rm -rf _build
|
||||
$ DUNE_CACHE_CHECK_PROBABILITY=1.0 dune build --cache=enabled reproducible non-reproducible
|
||||
Warning: cache store error [78f2c7a1ca2fb3c739b6adc7c2310f3d]: ((in_cache
|
||||
((non-reproducible 7378fb2d7d80dc4468d6558d864f0897))) (computed
|
||||
((non-reproducible 074ebdc1c3853f27c68566d8d183032c)))) after executing
|
||||
(echo 'build non-reproducible';cp dep non-reproducible)
|
||||
build reproducible
|
||||
build non-reproducible
|
||||
|
||||
$ rm -rf _build
|
||||
$ DUNE_CACHE_CHECK_PROBABILITY=1.0 dune build --cache-check-probability=0.0 --cache=enabled reproducible non-reproducible
|
||||
|
||||
$ rm -rf _build
|
||||
$ dune build --cache=enabled --cache-check-probability=1.0 reproducible non-reproducible
|
||||
Warning: cache store error [78f2c7a1ca2fb3c739b6adc7c2310f3d]: ((in_cache
|
||||
((non-reproducible 7378fb2d7d80dc4468d6558d864f0897))) (computed
|
||||
((non-reproducible 074ebdc1c3853f27c68566d8d183032c)))) after executing
|
||||
(echo 'build non-reproducible';cp dep non-reproducible)
|
||||
build reproducible
|
||||
build non-reproducible
|
||||
|
||||
$ dune build --cache=enabled --cache-check-probability=8 reproducible non-reproducible
|
||||
Error: The reproducibility check probability must be in the range [0, 1].
|
||||
[1]
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
This test checks that the `dune cache size` command returns the correct size of
|
||||
the cache.
|
||||
|
||||
$ export DUNE_CACHE=enabled
|
||||
$ export DUNE_CACHE_ROOT=$PWD/.cache
|
||||
|
||||
$ cat > config << EOF
|
||||
> (lang dune 3.7)
|
||||
> (cache enabled)
|
||||
> (cache-storage-mode copy)
|
||||
> EOF
|
||||
|
||||
$ cat > dune-project << EOF
|
||||
> (lang dune 3.7)
|
||||
> EOF
|
||||
|
||||
$ cat > dune << EOF
|
||||
> (rule
|
||||
> (targets target_a)
|
||||
> (action
|
||||
> (with-outputs-to
|
||||
> target_a
|
||||
> (echo Hello World!))))
|
||||
> EOF
|
||||
|
||||
We build a simple file with the contents of "Hello World!".
|
||||
|
||||
$ dune build target_a --display=short
|
||||
|
||||
Now we remove it so that we are checking the size of the file rather than the
|
||||
link Dune created.
|
||||
|
||||
$ rm _build/default/target_a
|
||||
|
||||
The size command reports the size of the cache in bytes in human-readable form.
|
||||
It correctly reports 12 bytes.
|
||||
|
||||
$ dune cache size
|
||||
12B
|
||||
|
||||
We also have a machine-readable version of the command which reports the size of
|
||||
the cache in bytes directly without any units.
|
||||
|
||||
$ dune cache size --machine-readable
|
||||
12
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
Check that: (i) we refuse to cache symbolic links, and (ii) rules that use the
|
||||
produced symbolic links work correctly and are appropriately cached.
|
||||
|
||||
$ export DUNE_CACHE=enabled
|
||||
$ export DUNE_CACHE_ROOT=$PWD/.cache
|
||||
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 2.1)
|
||||
> EOF
|
||||
$ cat > dune <<EOF
|
||||
> (rule
|
||||
> (deps source)
|
||||
> (targets link)
|
||||
> (action (bash "ln -s source link")))
|
||||
> (rule
|
||||
> (deps link)
|
||||
> (targets target)
|
||||
> (action (bash "cat link link > target")))
|
||||
> EOF
|
||||
$ cat > source <<EOF
|
||||
> \_o< COIN
|
||||
> EOF
|
||||
$ dune build target
|
||||
|
||||
Dune cache contains entries for [source] and [target] but not for [link]
|
||||
|
||||
$ (cd "$DUNE_CACHE_ROOT/meta/v5"; grep -rs . -e 'source' | dune_cmd count-lines)
|
||||
1
|
||||
$ (cd "$DUNE_CACHE_ROOT/meta/v5"; grep -rs . -e 'target' | dune_cmd count-lines)
|
||||
1
|
||||
$ (cd "$DUNE_CACHE_ROOT/meta/v5"; grep -rs . -e 'link' | dune_cmd count-lines)
|
||||
0
|
||||
|
||||
The files in the build directory are shared with the cache entries
|
||||
|
||||
$ dune_cmd stat hardlinks _build/default/source
|
||||
2
|
||||
$ dune_cmd stat hardlinks _build/default/target
|
||||
2
|
||||
|
|
@ -0,0 +1,197 @@
|
|||
$ export DUNE_CACHE=enabled
|
||||
$ export XDG_RUNTIME_DIR=$PWD/.xdg-runtime
|
||||
$ export XDG_CACHE_HOME=$PWD/.xdg-cache
|
||||
|
||||
$ cat > dune-project <<EOF
|
||||
> (lang dune 2.1)
|
||||
> EOF
|
||||
$ cat > dune <<EOF
|
||||
> (rule
|
||||
> (targets target_a)
|
||||
> (action (bash "touch beacon_a; echo target_a > target_a")))
|
||||
> (rule
|
||||
> (targets target_b)
|
||||
> (action (bash "touch beacon_b; echo target_b > target_b")))
|
||||
> (rule
|
||||
> (targets non-exe)
|
||||
> (action (bash "echo content > non-exe")))
|
||||
> (rule
|
||||
> (targets exe)
|
||||
> (action (bash "echo content > exe; chmod +x exe")))
|
||||
> (rule
|
||||
> (targets multi_a multi_b)
|
||||
> (action (bash "touch beacon_multi; echo multi_a > multi_a; echo multi_b > multi_b")))
|
||||
> EOF
|
||||
|
||||
Function to reset build tree and cache.
|
||||
|
||||
$ reset ()
|
||||
> {
|
||||
> rm -rf _build
|
||||
> rm -rf $XDG_CACHE_HOME/dune
|
||||
> }
|
||||
|
||||
Check that trimming does not crash when the cache directory does not exist.
|
||||
|
||||
$ dune cache trim --size 0B
|
||||
Freed 0B (0 files removed)
|
||||
|
||||
Check that the digest scheme for executable and non-executable digests hasn't
|
||||
changed. If it has, make sure to increment the version of the cache. Note that
|
||||
the current digests for both files match those computed by Jenga.
|
||||
|
||||
$ dune build exe non-exe
|
||||
|
||||
$ (cd "$PWD/.xdg-cache/dune/db/files/v4"; grep -rws . -e 'content' | sort)
|
||||
./5e/5ef8de0800908475d94747fef67940ed:content
|
||||
./e2/e293405959cb01f8de0d237b687714c8:content
|
||||
|
||||
Move all current entries to v3 and v4 to test trimming of old versions of cache.
|
||||
|
||||
$ mkdir "$PWD/.xdg-cache/dune/db/files/v3"
|
||||
$ mkdir "$PWD/.xdg-cache/dune/db/meta/v3"
|
||||
$ mkdir "$PWD/.xdg-cache/dune/db/meta/v4"
|
||||
$ mv "$PWD/.xdg-cache/dune/db/files/v4"/* "$PWD/.xdg-cache/dune/db/files/v3"
|
||||
$ cp -r "$PWD/.xdg-cache/dune/db/meta/v5"/* "$PWD/.xdg-cache/dune/db/meta/v4"
|
||||
$ mv "$PWD/.xdg-cache/dune/db/meta/v5"/* "$PWD/.xdg-cache/dune/db/meta/v3"
|
||||
|
||||
Build some more targets.
|
||||
|
||||
$ dune build target_a target_b
|
||||
|
||||
Dune stores the result of rule execution in a store keyed by "rule digests". If
|
||||
the way such rule digests are computed changes, we could end up in a situation
|
||||
where the same hash means something different before and after the change, which
|
||||
is bad. To reduce the risk, we inject a version number into rule digests.
|
||||
|
||||
If you see the test below breaking, this means you changed the metadata format
|
||||
or the way that digests are computed and you should increment the corresponding
|
||||
version number. More specifically:
|
||||
|
||||
- If a digest value changed, you should increment the [rule_digest_version]
|
||||
value in [build_system.ml].
|
||||
|
||||
- If the metadata format changed, you should increment the metadata version in
|
||||
[layout.ml] in the [dune_cache_storage] library, e.g. from [meta/v5] to [meta/v6].
|
||||
You will also need to make sure that the cache trimmer treats new and old cache
|
||||
entries uniformly.
|
||||
|
||||
$ (cd "$PWD/.xdg-cache/dune/db/meta/v5"; grep -rws . -e 'metadata' | sort ) > out
|
||||
$ cat out
|
||||
./23/2342c7575f2061e6e3396d6dcf8f0d5f:((8:metadata)(5:files(8:target_a32:7b362c0c7d2035084c9fc2d0e6815be5)))
|
||||
./f5/f54153a11ae41a60aa095c4671180b84:((8:metadata)(5:files(8:target_b32:d5b73f7b5d75090e1da54099e4458db3)))
|
||||
|
||||
$ digest="$(awk -F: '/target_b/ { digest=$1 } END { print digest }' < out)"
|
||||
|
||||
$ dune_cmd stat size "$PWD/.xdg-cache/dune/db/meta/v5/$digest"
|
||||
70
|
||||
|
||||
Trimming the cache at this point should not remove any file entries because all
|
||||
of them are still hard-linked from the build directory. However, we should trim
|
||||
all metadata entries in [meta/v4] since they are broken: remember, we moved all
|
||||
[files/v4] to [files/v3].
|
||||
|
||||
$ find "$PWD/.xdg-cache/dune/db/meta/v4" -mindepth 2 -maxdepth 2 -type f | dune_cmd count-lines
|
||||
4
|
||||
$ dune cache trim --trimmed-size 1B
|
||||
Freed 287B (4 files removed)
|
||||
$ dune_cmd stat hardlinks _build/default/target_a
|
||||
2
|
||||
$ dune_cmd stat hardlinks _build/default/target_b
|
||||
2
|
||||
$ find "$PWD/.xdg-cache/dune/db/meta/v4" -mindepth 2 -maxdepth 2 -type f | dune_cmd count-lines
|
||||
0
|
||||
|
||||
If we unlink a file in the build tree, then the corresponding file entry will be
|
||||
trimmed.
|
||||
|
||||
$ rm -f _build/default/target_a _build/default/beacon_a _build/default/beacon_b
|
||||
$ dune cache trim --trimmed-size 1B
|
||||
Freed 79B (2 files removed)
|
||||
$ dune build target_a target_b
|
||||
$ dune_cmd stat hardlinks _build/default/target_a
|
||||
2
|
||||
$ dune_cmd stat hardlinks _build/default/target_b
|
||||
2
|
||||
$ dune_cmd exists _build/default/beacon_a
|
||||
true
|
||||
$ dune_cmd exists _build/default/beacon_b
|
||||
false
|
||||
|
||||
Now let's remove the remaining targets, left from the very first build and rerun
|
||||
the trimmer. That will delete unused [files/v3] and the corresponding metadata
|
||||
entries in [meta/v3].
|
||||
|
||||
$ rm -rf _build
|
||||
$ find "$PWD/.xdg-cache/dune/db/files/v3" -mindepth 2 -maxdepth 2 -type f | dune_cmd count-lines
|
||||
4
|
||||
$ find "$PWD/.xdg-cache/dune/db/meta/v3" -mindepth 2 -maxdepth 2 -type f | dune_cmd count-lines
|
||||
4
|
||||
|
||||
We hide the output for reproducibility: some files are executable and their
|
||||
sizes might vary on different platforms
|
||||
|
||||
$ dune cache trim --size 0B > /dev/null
|
||||
|
||||
$ find "$PWD/.xdg-cache/dune/db/files/v3" -mindepth 2 -maxdepth 2 -type f | dune_cmd count-lines
|
||||
0
|
||||
$ find "$PWD/.xdg-cache/dune/db/meta/v3" -mindepth 2 -maxdepth 2 -type f | dune_cmd count-lines
|
||||
0
|
||||
|
||||
The cache deletes oldest files first.
|
||||
|
||||
$ reset
|
||||
$ dune build target_a target_b
|
||||
|
||||
The [rm] commands below update the [ctime] of the corresponding cache entries.
|
||||
By deleting [target_b] first, we make its [ctime] older. The trimmer deletes
|
||||
older entries first, which is why [target_b] is trimmed while [target_a] is not.
|
||||
We know that [target_b] was trimmed, because it had to be rebuilt as indicated
|
||||
by the existence of [beacon_b].
|
||||
|
||||
$ rm -f _build/default/beacon_b _build/default/target_b
|
||||
$ dune_cmd wait-for-fs-clock-to-advance
|
||||
$ rm -f _build/default/beacon_a _build/default/target_a
|
||||
$ dune cache trim --trimmed-size 1B
|
||||
Freed 79B (2 files removed)
|
||||
$ dune build target_a target_b
|
||||
$ dune_cmd stat hardlinks _build/default/target_a
|
||||
2
|
||||
$ dune_cmd stat hardlinks _build/default/target_b
|
||||
2
|
||||
$ dune_cmd exists _build/default/beacon_a
|
||||
false
|
||||
$ dune_cmd exists _build/default/beacon_b
|
||||
true
|
||||
|
||||
Now let's redo the same test but delete the two targets in the opposite order,
|
||||
thus making the trimmer delete [target_a] instead of [target_b] as above.
|
||||
|
||||
$ reset
|
||||
$ dune build target_a target_b
|
||||
$ rm -f _build/default/beacon_a _build/default/target_a
|
||||
$ dune_cmd wait-for-fs-clock-to-advance
|
||||
$ rm -f _build/default/beacon_b _build/default/target_b
|
||||
$ dune cache trim --trimmed-size 1B
|
||||
Freed 79B (2 files removed)
|
||||
$ dune build target_a target_b
|
||||
$ dune_cmd stat hardlinks _build/default/target_a
|
||||
2
|
||||
$ dune_cmd stat hardlinks _build/default/target_b
|
||||
2
|
||||
$ dune_cmd exists _build/default/beacon_a
|
||||
true
|
||||
$ dune_cmd exists _build/default/beacon_b
|
||||
false
|
||||
|
||||
Test garbage collection: both [multi_a] and [multi_b] must be removed as they
|
||||
are part of the same rule.
|
||||
|
||||
$ reset
|
||||
$ dune build multi_a multi_b
|
||||
$ rm -f _build/default/multi_a _build/default/multi_b
|
||||
$ dune cache trim --trimmed-size 1B
|
||||
Freed 123B (2 files removed)
|
||||
|
||||
TODO: Test trimming priority in the [copy] mode. In PR #4497 we added a test but
|
||||
it turned out to be flaky so we subsequently deleted it in #4511.
|
||||
Loading…
Add table
Add a link
Reference in a new issue