55 lines
2.3 KiB
ReStructuredText
55 lines
2.3 KiB
ReStructuredText
How Preprocessing Works
|
|
=======================
|
|
|
|
Preprocessing consists in transforming source code before it is compiled. The
|
|
goal of this document is to explain how this works in Dune.
|
|
|
|
Dune supports two separate ways of applying preprocessors, the "classic pipeline" (used
|
|
with ``(staged_pps)``), and the "fast pipeline" (used for all other
|
|
:doc:`preprocessing specifications <../reference/preprocessing-spec>` including
|
|
``(pps)``).
|
|
|
|
The OCaml compilers provide options for specifying a preprocessing step. The
|
|
``-pp`` option is used to invoke a textual preprocessor (something that reads
|
|
text and returns text). The ``-ppx`` option is used to invoke a `ppx rewriter`
|
|
(a function that takes an AST and outputs an AST).
|
|
|
|
This is the "classic pipeline": preprocessing is part of the compilation
|
|
itself. This is simple, but has a problem: in order to compute the dependencies
|
|
of a module, it is necessary to pass the same ``-pp`` or ``-ppx`` option to
|
|
``ocamldep``.
|
|
|
|
The classic pipeline has the following steps:
|
|
|
|
- preprocessing (as part of ``ocamldep``)
|
|
- dependency analysis
|
|
- preprocessing (as part of compilation)
|
|
- compilation
|
|
|
|
Dune supports a "fast pipeline" where the preprocessor is invoked separately
|
|
from the compiler and its output is saved. Afterwards the preprocessed code is
|
|
compiled directly.
|
|
|
|
The fast pipeline has the following steps:
|
|
|
|
- preprocessing
|
|
- dependency analysis
|
|
- compilation
|
|
|
|
It has several advantages: it only invokes the preprocessor once per file, and
|
|
the preprocessed code is reused between dependency analysis and different kinds
|
|
of compilation. Also, when several preprocessors use ``ppxlib``, they can be
|
|
combined in a preprocessing program that traverses the AST only once.
|
|
|
|
However, some specific code generators or preprocessors require direct
|
|
access to the compilation artefacts of their dependencies. Therefore they
|
|
need to be used with the classic pipeline, even if it is slower. Note that a
|
|
PPX is able to know if it was called as part of ``ocamldep -ppx`` or ``ocamlopt
|
|
-ppx``, so it can act differently in each phase.
|
|
|
|
Dune chooses which pipeline to use depending on the
|
|
provided :doc:`../reference/preprocessing-spec`. It will select the fast pipeline,
|
|
unless ``(staged_pps)`` is used. In that case, the classic pipeline is used.
|
|
|
|
In the case of the fast pipeline, a single executable is built and accepts
|
|
arguments for all preprocessors.
|