This is a developer QoL improvement, where running codegen will skip
writing (and especially formatting) any files that were not changed.
**Why?** While code generation in itself was pretty much instant, QL
formatting of generated code was starting to take a long time. This made
unconditionally running codegen quite annoying, for example before each
test run as part of an IDE workflow or as part of the pre-commit hook.
**How?** This was not completely straightforward as we could not work
with the contents of the file prior to code generation as that was
already post-processed by the QL formatting, so we had no chance of
comparing the output of template rendering with that. We therefore store
the hashes of the files _prior_ to QL formatting in a checked-in file
(`swift/ql/.generated.list`). We can therefore load those hashes at
the beginning of code generation, use them to compare the template
rendering output and update them in this special registry file.
**What else?** We also extend this mechanism to detect accidental
modification of generated files in a more robust way. Before this patch,
we were doing it with a rough regexp based heuristic. Now, we just store
the hashes of the files _after_ QL formatting in the same checked file,
so we can check that and stop generation if a generated file was
modified, or a stub was modified without removing the `// generated`
header.
While PEP8 mandates those, they look bad in the schema file.
`autopep8` already ignores those, and they were single newlines at some
point until an overeager IDE has "fixed" them at some point without me
realizing.
Also, the pre-commit configuration was updated to take `schema.py` into
account.
By explicitly marking children in the `schema.yml` file, an internal
`getAChild` predicate is implemented, that is in turn used in `AstNode`
to implement `getParent`.
This is yet to be used in the control flow library to replace the
hand-rolled implementation.
A further, more complex step is to use the same information to fully
generate the core implementation of `PrintAst` (including the
accessor string). This will be done later.
The `parent` tests use the same swift code as the extractor tests, and
this is currently enforced by `sync-files.py`. Notice that `qltest.sh`
had to be modified to deal with multiple files, which was not working
yet.
This checks in the trapgen script generating trap entries in C++.
The codegen suite has been slightly reorganized, moving the templates
directory up one level and chopping everything into smaller bazel
packages. Running tests is now done via
```
bazel run //swift/codegen/test
```
With respect to the PoC, the nested `codeql::trap` namespace has been
dropped in favour of a `Trap` prefix (or suffix in case of entries)
within the `codeql` namespace. Also, generated C++ code is not checked
in in git any more, and generated during build. Finally, labels get
printed in hex in the trap file.
`TrapLabel` is for the moment only default-constructible, so only one
single label is possible. `TrapArena`, that is responsible for creating
disjoint labels will come in a later commit.
Tests can be run with
```
bazel test //swift/codegen:tests
```
Coverage can be checked installing `pytest-cov` and running
```
pytest --cov=swift/codegen swift/codegen/test
```
This enables use of the `pre-commit` framework to run quick pre-commit
checks. In particular this allows to automatically fix:
* trailing white spaces
* absence or multiple newlines at the end of files
* QL code formatting
* file sync
More could be added in the future: anything that can be checked fast
can be added in the configuration (for example well-formedness of
`qldoc` files).
This is a purely opt-in feature. Instructions for enabling it and
possibly configuring its behaviour are in `pre-commit-hook-setup.md`.
Paolo Tranquilli
Alex Denisov