497f5fc3f2
* Make verona testsuite directory dump passes. * Add test case for issue #651 * Extend test to cover related bug. |
||
|---|---|---|
| .. | ||
| verona/basic | ||
| verona_nostdlib | ||
| CMakeLists.txt | ||
| README.md | ||
| compare.cmake | ||
| run_command.cmake | ||
| verona.cmake | ||
| verona_nostdlib.cmake | ||
README.md
Verona testsuite
Test cases are divided in top-level directories, for each tool they are testing.
For each top-level directory there is a [toplevel].cmake, e.g. verona.cmake is the
settings for the verona directory, which is used to test verona. The cmake file
provides two things:
- A macro to provide the invoke the tool with appropriate command line arguments given the test file, and the output directory.
- A list of extensions for tests, e.g.
set(TEST_EXTENSION verona).
For each test, there is an associate golden output directory. For example, the test
testsuite/verona/basic/ex1.verona
has an associated output directory
testsuite/verona/basic/ex1.out/
This can contain any files that were generated by the command using the output directory.
Additionally, the harness can generate stdout.txt, stderr.txt and exit_code.txt for the
values of the output and error streams, and the exit code of the test. If the directory does
not exist CMake will emit a warning. Only files in the directory will be checked that the tool
generated the correct files.
Updating tests
The files that exist in the .out directories can be automatically updated. There is a build
target update-dump that will automatically update the contents of any .out directory. It
will only update the files that already exist in the directory, so if there is no stdout.txt
file, then after calling update-dump the file will still not exist. If you wish to add the file,
create the file, and then run update-dump.
There are also targets for individual testsuite directories, to just regenerate the tests in those
directories. For instance, verona-update-dump will just update the output in that directory.
Adding new tests
The update-dump target will also generate new output directories for any test that does not
currently have an output directory. The output directory will contain all the output files that
the test can generate. You might not wish to test all the phases with this test. Remove any files
you do not wish to test, and commit the rest.
To add a new tool
If you need to add a new tool, or a new invocation of a tool, then you create a new top-level
directory, and create the [directoryname].cmake file to find tests and generate
the command line.