Since we are launching a completely different process for the extension
tests than the process that is launched by VSCode, we need to add some
special handling for the debugging.
This will let the extension host/VSCode expose a debugging port, which
VSCode will then connect to. This is "less desirable than letting the
bootloader do its thing", but a packaged VSCode application does not
allow using the bootloader (`NODE_OPTIONS`=`--require=...`). Therefore,
we have to fallback to this option.
See: 47c60558ec/src/configuration.ts (L405-L411)
This converts all pure tests to Jest. This was done by first running
`npx jest-codemods` with the Mocha transformation, then manually fixing
any places where it hadn't automatically converted the correct thing
or had missed things (mostly Sinon).
This also sets up VSCode correctly for running Jest.
This will change the pure tests Mocha setup to actually use the
`tsconfig.json` located in the `test` directory. Before, it was using
the root-level `tsconfig.json`. To ensure we are still using mostly the
same settings, this will extend the `test/tsconfig.json` from the
root-level `tsconfig.json`.
This sets up Storybook for testing of React components. It adds stories
for some of the MRVA components. It does not add stories for the main
MRVA views since those are not independent of VSCode and need to be run
from within VSCode.
The main problem this commit fixes is with vscode 1.67.0, an error is
thrown when inside of integration tests and a dialog box is opened. We
were opening the telemetry dialog box. Now, an env variable is set
during cli-integration tests that prevents the dialog from being
opened.
There are also other cleanups and improvements with cli-integration
tests that assist with running locally:
- `vscode-test` dependency has been renamed to `@vscode/test-electron`,
so use that instead and make the small API changes to support it.
- Commit the codeql-pack.lock.yml file so it isn't recreated on each
test run.
- Ensure all databases are removed before _and after_ each test run
that manipulates the set of installed databases
- Similarly, for quick query files, delete them before and after each
test.
- Change some async `forEach` blocks to for loops in order to support
sequential operations more easily.
* Requires that QL_PATH environment variable is set and points to a
checkout of github/codeql
* Adds the `quiet` flag to the cli. When set, this flag will prevent
some modal dialogs from disrupting the flow. Currently, we only ensure
that the upgrades dialog is avoided.
* Update the main.yml workflow to checkout the codeql repo
This commit adds integration tests that run commands using the CLI. This
change introduces a number of enhancements in order to get there.
1. Augments the index-template.ts file so that it downloads an
appropriate cli version if requested.
2. Adds the ensureCli.ts that performs the download if a a suitable
version is not already installed. See the comments in the file for how
this is done.
3. Changes how run-integration-tests is done so that the directories
run are specified through a cli argument.
4. Updates the main.yml workflow so that it also runs the
cli-integration tests.
5. Takes advantage of the return value of the call to `activate` on the
extension. This allows the integration tests to have access to internal
variables of the extension like the context, cli, and query server.
6. And of course, adds a handful of simple tests that ensure we have a
cli installed of the correct version.
And separate flag for IDE server. Setting these flags to `true` will
start the respective Java processes in debug mode so that they can
be attached to a debugger.
We need to provide the `--extensionDevelopmentPath` flag in these
launch configurations.
It appears to be unnecessary to include
`${workspaceRoot}/extensions/ql-vscode/out/vscode-tests/**/*.js`
in addition to the strictly more general pattern
${workspaceRoot}/extensions/ql-vscode/out/**/*.js
An unfortunate UI fact appears to be that the log of the gulp build is
focused whenever these tasks are run, even though the log you actually
care about seeing is in the `Debug Console` section. Not sure how to
fix that.
The behavior without this line is to use whichever workspace was
opened last when testing. I find this more convenient, since I have
several (non-vscode-codeql-starter-workspace) local workspaces I use
for manual testing, and it's nice to have them persist from one run to
the next.
Adds section in CONTRIBUTING.md to document how to run tests. Also,
fixes some markdown linting warnings.
And fixes the launch config for running unit tests.
The PR contains the initial implementing of QL Test support in CodeQL for Visual Studio Code. Because QL Test support isn't quite ready in the CLI yet, this PR uses `odasa` to run the tests for now. As CLI support comes online, it should be straightforward to swap out the implementation to use the CLI.
The treeview UI for the tests is implemented via the `hbenl.vscode-test-explorer` extension. This extension is open source, and appears to be actively maintained. It's used by a couple dozen existing extensions for tests for various languages. The extension doesn't really do anything on its own, so taking it as a dependency isn't introducing any unwanted UI clutter. Note that I did have to remove the `--disable-extensions` argument from `launch.json`, because otherwise the test explorer extension gets disabled, preventing our own extension from loading.
The UI will display a root node for each QL pack that contains tests, with the actual test directories and files as descendants of that root node. We consider only those QL packs in the workspace; QL packs on the default CodeQL search path are ignored. We use `codeql resolve qlpacks` to find the packs, and then watch all `qlpack.yml` files in the workspace for changes in order to refresh the pack discovery when necessary. Ideally, we'd have the CLI return a set of path patterns to watch, but for now the current implementation works fine.
To discover the tests within a given pack, we walk the pack's directory tree manually for now, until the relevant CLI command is available. Because we do not yet have a mechanism in `qlpack.yml` to specify whether or not the pack contains tests, we assume that any pack whose name ends with "-tests" to contain nothing but tests, and any other pack to contain no tests. This is sufficient for the tests in the QL repo. As with QL pack discovery, we watch the file system for changes in `.ql` and `.qlref` files in order to refresh the tree of tests if anything changes.
To actually run the tests, we just invoke `odasa qltest` with the appropriate arguments. This code is pretty much a straight copy-and-paste from the repo where I've had a private version of QL Test support for several months. Once we can run tests via the CLI, this will all be deleted.
The `test-ui.ts` file implements a couple of additional commands for the context menu of the test treeview. You can accept the output of a failing test (copying the `.actual` file to the `.expected` file), and you can bring up a diff view of the `.expected` and `.actual` files).
This PR includes a couple of related utility classes. `UIService` makes it a little easier to implement a service that handles VS Code commands. `Discovery` is a base class that handles most of the work that is shared between the different kinds of discovery that we do, like avoiding running multiple discovery operations simultaneously if we get a storm of file change notifications.
Elena Tanasoiu
Charis Kyriakou
Nora
Koen Vlaswinkel
Shati Patel
Andrew Eisenberg
Angela P Wen
Jason Reed
Dave Bartolomeo
Andrew Eisenberg
Aditya Sharad