This commit adds telemetry capturing for command execution. The data
captured explicitly captured and sent to application insights is only
the command id, execution time, and command completion status. We also
capture errors thrown by any command execution, but these are not sent
to application insights.
Telemetry capturing is opt-in. No data will be sent to application
insights unless the user explicitly allows it.
There are two new config settings added. The first controls whether or
not telemetry should be sent. This setting AND the global telemetry setting
must be enabled in order for telemetry to be sent.
The second setting controls whether or not telemetry event data should
be logged to the extension console. The hope here is that users can
inspect exactly what data is sent to the server and can have confidence
that nothing concerning is being leaked.
Note that the global setting for disabling telemetry collection is
handled inside the `vscode-extension-telemetry` package implicitly, so
this extension doesn't touch that setting explicitly.
The `codeql.canary` setting is being used to add an additional flag to
telemetry events. This flag will help us determine if a user in internal
or not.
The application insights key is injected at build time through a
repository secret.
This commit also includes a new `TELEMETRY.md` file that explains what
is being captured, and why.
This commit fixes#733. It does it by ensuring that the query server
emits an event when it restarts the query server. The database manager
listens for this even and properly re-registers its databases.
A few caveats though:
1. Convert query restarts to using a command that includes progress.
This will ensure that errors on restart are logged properly.
2. Because we want to log errors, we cannot use the vscode standard
EventEmitters. They run in the next tick and therefore any errors
will not be associated with this command execution.
3. Update the default cli version to run integration tests against to
2.4.2.
4. Add a new integration test that fails if databases are not
re-registered.
This adds two new jobs to the `Release` workflow. These
jobs are blocked behind an environment. When approved
by a committer, the extension will be deployed to
Open VSX and VS Code marketplace.
Also, update contributing docs for open-vsx publishing.
* Now query.test.ts runs on multiple cli versions
* Removed most `dispose` calls in cli tests because each test shares the
same instance of the extension and all of its properties. So, we
shouldn't be disposing until the last test completes. It's likely that
we will need to be more careful about cleaning up state between test
runs, but we haven't hit that yet and this can happen in a later
commit.
* Requires that QL_PATH environment variable is set and points to a
checkout of github/codeql
* Adds the `quiet` flag to the cli. When set, this flag will prevent
some modal dialogs from disrupting the flow. Currently, we only ensure
that the upgrades dialog is avoided.
* Update the main.yml workflow to checkout the codeql repo
This commit adds integration tests that run commands using the CLI. This
change introduces a number of enhancements in order to get there.
1. Augments the index-template.ts file so that it downloads an
appropriate cli version if requested.
2. Adds the ensureCli.ts that performs the download if a a suitable
version is not already installed. See the comments in the file for how
this is done.
3. Changes how run-integration-tests is done so that the directories
run are specified through a cli argument.
4. Updates the main.yml workflow so that it also runs the
cli-integration tests.
5. Takes advantage of the return value of the call to `activate` on the
extension. This allows the integration tests to have access to internal
variables of the extension like the context, cli, and query server.
6. And of course, adds a handful of simple tests that ensure we have a
cli installed of the correct version.
This query ensures that all of our files marked as "pure" remain that
way. In this case "pure" means that it does not depend on vscode and
can therefore be run in tests outside of a runtime environment.
This commit also explicitly moves all of our "pure" files to the
`src/pure` directory.
I have already pushed `main` upstream pointing at the same commit as
master, so this PR should be safe to merge. To the best of my
knowledge all that's necessary after that is to change the
default branch
in https://github.com/github/vscode-codeql/settings/branches.
Andrew Eisenberg
Aditya Sharad
Shati Patel
Aditya Sharad
Jason Reed
Andrew Eisenberg
Dave Bartolomeo
Jason Reed
Nick Rolfe
Henry Mercer
Alexander Eyers-Taylor