From ae0d0f530eb793a44cd005e5bf2ae85fd49ffb89 Mon Sep 17 00:00:00 2001 From: Shati Patel <42641846+shati-patel@users.noreply.github.com> Date: Mon, 28 Sep 2020 18:42:34 +0100 Subject: [PATCH] Code scanning troubleshooting: How to set up environment with CodeQL runner (#15746) * Code scanning troubleshooting: Set up environment with CodeQL runner * Change "trace" to "monitor" * Fix note syntax --- ...running-code-scanning-in-your-ci-system.md | 4 ++-- ...hooting-code-scanning-in-your-ci-system.md | 24 +++++++++++++++++-- .../troubleshooting-the-codeql-workflow.md | 2 +- 3 files changed, 25 insertions(+), 5 deletions(-) diff --git a/content/github/finding-security-vulnerabilities-and-errors-in-your-code/running-code-scanning-in-your-ci-system.md b/content/github/finding-security-vulnerabilities-and-errors-in-your-code/running-code-scanning-in-your-ci-system.md index c280d89869..f22e4fb2b2 100644 --- a/content/github/finding-security-vulnerabilities-and-errors-in-your-code/running-code-scanning-in-your-ci-system.md +++ b/content/github/finding-security-vulnerabilities-and-errors-in-your-code/running-code-scanning-in-your-ci-system.md @@ -97,7 +97,7 @@ The server has access to download the {{ site.data.variables.product.prodname_co #### Compiled language example -This example is similar to the previous example, however this time the repository has code in C/C++, C#, or Java. To create a {{ site.data.variables.product.prodname_codeql }} database for these languages, the CLI needs to trace the build. At the end of the initialization process, the runner reports the command you need to set up the environment before building the code. You need to run this command, before calling the normal CI build process, and then running the `analyze` command. +This example is similar to the previous example, however this time the repository has code in C/C++, C#, or Java. To create a {{ site.data.variables.product.prodname_codeql }} database for these languages, the CLI needs to monitor the build. At the end of the initialization process, the runner reports the command you need to set up the environment before building the code. You need to run this command, before calling the normal CI build process, and then running the `analyze` command. 1. Check out the repository to analyze. 1. Move into the directory where the repository is checked out. @@ -114,7 +114,7 @@ This example is similar to the previous example, however this time the repositor . /srv/checkout/example-repo-2/codeql-runner/codeql-env.sh". ``` -1. Run the script generated by the `init` action to set up the environment to trace the build. +1. Run the script generated by the `init` action to set up the environment to monitor the build. ```shell $ . /srv/checkout/example-repo-2/codeql-runner/codeql-env.sh diff --git a/content/github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-code-scanning-in-your-ci-system.md b/content/github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-code-scanning-in-your-ci-system.md index 986c17a0a4..e10d6f6719 100644 --- a/content/github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-code-scanning-in-your-ci-system.md +++ b/content/github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-code-scanning-in-your-ci-system.md @@ -22,10 +22,30 @@ To avoid this automatic download, you can manually download the {{ site.data.var ### No code found during the build -If the `analyze` command for the {{ site.data.variables.product.prodname_codeql_runner }} fails with an error `No source code was seen during the build`, this indicates that {{ site.data.variables.product.prodname_codeql }} was unable to trace your code. Several reasons can explain such a failure. +If the `analyze` command for the {{ site.data.variables.product.prodname_codeql_runner }} fails with an error `No source code was seen during the build`, this indicates that {{ site.data.variables.product.prodname_codeql }} was unable to monitor your code. Several reasons can explain such a failure. 1. Automatic language detection identified a supported language, but there is no analyzable code of that language in the repository. A typical example is when our language detection service finds a file associated with a particular programming language like a `.h`, or `.gyp` file, but no corresponding executable code is present in the repository. To solve the problem, you can manually define the languages you want to analyze by using the `--languages` flag of the `init` command. For more information, see "[Configuring {{ site.data.variables.product.prodname_code_scanning }} in your CI system](/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning-in-your-ci-system)." -1. You're analyzing a compiled language without using the `autobuild` command and you run the build steps yourself after the `init` step. For the build to work, you must set up the environment such that the {{ site.data.variables.product.prodname_codeql_runner }} can trace the code. The `init` command generates instructions for how to export the required environment variables, so you can copy and run the script. For more information, see "[Running {{ site.data.variables.product.prodname_code_scanning }} in your CI system](/github/finding-security-vulnerabilities-and-errors-in-your-code/running-code-scanning-in-your-ci-system#compiled-language-example)." +1. You're analyzing a compiled language without using the `autobuild` command and you run the build steps yourself after the `init` step. For the build to work, you must set up the environment such that the {{ site.data.variables.product.prodname_codeql_runner }} can monitor the code. The `init` command generates instructions for how to export the required environment variables, so you can copy and run the script after you've run the `init` command. + - On macOS and Linux: + ```shell + $ . codeql-runner/codeql-env.sh + ``` + - On Windows, using the Command shell (`cmd`) or a batch file (`.bat`): + ```shell + > call codeql-runner\codeql-env.bat + ``` + - On Windows, using PowerShell: + ```shell + > cat codeql-runner\codeql-env.sh | Invoke-Expression + ``` + + The environment variables are also stored in the file `codeql-runner/codeql-env.json`. This file contains a single JSON object which maps environment variable keys to values. If you can't run the script generated by the `init` command, then you can use the data in JSON format instead. + + {% note %} + + **Note:** If you used the `--temp-dir` flag of the `init` command to specify a custom directory for temporary files, the path to the `codeql-env` files might be different. + + {% endnote %} 1. The code is built in a container or on a separate machine. If you use a containerized build or if you outsource the build to another machine, make sure to run the {{ site.data.variables.product.prodname_codeql_runner }} in the container or on the machine where your build task takes place. diff --git a/content/github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-the-codeql-workflow.md b/content/github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-the-codeql-workflow.md index 6e0d0b95dc..601b000049 100644 --- a/content/github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-the-codeql-workflow.md +++ b/content/github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-the-codeql-workflow.md @@ -44,7 +44,7 @@ If an automatic build of code for a compiled language within your project fails, ### No code found during the build -If your workflow fails with an error `No source code was seen during the build` or `The process '/opt/hostedtoolcache/CodeQL/0.0.0-20200630/x64/codeql/codeql' failed with exit code 32`, this indicates that {{ site.data.variables.product.prodname_codeql }} was unable to trace your code. Several reasons can explain such a failure: +If your workflow fails with an error `No source code was seen during the build` or `The process '/opt/hostedtoolcache/CodeQL/0.0.0-20200630/x64/codeql/codeql' failed with exit code 32`, this indicates that {{ site.data.variables.product.prodname_codeql }} was unable to monitor your code. Several reasons can explain such a failure: 1. Automatic language detection identified a supported language, but there is no analyzable code of that language in the repository. A typical example is when our language detection service finds a file associated with a particular programming language like a `.h`, or `.gyp` file, but no corresponding executable code is present in the repository. To solve the problem, you can manually define the languages you want to analyze by updating the list of languages in the `language` matrix. For example, the following configuration will analyze only Go, and JavaScript.