|
|
|
|
@ -49,7 +49,7 @@ To produce more detailed logging output, you can enable step debug logging. For
|
|
|
|
|
## Creating {% data variables.product.prodname_codeql %} debugging artifacts
|
|
|
|
|
|
|
|
|
|
You can obtain artifacts to help you debug {% data variables.product.prodname_codeql %}.
|
|
|
|
|
The debug artifacts will be uploaded to the workflow run as an artifact named `debug-artifacts`. The data contains the {% data variables.product.prodname_codeql %} logs, {% data variables.product.prodname_codeql %} database(s), and any SARIF file(s) produced by the workflow.
|
|
|
|
|
The debug artifacts will be uploaded to the workflow run as an artifact named `debug-artifacts`. The data contains the {% data variables.product.prodname_codeql %} logs, {% data variables.product.prodname_codeql %} database(s), and any SARIF file(s) produced by the workflow.
|
|
|
|
|
|
|
|
|
|
These artifacts will help you debug problems with {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %}. If you contact GitHub support, they might ask for this data.
|
|
|
|
|
|
|
|
|
|
@ -59,11 +59,10 @@ These artifacts will help you debug problems with {% data variables.product.prod
|
|
|
|
|
|
|
|
|
|
### Creating {% data variables.product.prodname_codeql %} debugging artifacts by re-running jobs with debug logging enabled
|
|
|
|
|
|
|
|
|
|
You can create {% data variables.product.prodname_codeql %} debugging artifacts by enabling debug logging and re-running the jobs. For more information about re-running {% data variables.product.prodname_actions %} workflows and jobs, see "[Re-running workflows and jobs](/actions/managing-workflow-runs/re-running-workflows-and-jobs)."
|
|
|
|
|
You can create {% data variables.product.prodname_codeql %} debugging artifacts by enabling debug logging and re-running the jobs. For more information about re-running {% data variables.product.prodname_actions %} workflows and jobs, see "[Re-running workflows and jobs](/actions/managing-workflow-runs/re-running-workflows-and-jobs)."
|
|
|
|
|
|
|
|
|
|
You need to ensure that you select **Enable debug logging** . This option enables runner diagnostic logging and step debug logging for the run. You'll then be able to download `debug-artifacts` to investigate further. You do not need to modify the workflow file when creating {% data variables.product.prodname_codeql %} debugging artifacts by re-running jobs.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
{% endif %}
|
|
|
|
|
|
|
|
|
|
{% ifversion fpt or ghec or ghes > 3.3 or ghae > 3.3 %}
|
|
|
|
|
@ -87,7 +86,7 @@ If an automatic build of code for a compiled language within your project fails,
|
|
|
|
|
|
|
|
|
|
- Remove the `autobuild` step from your {% data variables.product.prodname_code_scanning %} workflow and add specific build steps. For information about editing the workflow, see "[Configuring {% data variables.product.prodname_code_scanning %}](/code-security/secure-coding/configuring-code-scanning#editing-a-code-scanning-workflow)." For more information about replacing the `autobuild` step, see "[Configuring the {% data variables.product.prodname_codeql %} workflow for compiled languages](/code-security/secure-coding/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
|
|
|
|
|
|
|
|
|
- If your workflow doesn't explicitly specify the languages to analyze, {% data variables.product.prodname_codeql %} implicitly detects the supported languages in your code base. In this configuration, out of the compiled languages C/C++, C#, and Java, {% data variables.product.prodname_codeql %} only analyzes the language with the most source files. Edit the workflow and add a matrix specifying the languages you want to analyze. The default CodeQL analysis workflow uses such a matrix.
|
|
|
|
|
- If your workflow doesn't explicitly specify the languages to analyze, {% data variables.product.prodname_codeql %} implicitly detects the supported languages in your code base. In this configuration, out of the compiled languages C/C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} and Java, {% data variables.product.prodname_codeql %} only analyzes the language with the most source files. Edit the workflow and add a matrix specifying the languages you want to analyze. The default CodeQL analysis workflow uses such a matrix.
|
|
|
|
|
|
|
|
|
|
The following extracts from a workflow show how you can use a matrix within the job strategy to specify languages, and then reference each language within the "Initialize {% data variables.product.prodname_codeql %}" step:
|
|
|
|
|
|
|
|
|
|
@ -131,14 +130,15 @@ If your workflow fails with an error `No source code was seen during the build`
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
For more information, see the workflow extract in "[Automatic build for a compiled language fails](#automatic-build-for-a-compiled-language-fails)" above.
|
|
|
|
|
1. Your {% data variables.product.prodname_code_scanning %} workflow is analyzing a compiled language (C, C++, C#, or Java), but the code was not compiled. By default, the {% data variables.product.prodname_codeql %} analysis workflow contains an `autobuild` step, however, this step represents a best effort process, and may not succeed in building your code, depending on your specific build environment. Compilation may also fail if you have removed the `autobuild` step and did not include build steps manually. For more information about specifying build steps, see "[Configuring the {% data variables.product.prodname_codeql %} workflow for compiled languages](/code-security/secure-coding/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
|
|
|
|
1. Your workflow is analyzing a compiled language (C, C++, C#, or Java), but portions of your build are cached to improve performance (most likely to occur with build systems like Gradle or Bazel). Since {% data variables.product.prodname_codeql %} observes the activity of the compiler to understand the data flows in a repository, {% data variables.product.prodname_codeql %} requires a complete build to take place in order to perform analysis.
|
|
|
|
|
1. Your workflow is analyzing a compiled language (C, C++, C#, or Java), but compilation does not occur between the `init` and `analyze` steps in the workflow. {% data variables.product.prodname_codeql %} requires that your build happens in between these two steps in order to observe the activity of the compiler and perform analysis.
|
|
|
|
|
1. Your compiled code (in C, C++, C#, or Java) was compiled successfully, but {% data variables.product.prodname_codeql %} was unable to detect the compiler invocations. The most common causes are:
|
|
|
|
|
|
|
|
|
|
* Running your build process in a separate container to {% data variables.product.prodname_codeql %}. For more information, see "[Running CodeQL code scanning in a container](/code-security/secure-coding/running-codeql-code-scanning-in-a-container)."
|
|
|
|
|
* Building using a distributed build system external to GitHub Actions, using a daemon process.
|
|
|
|
|
* {% data variables.product.prodname_codeql %} isn't aware of the specific compiler you are using.
|
|
|
|
|
1. Your {% data variables.product.prodname_code_scanning %} workflow is analyzing a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but the code was not compiled. By default, the {% data variables.product.prodname_codeql %} analysis workflow contains an `autobuild` step, however, this step represents a best effort process, and may not succeed in building your code, depending on your specific build environment. Compilation may also fail if you have removed the `autobuild` step and did not include build steps manually. For more information about specifying build steps, see "[Configuring the {% data variables.product.prodname_codeql %} workflow for compiled languages](/code-security/secure-coding/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
|
|
|
|
1. Your workflow is analyzing a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but portions of your build are cached to improve performance (most likely to occur with build systems like Gradle or Bazel). Since {% data variables.product.prodname_codeql %} observes the activity of the compiler to understand the data flows in a repository, {% data variables.product.prodname_codeql %} requires a complete build to take place in order to perform analysis.
|
|
|
|
|
1. Your workflow is analyzing a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but compilation does not occur between the `init` and `analyze` steps in the workflow. {% data variables.product.prodname_codeql %} requires that your build happens in between these two steps in order to observe the activity of the compiler and perform analysis.
|
|
|
|
|
1. Your compiled code (in C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java) was compiled successfully, but {% data variables.product.prodname_codeql %} was unable to detect the compiler invocations. The most common causes are:
|
|
|
|
|
|
|
|
|
|
- Running your build process in a separate container to {% data variables.product.prodname_codeql %}. For more information, see "[Running CodeQL code scanning in a container](/code-security/secure-coding/running-codeql-code-scanning-in-a-container)."
|
|
|
|
|
- Building using a distributed build system external to GitHub Actions, using a daemon process.
|
|
|
|
|
- {% data variables.product.prodname_codeql %} isn't aware of the specific compiler you are using.
|
|
|
|
|
|
|
|
|
|
For .NET Framework projects, and for C# projects using either `dotnet build` or `msbuild`, you should specify `/p:UseSharedCompilation=false` in your workflow's `run` step, when you build your code.
|
|
|
|
|
|
|
|
|
|
@ -151,9 +151,10 @@ If your workflow fails with an error `No source code was seen during the build`
|
|
|
|
|
|
|
|
|
|
If you encounter another problem with your specific compiler or configuration, contact {% data variables.contact.contact_support %}.
|
|
|
|
|
|
|
|
|
|
For more information about specifying build steps, see "[Configuring the {% data variables.product.prodname_codeql %} workflow for compiled languages](/code-security/secure-coding/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
|
|
|
|
For more information about specifying build steps, see "[Configuring the {% data variables.product.prodname_codeql %} workflow for compiled languages](/code-security/secure-coding/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
|
|
|
|
|
|
|
|
|
{% ifversion fpt or ghes > 3.1 or ghae or ghec %}
|
|
|
|
|
|
|
|
|
|
## Lines of code scanned are lower than expected
|
|
|
|
|
|
|
|
|
|
For compiled languages like C/C++, C#, Go, and Java, {% data variables.product.prodname_codeql %} only scans files that are built during the analysis. Therefore the number of lines of code scanned will be lower than expected if some of the source code isn't compiled correctly. This can happen for several reasons:
|
|
|
|
|
@ -163,12 +164,13 @@ For compiled languages like C/C++, C#, Go, and Java, {% data variables.product.p
|
|
|
|
|
|
|
|
|
|
If your {% data variables.product.prodname_codeql %} analysis scans fewer lines of code than expected, there are several approaches you can try to make sure all the necessary source files are compiled.
|
|
|
|
|
|
|
|
|
|
### Replace the `autobuild` step
|
|
|
|
|
### Replace the `autobuild` step
|
|
|
|
|
|
|
|
|
|
Replace the `autobuild` step with the same build commands you would use in production. This makes sure that {% data variables.product.prodname_codeql %} knows exactly how to compile all of the source files you want to scan.
|
|
|
|
|
For more information, see "[Configuring the {% data variables.product.prodname_codeql %} workflow for compiled languages](/code-security/secure-coding/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
|
|
|
|
For more information, see "[Configuring the {% data variables.product.prodname_codeql %} workflow for compiled languages](/code-security/secure-coding/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
|
|
|
|
|
|
|
|
|
### Inspect the copy of the source files in the {% data variables.product.prodname_codeql %} database
|
|
|
|
|
|
|
|
|
|
You may be able to understand why some source files haven't been analyzed by inspecting the copy of the source code included with the {% data variables.product.prodname_codeql %} database. To obtain the database from your Actions workflow, modify the `init` step of your {% data variables.product.prodname_codeql %} workflow file and set `debug: true`.
|
|
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
|
@ -188,12 +190,13 @@ The artifact will contain an archived copy of the source files scanned by {% dat
|
|
|
|
|
|
|
|
|
|
## Extraction errors in the database
|
|
|
|
|
|
|
|
|
|
The {% data variables.product.prodname_codeql %} team constantly works on critical extraction errors to make sure that all source files can be scanned. However, the {% data variables.product.prodname_codeql %} extractors do occasionally generate errors during database creation. {% data variables.product.prodname_codeql %} provides information about extraction errors and warnings generated during database creation in a log file.
|
|
|
|
|
The {% data variables.product.prodname_codeql %} team constantly works on critical extraction errors to make sure that all source files can be scanned. However, the {% data variables.product.prodname_codeql %} extractors do occasionally generate errors during database creation. {% data variables.product.prodname_codeql %} provides information about extraction errors and warnings generated during database creation in a log file.
|
|
|
|
|
The extraction diagnostics information gives an indication of overall database health. Most extractor errors do not significantly impact the analysis. A small number of extractor errors is healthy and typically indicates a good state of analysis.
|
|
|
|
|
|
|
|
|
|
However, if you see extractor errors in the overwhelming majority of files that were compiled during database creation, you should look into the errors in more detail to try to understand why some source files weren't extracted properly.
|
|
|
|
|
|
|
|
|
|
{% else %}
|
|
|
|
|
|
|
|
|
|
## Portions of my repository were not analyzed using `autobuild`
|
|
|
|
|
|
|
|
|
|
The {% data variables.product.prodname_codeql %} `autobuild` feature uses heuristics to build the code in a repository, however, sometimes this approach results in incomplete analysis of a repository. For example, when multiple `build.sh` commands exist in a single repository, the analysis may not complete since the `autobuild` step will only execute one of the commands. The solution is to replace the `autobuild` step with build steps which build all of the source code which you wish to analyze. For more information, see "[Configuring the {% data variables.product.prodname_codeql %} workflow for compiled languages](/code-security/secure-coding/configuring-the-codeql-workflow-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
|
|
|
|
@ -201,7 +204,7 @@ The {% data variables.product.prodname_codeql %} `autobuild` feature uses heuris
|
|
|
|
|
|
|
|
|
|
## The build takes too long
|
|
|
|
|
|
|
|
|
|
If your build with {% data variables.product.prodname_codeql %} analysis takes too long to run, there are several approaches you can try to reduce the build time.
|
|
|
|
|
If your build with {% data variables.product.prodname_codeql %} analysis takes too long to run, there are several approaches you can try to reduce the build time.
|
|
|
|
|
|
|
|
|
|
### Increase the memory or cores
|
|
|
|
|
|
|
|
|
|
@ -225,7 +228,7 @@ If your analysis is still too slow to be run during `push` or `pull_request` eve
|
|
|
|
|
|
|
|
|
|
### Check which query suites the workflow runs
|
|
|
|
|
|
|
|
|
|
By default, there are three main query suites available for each language. If you have optimized the CodeQL database build and the process is still too long, you could reduce the number of queries you run. The default query suite is run automatically; it contains the fastest security queries with the lowest rates of false positive results.
|
|
|
|
|
By default, there are three main query suites available for each language. If you have optimized the CodeQL database build and the process is still too long, you could reduce the number of queries you run. The default query suite is run automatically; it contains the fastest security queries with the lowest rates of false positive results.
|
|
|
|
|
|
|
|
|
|
You may be running extra queries or query suites in addition to the default queries. Check whether the workflow defines an additional query suite or additional queries to run using the `queries` element. You can experiment with disabling the additional query suite or queries. For more information, see "[Configuring {% data variables.product.prodname_code_scanning %}](/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs)."
|
|
|
|
|
|
|
|
|
|
@ -237,6 +240,7 @@ You may be running extra queries or query suites in addition to the default quer
|
|
|
|
|
{% endif %}
|
|
|
|
|
|
|
|
|
|
{% ifversion fpt or ghec %}
|
|
|
|
|
|
|
|
|
|
## Results differ between analysis platforms
|
|
|
|
|
|
|
|
|
|
If you are analyzing code written in Python, you may see different results depending on whether you run the {% data variables.product.prodname_codeql_workflow %} on Linux, macOS, or Windows.
|
|
|
|
|
@ -256,11 +260,13 @@ On very large projects, {% data variables.product.prodname_codeql %} may run out
|
|
|
|
|
{% else %}If you encounter this issue, try increasing the memory on the runner.{% endif %}
|
|
|
|
|
|
|
|
|
|
{% ifversion fpt or ghec %}
|
|
|
|
|
|
|
|
|
|
## Error: 403 "Resource not accessible by integration" when using {% data variables.product.prodname_dependabot %}
|
|
|
|
|
|
|
|
|
|
{% data variables.product.prodname_dependabot %} is considered untrusted when it triggers a workflow run, and the workflow will run with read-only scopes. Uploading {% data variables.product.prodname_code_scanning %} results for a branch usually requires the `security_events: write` scope. However, {% data variables.product.prodname_code_scanning %} always allows the uploading of results when the `pull_request` event triggers the action run. This is why, for {% data variables.product.prodname_dependabot %} branches, we recommend you use the `pull_request` event instead of the `push` event.
|
|
|
|
|
|
|
|
|
|
A simple approach is to run on pushes to the default branch and any other important long-running branches, as well as pull requests opened against this set of branches:
|
|
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
|
on:
|
|
|
|
|
push:
|
|
|
|
|
@ -270,7 +276,9 @@ on:
|
|
|
|
|
branches:
|
|
|
|
|
- main
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
An alternative approach is to run on all pushes except for {% data variables.product.prodname_dependabot %} branches:
|
|
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
|
on:
|
|
|
|
|
push:
|
|
|
|
|
@ -282,6 +290,7 @@ on:
|
|
|
|
|
### Analysis still failing on the default branch
|
|
|
|
|
|
|
|
|
|
If the {% data variables.product.prodname_codeql_workflow %} still fails on a commit made on the default branch, you need to check:
|
|
|
|
|
|
|
|
|
|
- whether {% data variables.product.prodname_dependabot %} authored the commit
|
|
|
|
|
- whether the pull request that includes the commit has been merged using `@dependabot squash and merge`
|
|
|
|
|
|
|
|
|
|
|
Angela P Wen
GitHub