Fix bug in which excluded reports would be published to the wrong
folder. This was accidentally introduced in a recent refactor
and no reports were actually published to the wrong folder (this would
have been caught before submit if it had occurred).
Change-Id: Icc9eb99b2ceb185310e99eaa39e45072e0ae6c80
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/590280
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Remove error return value from xref, which always returns nil.
(Caught by unparam and blocking deploy of vulndb)
Change-Id: I4c9423f0d333d7beb9422ee558ed83f3dd99aebf
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/590115
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Tatiana Bradley <tatianabradley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
With no arguments, "vulnreport commit" now individually commits all
reports that have been added/changed (according to git status).
(To commit them all as a single commit, use "vulnreport -batch commit").
The flag "-status=<REVIEW_STATUS>" can additionally be used
to only commit reports with a certain review status.
Change-Id: I4efb4e866166b6153d556409408021dc861656fb
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/590035
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Adds a check to "vulnreport fix" that errors if any URLs in the
"references" section return an error or status 404 on HTTP HEAD.
We don't check for other status codes yet.
An experiment to error on all non-200 status codes brought up some
ambiguous cases where the link is still viewable in a browser, e.g.:
- 429 Too Many Requests (https://vuldb.com/?id.256304)
- 503 Service Unavailable (http://blog.recurity-labs.com/2017-08-10/scm-vulns):
- 403 Forbidden (https://www.sap.com/documents/2022/02/fa865ea4-167e-0010-bca6-c68f7e60039b.html)
For now, this is a fix check and not a lint check, meaning it only
applies to new reports, and can technically be ignored (by manually
creating a CL that adds the report).
This CL also deletes existing URLs in the corpus that don't exist
according to this check.
Change-Id: Id14fb79fc2f2c2d4c8145fdc88d11aa33708c94b
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/588761
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Command vulnreport create now decides whether to generate a REVIEWED
or UNREVIEWED report based on issue's labels.
This can be overridden with flag "-status=<REVIEW_STATUS>". The "unreviewed"
flag is removed.
Change-Id: I8f8b808c6f9bbcaeb0dc176fb6cb875b8f9ccee4
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/587976
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Update the vulnreport triage command to label any issues it triages
as "triaged", and skip issues that are already labeled as such.
(The flag "-f" overrides the skip).
This allows the triager to simply run "vulnreport triage" to triage all
untriaged issues.
Change-Id: I24611166d972c46100b1e8fd38bed1fb87071d11
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/587915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Zvonimir Pavlinovic <zpavlinovic@google.com>
Move the check for outstanding TODOs from the vulnreport
fix function to the report.Lint function.
Change-Id: I909dae66400423453c6178ec452462e0cf1f4273
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/585417
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Flag -batch causes vulnreport commit to (attempt to) create a single
commit that commits all reports indicated by its arguments, instead
of committing each report separately.
Change-Id: Ib0c9b3d52c7654ac952e78ca042ce5f29b98f48a
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/585439
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Flag -unreviewed causes create to generate unreviewed reports
(with no description and no TODOs).
Change-Id: If60eeb22983b3b6480d440de740e6eedf3fcf8d9
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/585438
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In report.New, make UNREVIEWED the default status. This
can be overridden via the WithReviewStatus() option.
(This does not change the fact that vulnreport create creates
REVIEWED reports by default).
Change-Id: Id4c6d453a2b977986381eb81b7e2fb1087c9b735
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/585516
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add support for the field "review_status" for both YAML
and OSV, but don't yet publish any reports with the field.
Newly created reports will be marked REVIEWED and newly "unexcluded"
reports will be marked UNREVIEWED by default.
Change-Id: Id04c9d1c9f3240b1fa277c23c2351627c99b1fc4
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/585515
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Replace "vulnreport duplicates" with a new command, "vulnreport triage"
that checks the current open issues for duplicates and issues that are likely
not affecting Go code, and determines the issues' priority.
Likely duplicates are labeled "possible duplicate" and issues that may
not affect Go code are labeled "possibly not Go". (This is determined
based on the number of excluded issues labeled "NOT_GO_CODE" that also
affect this module).
An issue is considered high priority if its module has more than 100
importers and has more existing regular reports than excluded reports.
Issues that are high priority are automatically labeled "high priority".
Number of importers is currently determined based on a static file that
is checked into the repo.
Change-Id: Iecb311a68c8c15851417c0f8561df23bcd0e467d
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/584976
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Re-structure the vulnreport commands as collections of composable
types that perform common operations, such as reading GitHub issues
and creating new reports.
This allows us to re-use code & increase the consistency of
the various commands' behavior. It also makes it easier to create new
commands and operate on batches of issues / reports.
Change-Id: I5ffb9c5cf2c9169ca755693d460ee13dc94c18f6
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/584217
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
For new reports, the default created time is now time.Now(). An
alternate time (e.g., for testing) can be provided via the WithCreated()
option.
Update tests to include a placeholder created time.
Change-Id: I2c48ac56c89d2f33310fca58ae44ff7e9035f609
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/583837
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
- Move some functions related to CVE/GHSA regex matching
to a new "idstr" package, as they are not related specifically to the
CVE5 or GHSA GraphQL format.
- Move all logic related to the cve5, cve4 and legacyGHSA formats in
the "internal/report" package to their own files, so it is easier to
(potentially) move these to their own packages in the future.
The goal of this CL is to reduce the risk of import cycles
for some upcoming refactors.
Change-Id: I7e14c31c17882230b783cc62e1ecdf43dcb98995
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/581717
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
- Don't error when an issue is already marked "duplicate"
- Don't mark an issue as a duplicate if it already has a report
(corresponding to the issue itself)
Change-Id: I8909a0963727b070484993fd9e6324ada3c828f5
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/584377
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add a new interface, report.Source, which represents a source
(e.g., a GHSA or CVE) that can be converted into a Report.
This is part of an effort to make report creation more uniform
across various sources, and collect together common logic.
Change-Id: If78bab9de143bca72b88ca7f98bd01a51d17ccae
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/579855
Commit-Queue: Tatiana Bradley <tatianabradley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Adds a new field, source, which holds metadata about the
original source of the report. For now, it is either a CVE,
a GHSA, or the Go team.
This is not required and is not published to OSV, but will
assist in our automation efforts.
Change-Id: Ief5ea6eca35d8799655b3a86e7a22cf8ff49d4e5
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/576999
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Some operations in "unexclude" were incorrectly operating
on the old report instead of the new report; fix this.
Change-Id: Iaae67a6292bb10e0840e9841cccd0ffe1cc5d4ba
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/576995
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
In vulnreport commit, automatically change the commit message
is based on the files in the git staging area. For example, if a report
is being updated, use the word "update" to describe the commit. If an
excluded report is being deleted and a corresponding regular report added,
use the word "unexclude".
Change-Id: I021c4f5e8e6b522a9e4ef0597501dfb8426efec2
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/569598
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Instead of searching for specific derived files to add/commit,
use the pattern "*<Go ID>*" to add/commit all files mentioning the
current Go ID. This will allow commit to work when "unexcluding" a report.
Change-Id: I22f8b91b1fc5f0255db26b9b05d680578817c899
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/569596
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Adds a flag, -user, which can be used to specify a GitHub username.
When applied to the vulnreport create-excluded command, issues assigned
to other users will be ignored. This can also be applied using the
GITHUB_USER environment variable.
Change-Id: Icc4104823249a393adc5755585fbb3e339ccc945
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/565380
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Command vulnreport duplicates (with no args), finds likely
duplicates for all open issues on the tracker. With args, it finds
likely duplicates for the given issue(s).
This can be used by the vulndb triager to quickly identify duplicate
issues on the tracker.
Change-Id: I863d733a1dbdfcc201457fcf7156698f0012e671
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/567196
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
This change removes all code related to generating and validating
vulnerability databases in the legacy format.
Fixesgolang/go#65320
Change-Id: I83522efec90aa04df54976d5c29d485acd13e53c
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/558658
Auto-Submit: Tatiana Bradley <tatianabradley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Package the logic to read reports from a vulndb-like repo into
a Client, and update all callers.
For now, the behavior under the hood is basically the same as it was, but
the callers do not need to carry around the specific maps needed
to access certain information.
Change-Id: I7bd28bfbf2a40d5d2bc37453051f866517086f23
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/562177
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Adds an -update flag to the vulnreport symbols command.
Previously, vulnreport would automatically add all fix links that
resulted in vulnerable symbols for a given module to that module's
FixLinks field.
Now, if the FixLinks field is populated it is considered the source of
truth, and vulnreport symbols will use only the links in that field.
If FixLinks is empty, vulnreport will derive fix links from the report's
References field and use those to find vulnerable symbols.
Change-Id: Ibab159987b60d445707083885a56ce6dc0e45302
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/562236
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
* When creating or unexcluding a report, add a note to the report that
an error occurred in addition to logging a warning.
* When unexcluding reports, fix and lint the report before writing it.
* In Fix, attempt to add an advisory if the report needs one.
All of these changes are intended to make it easier to batch create &
unexclude reports, and notice when there are issues in the automated
process.
Change-Id: I33ec94a619cf0b89112a682386439a11381b6c57
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/562246
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Refactor logging and add colors.
Colors are on by default but can be turned off with
`vulnreport -color=false [subcommand]...`, or by setting
the environment variable `NO_COLOR`.
Change-Id: Iaea30dfeb045b2b690683f257cf49676c785951a
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/562555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Split Fix into FixText (which doesn't require a proxy client) and
FixVersions (which does), and only call the FixText portion from
applySuggestion in vulnreport. This avoids a possible nil pointer
dereference.
Change-Id: Ic57cb0738ac64ce892e62c2e16c76efb25c22147
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/562245
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Add flag "symbols" which causes vulnreport create to attempt to automatically
add symbols to a newly created report.
Change-Id: I0ba9e4fc1365fd05f06915e82d6b33ab9528fbc2
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/560779
Reviewed-by: Maceo Thompson <maceothompson@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Move the logic (but don't modify it) to populate symbols to its own file in
internal/symbols. Add some basic tests that confirm the current behavior
(which will likely be tweaked in follow up CLs).
Change-Id: I10593154c343adb680733ebd66a4dd97abed2c43
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/560778
Reviewed-by: Maceo Thompson <maceothompson@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Adds a new unified framework for vulnreport subcommands. Each subcommand
now implements an interface, command, which has functions for setting up,
running, and tearing down the command.
Change-Id: I7c6ab5cf1b4c19b300dbdef6df0d76e2f1f303ea
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/559955
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Command `vulnreport unexclude` converts an excluded report to a
regular report. Will be used as part of an experiment to add back reports
that were previously excluded from vulndb.
Change-Id: I420a8262144fe6b0f459cb1a8438062931a15700
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/559815
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Zvonimir Pavlinovic <zpavlinovic@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add a command, `issue placeholder` which can be used to publish a
placeholder issue for a CVE.
Example usage: `issue placeholder CVE-1234-5678 CVE-0000-1111` would
create two issues on the x/vulndb tracker, one for each CVE.
The placeholder issue does not reveal anything about the CVE, and the
command is intended to be used to create tracking issues for CVEs that
have been preannounced but not yet published.
Change-Id: I95ace0eaffe83f77ebc58d4ec755f0276e748c02
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/559601
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Move all logic for vulnreport logging into a new inner package, log, which
will allow us to more easily change the logging internals (e.g., if we
want to use slog in the future).
Change-Id: I8287fc186451e6dc2e846ed071bd3c6fefdad359
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/559600
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Unlike the PaLM API, which could generate multiple candidate responses
at once, the Gemini API only gives one response at a time (at least for
now).
To accommodate this, modify suggest to make a separate request to the
API to generate each suggestion.
Change-Id: I5c3adce302abdb5228aef7a8a94d8b3ee5881371
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/559599
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
The Gemini API has fully replaced the PaLM API so there is no need for
us to maintain support.
Change-Id: I1e6581313b481a7ce9042d5ef82e99cb45ee48cf
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/559598
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add flag "-ai" that causes `vulnreport create` to attempt to auto-populate
an AI-generated draft of a summary and description for a report.
(This functionality can still be accessed post-create via command
`vulnreport suggest`.)
Change-Id: I0a22579b56fa1fb5c302d54afebea96fdcc5504a
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/559597
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The main.go file was becoming somewhat hard to navigate, so this change
moves most of its code to separate files, organized by the subcommand
the code is primarily associated with. This change only moves code and
does not modify its behavior.
Change-Id: I684cda7e2e65eb8a043e8c04fdeda187420a95f0
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/559596
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
This change integrates symbols.Patched into vulnreport.
One can call `vulnreport symbols [REPORTNUMBER]` to use this.
Additionally, if vulnreport is unable to find the correct fix link (this
may happen in cases where there are multiple modules or the report's fix
link is a pull request and not a commit hash), they can populate the
FixLink field per-module.
Change-Id: I6566c67538c4de29547432648a28f50cb735b3ce
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/559156
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In the deploy script, additionally publish the Go vulnerability database
as a zip file.
This will allow clients to easily download the database and, for example,
run govulncheck offline.
Once deployed, this new endpoint will be accessible via
https://vuln.go.dev/vulndb.zip
Change-Id: I8e5f3c4e957d3d980e692ac6ad2ca7afefc42086
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/557895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Zvonimir Pavlinovic <zpavlinovic@google.com>
In vulnreport create, allow creation of a basic report if no CVE/GHSA
is provided.
As an unrelated small fix, don't print an empty log message if
there are no xrefs for a created report.
Change-Id: I10df39118f245f1cf898ecf4b8f6bbd4585497be
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/557075
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Tatiana Bradley
Maceo Thompson