When creating unreviewed reports, automatically remove references
that do not exist.
Also remove package-level data, as it can cause false-negatives
if it is not correct.
(For reviewed reports, we preserve these pieces of info as a human
will review them and manually determine if they are useful).
Change-Id: I2ff6bde62320d2f56f9d5a67ef438f4cafbaf6e5
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/591200
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
If a package name doesn't have any slashes, it is probably not useful,
so remove it when creating a report.
Change-Id: I5ad653f0870853ad992b4162a115b94ce703c2e1
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/591198
Reviewed-by: Damien Neil <dneil@google.com>
Commit-Queue: Tatiana Bradley <tatianabradley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In test cases, remember the lints that would apply to each generated
report. This makes it more clear which reports have errors that would
need to be fixed manually in order to submit, and gives us a starting
point from which to automatically address these lints.
Change-Id: Ifdf3f60edd7606d87b507c6b6ff147c19ad894a2
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/591197
Commit-Queue: Tatiana Bradley <tatianabradley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Add some more test cases for real CVEs that reveal issues
with our report creation process (e.g., preserving package names
that are malformed, dropping reference metadata, and being unable
to recognize a vuln that doesn't actually affect Go.)
This gives us a starting point from which to fix these issues.
(Note that this CL also updates all the test cases to the latest
available CVE content and updates the proxy responses, so there are
some updates to test cases unrelated to this change.)
Change-Id: Id4d8c87021b9077f042c0029ec7f1655f871df46
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/591196
Commit-Queue: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Modify the prioritization algorithm so that it only considers the number
of reviewed vs. likely-binary reports, instead of comparing the number
of excluded vs. non-excluded reports. This means that the number of unreviewed
reports does not (currently) affect the prioritization result.
In addition, separate the code used to prioritize modules (for vulnreport triage)
into its own package so it can be tested in isolation.
Also add a basic command line tool "priority" that can be used to find the
priority result for a module directly.
Change-Id: Ic7ebe76d8f5091f56bc3eb65a5064391136b2064
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/591195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Command vulnreport regen regenerates a report based on
the latest version of the source.
Intended for UNREVIEWED reports.
Use the following command to regenerate all unreviewed
reports:
$ vulnreport regen data/reports/*.yaml
Change-Id: I3f956fde473b8375bd523049118d8f6817aad9ae
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/590856
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The vulndb worker was timing out due to a restrictive rate limit
for requests to pkgsite. Relax this rate limit as a quick fix.
Change-Id: I1938c0e4182c476ac8d9472283534ace5681706a
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/591435
Auto-Submit: Tatiana Bradley <tatianabradley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Fix a case where the worker refuses to restart if it thinks
(incorrectly) that a previous update has not yet finished. (Because
the previous update errored).
Do this by populating the EndedAt field of the UpdateRecord in
cveUpdater.update on error as well as success.
Change-Id: Ibc98f23b51da21397612479d539dfefe906acb8c
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/589996
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Tatiana Bradley <tatianabradley@google.com>
Modify the corpus-wide report lint check to allow unreviewed reports
to have the same summary as other (reviewed or unreviwed) reports.
Reviewed reports must still have unique summaries (but may share a summary
with one or more unreviewed reports).
Change-Id: I8ab4fc259e019c0fb529ed0ef332cc9cfe634483
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/590279
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Add a lint check to ensure that original reports created
by the Go team are always marked REVIEWED.
Change-Id: I5d72998be1597e42ec5ae5e05d4a5d9a4324cb40
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/590276
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Fix bug in which excluded reports would be published to the wrong
folder. This was accidentally introduced in a recent refactor
and no reports were actually published to the wrong folder (this would
have been caught before submit if it had occurred).
Change-Id: Icc9eb99b2ceb185310e99eaa39e45072e0ae6c80
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/590280
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Add tests to check contents of the update record and for error
cases. This is to prep for a change to this function's behavior.
Change-Id: I9380f661725aa4a50db0691906d3d6a5a925f8d1
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/589995
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Remove error return value from xref, which always returns nil.
(Caught by unparam and blocking deploy of vulndb)
Change-Id: I4c9423f0d333d7beb9422ee558ed83f3dd99aebf
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/590115
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Tatiana Bradley <tatianabradley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
With no arguments, "vulnreport commit" now individually commits all
reports that have been added/changed (according to git status).
(To commit them all as a single commit, use "vulnreport -batch commit").
The flag "-status=<REVIEW_STATUS>" can additionally be used
to only commit reports with a certain review status.
Change-Id: I4efb4e866166b6153d556409408021dc861656fb
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/590035
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Adds a check to "vulnreport fix" that errors if any URLs in the
"references" section return an error or status 404 on HTTP HEAD.
We don't check for other status codes yet.
An experiment to error on all non-200 status codes brought up some
ambiguous cases where the link is still viewable in a browser, e.g.:
- 429 Too Many Requests (https://vuldb.com/?id.256304)
- 503 Service Unavailable (http://blog.recurity-labs.com/2017-08-10/scm-vulns):
- 403 Forbidden (https://www.sap.com/documents/2022/02/fa865ea4-167e-0010-bca6-c68f7e60039b.html)
For now, this is a fix check and not a lint check, meaning it only
applies to new reports, and can technically be ignored (by manually
creating a CL that adds the report).
This CL also deletes existing URLs in the corpus that don't exist
according to this check.
Change-Id: Id14fb79fc2f2c2d4c8145fdc88d11aa33708c94b
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/588761
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Command vulnreport create now decides whether to generate a REVIEWED
or UNREVIEWED report based on issue's labels.
This can be overridden with flag "-status=<REVIEW_STATUS>". The "unreviewed"
flag is removed.
Change-Id: I8f8b808c6f9bbcaeb0dc176fb6cb875b8f9ccee4
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/587976
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Adds the beginnings of a guide that can be used as a
reference for the current vulndb triager, in combination
with the older triage guide.
The goal is to document new features / processes
so we can get started with experimenting with them
without needing to overhaul the whole guide before we
have worked out the final process.
Change-Id: Iad8256414fda78ebbdbfc44776a46786cbbb034c
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/587975
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Update the vulnreport triage command to label any issues it triages
as "triaged", and skip issues that are already labeled as such.
(The flag "-f" overrides the skip).
This allows the triager to simply run "vulnreport triage" to triage all
untriaged issues.
Change-Id: I24611166d972c46100b1e8fd38bed1fb87071d11
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/587915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Zvonimir Pavlinovic <zpavlinovic@google.com>
This is part of computation of the name of the function in vuln. db
format. For "func (A[T1, T1, T3]) Foo", the name should be "A.Foo".
Change-Id: Iaeb7eccf3d72504484e697de972297ff37481255
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/586719
Run-TryBot: Zvonimir Pavlinovic <zpavlinovic@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Maceo Thompson <maceothompson@google.com>
Add the first UNREVIEWED report.
- data/reports/GO-2024-2864.yaml
Fixesgolang/vulndb#2864
Change-Id: Ib67d84b1da34f0a9ede9af69fdef084efa44db17
Reviewed-on: https://go-review.googlesource.com/c/vulndb/+/586295
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Tatiana Bradley
Gopher Robot
Zvonimir Pavlinovic