Go analysis support for CodeQL.

This commit is contained in:
Max Schaefer 2019-11-08 12:14:43 +00:00
Коммит d14eb855fc
583 изменённых файлов: 52850 добавлений и 0 удалений

3
.codeqlmanifest.json Normal file
Просмотреть файл

@ -0,0 +1,3 @@
{ "provide": [ "ql/src/qlpack.yml",
"ql/config/legacy-support/qlpack.yml" ],
"ignore": [ "the-extractor-which-needs-to-be-built" ] }

21
.gitignore поставляемый Normal file
Просмотреть файл

@ -0,0 +1,21 @@
# editor and OS artifacts
*~
.DS_STORE
# query compilation caches
.cache
# build artifacts
build/*
# qltest projects and artifacts
ql/test/**/*.testproj
ql/test/**/*.actual
ql/test/**/go.sum
# Java class files
**/*.class
# binaries
tools/bin
tools/tokenizer.jar

76
CODE_OF_CONDUCT.md Normal file
Просмотреть файл

@ -0,0 +1,76 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to make participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies within all project spaces, and it also applies when
an individual is representing the project or its community in public spaces.
Examples of representing a project or community include using an official
project e-mail address, posting via an official social media account, or acting
as an appointed representative at an online or offline event. Representation of
a project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at opensource@github.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq

44
CONTRIBUTING.md Normal file
Просмотреть файл

@ -0,0 +1,44 @@
## Contributing
Hi there! We're thrilled that you'd like to contribute to this project. Your help is essential for keeping it great.
Contributions to this project are [released](https://help.github.com/articles/github-terms-of-service/#6-contributions-under-repository-license) to the public under the [project's open source license](LICENSE).
Please note that this project is released with a [Contributor Code of Conduct][CODE_OF_CONDUCT.md]. By participating in this project you agree to abide by its terms.
## Adding a new query
If you have an idea for a query that you would like to share with other CodeQL users, please open a pull request to add it to this repository.
Follow the steps below to help other users understand what your query does, and to ensure that your query is consistent with the other CodeQL queries.
1. **Consult the documentation for query writers**
There is lots of useful documentation to help you write CodeQL queries, ranging from information about query file structure to language-specific tutorials. For more information on the documentation available, see [Writing QL queries](https://help.semmle.com/QL/learn-ql/writing-queries/writing-queries.html) on [help.semmle.com](https://help.semmle.com).
2. **Format your code correctly**
All of the standard CodeQL queries and libraries are uniformly formatted for clarity and consistency, so we strongly recommend that all contributions follow the same formatting guidelines. If you use the CodeQL extension for Visual Studio Code, you can auto-format your query in the [QL editor](https://help.semmle.com/ql-for-eclipse/Content/WebHelp/ql-editor.html). For more information, see the [QL style guide](https://github.com/Semmle/ql/blob/master/docs/ql-style-guide.md).
3. **Make sure your query has the correct metadata**
Query metadata is used by Semmle's analysis to identify your query and make sure the query results are displayed properly.
The most important metadata to include are the `@name`, `@description`, and the `@kind`.
Other metadata properties (`@precision`, `@severity`, and `@tags`) are usually added after the query has been reviewed by the maintainers.
For more information on writing query metadata, see the [Query metadata style guide](https://github.com/Semmle/ql/blob/master/docs/query-metadata-style-guide.md).
4. **Make sure the `select` statement is compatible with the query type**
The `select` statement of your query must be compatible with the query type (determined by the `@kind` metadata property) for alert or path results to be displayed correctly in LGTM and Visual Studio Code.
For more information on `select` statement format, see [Introduction to query files](https://help.semmle.com/QL/learn-ql/writing-queries/introduction-to-queries.html#select-clause) on help.semmle.com.
5. **Write a query help file**
Query help files explain the purpose of your query to other users. Write your query help in a `.qhelp` file and save it in the same directory as your new query.
For more information on writing query help, see the [Query help style guide](https://github.com/Semmle/ql/blob/master/docs/query-help-style-guide.md).
## Resources
- [How to Contribute to Open Source](https://opensource.guide/how-to-contribute/)
- [Using Pull Requests](https://help.github.com/articles/about-pull-requests/)
- [GitHub Help](https://help.github.com)
- [A Note About Git Commit Messages](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html)

13
COPYRIGHT Normal file
Просмотреть файл

@ -0,0 +1,13 @@
Copyright (c) Semmle Inc and other contributors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License. You may obtain a copy of the
License at http://www.apache.org/licenses/LICENSE-2.0
THIS CODE IS PROVIDED ON AN *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED
WARRANTIES OR CONDITIONS OF TITLE, FITNESS FOR A PARTICULAR PURPOSE,
MERCHANTABLITY OR NON-INFRINGEMENT.
See the Apache Version 2.0 License for specific language governing permissions
and limitations under the License.

21
LICENSE Normal file
Просмотреть файл

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 GitHub
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

61
Makefile Normal file
Просмотреть файл

@ -0,0 +1,61 @@
all: tools ql/src/go.dbscheme
ifeq ($(OS),Windows_NT)
EXE = .exe
else
EXE =
endif
.PHONY: tools
tools: tools/bin/go-extractor$(EXE) tools/bin/go-tokenizer$(EXE) tools/bin/go-autobuilder$(EXE) tools/tokenizer.jar tools/bin/go-bootstrap$(EXE)
tools/bin/go-extractor$(EXE): FORCE
go build -mod=vendor -o $@ ./extractor/cli/go-extractor
tools/bin/go-tokenizer$(EXE): FORCE
go build -mod=vendor -o $@ ./extractor/cli/go-tokenizer
tools/bin/go-autobuilder$(EXE): FORCE
go build -mod=vendor -o $@ ./extractor/cli/go-autobuilder
tools/bin/go-bootstrap$(EXE): FORCE
go build -mod=vendor -o $@ ./extractor/cli/go-bootstrap
FORCE:
tools/tokenizer.jar: tools/net/sourceforge/pmd/cpd/GoLanguage.class
jar cf $@ -C tools net
jar uf $@ -C tools opencsv
tools/net/sourceforge/pmd/cpd/GoLanguage.class: extractor/net/sourceforge/pmd/cpd/GoLanguage.java
javac -cp extractor -d tools $^
rm tools/net/sourceforge/pmd/cpd/AbstractLanguage.class
rm tools/net/sourceforge/pmd/cpd/SourceCode.class
rm tools/net/sourceforge/pmd/cpd/TokenEntry.class
rm tools/net/sourceforge/pmd/cpd/Tokenizer.class
ql/src/go.dbscheme: tools/bin/go-extractor$(EXE)
env TRAP_FOLDER=/tmp tools/bin/go-extractor --dbscheme $@
ql/src/go.dbscheme.stats: ql/src/go.dbscheme
odasa createProject --force --template templates/project --threads 4 \
--variable repository https://github.com/golang/tools \
--variable revision 6e04913c \
--variable SEMMLE_REPO_URL golang.org/x/tools \
build/stats-project
odasa addSnapshot --latest --overwrite --name revision --project build/stats-project
odasa buildSnapshot --latest --project build/stats-project
odasa collectStats --dbscheme $^ --db build/stats-project/revision/working/db-go --outputFile $@
test: all build/testdb/check-upgrade-path
odasa qltest --language go --library ql/src ql/test
cd extractor; go test -mod=vendor ./... | grep -vF "[no test files]"
.PHONY: build/testdb/check-upgrade-path
build/testdb/check-upgrade-path : build/testdb/go.dbscheme ql/src/go.dbscheme
odasa upgradeDatabase --db build/testdb --upgrade-packs upgrades
diff -q build/testdb/go.dbscheme ql/src/go.dbscheme
build/testdb/go.dbscheme: upgrades/initial/go.dbscheme
echo >build/empty.trap
odasa cli --dbscheme upgrades/initial/go.dbscheme --import build/empty.trap --db build/testdb

45
README.md Normal file
Просмотреть файл

@ -0,0 +1,45 @@
# Go analysis support for CodeQL
This open-source repository contains the extractor, CodeQL libraries, and queries that power Go
support in [LGTM](https://lgtm.com), CodeQL, and other Semmle products.
It contains two major components:
- an extractor, itself written in Go, that parses Go source code and converts it into a database
that can be queried using CodeQL.
- static analysis libraries and queries written in [QL](https://help.semmle.com/QL) that can be
used to analyze such a database to find coding mistakes or security vulnerabilities.
The goal of this project is to provide comprehensive static analysis support for Go in CodeQL.
## Installation
Simply clone this repository. There are no external dependencies.
If you want to use the CodeQL extension for Visual Studio Code, import this repository into your VS
Code workspace.
## Usage
To analyze a Go codebase, either use the CodeQL command-line interface to create a database
yourself, or download a pre-built database from LGTM.com. You can then run any of the queries
contained in this repository either on the command line or using the VS Code extension.
Note that the [lgtm.com](https://github.com/github/codeql-go/tree/lgtm.com) branch of this
repository corresponds to the version of the queries that is currently deployed on LGTM.com.
The [master](https://github.com/github/codeql-go/tree/master) branch may contain changes that
have not been deployed yet, so you may need to upgrade databases downloaded from LGTM.com before
running queries on them.
## Contributions
Contributions are welcome! Please see our [contribution guidelines](CONTRIBUTING.md) and our
[code of conduct](CODE_OF_CONDUCT.md) for details on how to participate in our community.
## Licensing
The code in this repository is licensed under the [MIT license](LICENSE).
## Resources
- [Writing CodeQL queries](https://help.semmle.com/QL/learn-ql/ql/writing-queries/writing-queries.html)
- [Learning CodeQL](https://help.semmle.com/QL/learn-ql/index.html)

3
SECURITY.md Normal file
Просмотреть файл

@ -0,0 +1,3 @@
If you discover a security issue in this repo, please submit it through the [GitHub Security Bug Bounty](https://hackerone.com/github).
Thanks for helping make CodeQL safe for everyone.

Просмотреть файл

@ -0,0 +1,3 @@
precision = ("veryhigh", "high", "medium", "low")
severity = ("error", "warning", "recommendation")
security = ("true", "false")

0
build/.gitkeep Normal file
Просмотреть файл

16
codeql-extractor.yml Normal file
Просмотреть файл

@ -0,0 +1,16 @@
name: "go"
display_name: "Go"
version: 0.1.0
pull_request_triggers:
- "**/go.mod"
- "**/glide.yaml"
- "**/Gopkg.toml"
column_kind: "utf8"
extra_env_vars:
SOURCE_ARCHIVE: ${env.CODEQL_EXTRACTOR_GO_SOURCE_ARCHIVE_DIR}
TRAP_FOLDER: ${env.CODEQL_EXTRACTOR_GO_TRAP_DIR}
file_types:
- name: go
display_name: Go
extensions:
- .go

Просмотреть файл

@ -0,0 +1,10 @@
@echo off
SETLOCAL EnableDelayedExpansion
rem Some legacy environment variables for the autobuilder.
set LGTM_SRC=%CD%
type NUL && "%CODEQL_EXTRACTOR_GO_ROOT%/tools/%CODEQL_PLATFORM%/go-autobuilder.exe"
exit /b %ERRORLEVEL%
ENDLOCAL

14
codeql-tools/autobuild.sh Executable file
Просмотреть файл

@ -0,0 +1,14 @@
#!/bin/sh
set -eu
if [ "$CODEQL_PLATFORM" != "linux64" ] && [ "$CODEQL_PLATFORM" != "osx64" ] ; then
echo "Automatic build detection for $CODEQL_PLATFORM is not implemented."
exit 1
fi
# Some legacy environment variables used by the autobuilder.
LGTM_SRC="$(pwd)"
export LGTM_SRC
"$CODEQL_EXTRACTOR_GO_ROOT/tools/$CODEQL_PLATFORM/go-autobuilder"

Просмотреть файл

@ -0,0 +1,350 @@
package main
import (
"fmt"
"io/ioutil"
"log"
"net/url"
"os"
"os/exec"
"path/filepath"
"regexp"
"runtime"
"strings"
)
func usage() {
fmt.Fprintf(os.Stderr,
`%s is a wrapper script that installs dependencies and calls the extractor.
When LGTM_SRC is not set, the script installs dependencies as described below, and then invokes the
extractor in the working directory.
If LGTM_SRC is set, it checks for the presence of the files 'go.mod', 'Gopkg.toml', and
'glide.yaml' to determine how to install dependencies: if a 'Gopkg.toml' file is present, it uses
'dep ensure', if there is a 'glide.yaml' it uses 'glide install', and otherwise 'go get'.
Additionally, unless a 'go.mod' file is detected, it sets up a temporary GOPATH and moves all
source files into a folder corresponding to the package's import path before installing
dependencies.
This behavior can be further customized using environment variables: setting LGTM_INDEX_NEED_GOPATH
to 'false' disables the GOPATH set-up, LGTM_INDEX_BUILD_COMMAND can be set to a newline-separated
list of commands to run in order to install dependencies, and LGTM_INDEX_IMPORT_PATH can be used to override the package import path, which is otherwise inferred from the SEMMLE_REPO_URL environment
variable.
`,
os.Args[0])
fmt.Fprintf(os.Stderr, "Usage:\n\n %s\n", os.Args[0])
}
func fileExists(filename string) bool {
_, err := os.Stat(filename)
if err != nil && !os.IsNotExist(err) {
log.Printf("Unable to stat %s: %s\n", filename, err.Error())
}
return err == nil
}
func getImportPath() (importpath string) {
importpath = os.Getenv("LGTM_INDEX_IMPORT_PATH")
if importpath == "" {
repourl := os.Getenv("SEMMLE_REPO_URL")
if repourl == "" {
return ""
}
importpath = getImportPathFromRepoURL(repourl)
}
log.Printf("Import path is %s\n", importpath)
return
}
func getImportPathFromRepoURL(repourl string) string {
// check for scp-like URL as in "git@github.com:Semmle/go.git"
shorturl := regexp.MustCompile("^([^@]+@)?([^:]+):([^/].*?)(\\.git)?$")
m := shorturl.FindStringSubmatch(repourl)
if m != nil {
return m[2] + "/" + m[3]
}
// otherwise parse as proper URL
u, err := url.Parse(repourl)
if err != nil {
log.Fatalf("Malformed repository URL %s.\n", repourl)
}
host := u.Hostname()
path := u.Path
// strip off leading slashes and trailing `.git` if present
path = regexp.MustCompile("^/+|\\.git$").ReplaceAllString(path, "")
return host + "/" + path
}
// DependencyInstallerMode is an enum describing how dependencies should be installed
type DependencyInstallerMode int
const (
// GoGetNoModules represents dependency installation using `go get` without modules
GoGetNoModules DependencyInstallerMode = iota
// GoGetWithModules represents dependency installation using `go get` with modules
GoGetWithModules
// Dep represent dependency installation using `dep ensure`
Dep
// Glide represents dependency installation using `glide install`
Glide
)
func main() {
if len(os.Args) > 1 {
usage()
os.Exit(2)
}
srcdir := os.Getenv("LGTM_SRC")
inLGTM := srcdir != ""
if inLGTM {
log.Printf("LGTM_SRC is %s\n", srcdir)
} else {
cwd, err := os.Getwd()
if err != nil {
log.Fatalln("Failed to get current working directory.")
}
log.Printf("LGTM_SRC is not set; defaulting to current working directory %s\n", cwd)
srcdir = cwd
}
// we set `SEMMLE_PATH_TRANSFORMER` ourselves in some cases, so blank it out first for consistency
os.Setenv("SEMMLE_PATH_TRANSFORMER", "")
// determine how to install dependencies and whether a GOPATH needs to be set up before
// extraction
depMode := GoGetNoModules
needGopath := true
if fileExists("go.mod") {
depMode = GoGetWithModules
needGopath = false
log.Println("Found go.mod, enabling go modules")
} else if fileExists("Gopkg.toml") {
depMode = Dep
log.Println("Found Gopkg.toml, using dep instead of go get")
} else if fileExists("glide.yaml") {
depMode = Glide
log.Println("Found glide.yaml, enabling go modules")
}
// if `LGTM_INDEX_NEED_GOPATH` is set, it overrides the value for `needGopath` inferred above
if needGopathOverride := os.Getenv("LGTM_INDEX_NEED_GOPATH"); needGopathOverride != "" {
inLGTM = true
if needGopathOverride == "true" {
needGopath = true
} else if needGopathOverride == "false" {
needGopath = false
} else {
log.Fatalf("Unexpected value for Boolean environment variable LGTM_NEED_GOPATH: %v.\n", needGopathOverride)
}
}
importpath := getImportPath()
if needGopath && importpath == "" {
log.Printf("Failed to determine import path, not setting up GOPATH")
needGopath = false
}
if inLGTM && needGopath {
// a temporary directory where everything is moved while the correct
// directory structure is created.
scratch, err := ioutil.TempDir(srcdir, "scratch")
if err != nil {
log.Fatalf("Failed to create temporary directory %s in directory %s: %s\n",
scratch, srcdir, err.Error())
}
log.Printf("Temporary directory is %s.\n", scratch)
// move all files in `srcdir` to `scratch`
dir, err := os.Open(srcdir)
if err != nil {
log.Fatalf("Failed to open source directory %s for reading: %s\n", srcdir, err.Error())
}
files, err := dir.Readdirnames(-1)
if err != nil {
log.Fatalf("Failed to read source directory %s: %s\n", srcdir, err.Error())
}
for _, file := range files {
if file != filepath.Base(scratch) {
log.Printf("Moving %s/%s to %s/%s.\n", srcdir, file, scratch, file)
err := os.Rename(filepath.Join(srcdir, file), filepath.Join(scratch, file))
if err != nil {
log.Fatalf("Failed to move file %s to the temporary directory: %s\n", file, err.Error())
}
}
}
// create a new folder which we will add to GOPATH below
root := filepath.Join(srcdir, "root")
// move source files to where Go expects them to be
newdir := filepath.Join(root, "src", importpath)
err = os.MkdirAll(filepath.Dir(newdir), 0755)
if err != nil {
log.Fatalf("Failed to create directory %s: %s\n", newdir, err.Error())
}
log.Printf("Moving %s to %s.\n", scratch, newdir)
err = os.Rename(scratch, newdir)
if err != nil {
log.Fatalf("Failed to rename %s to %s: %s\n", scratch, newdir, err.Error())
}
err = os.Chdir(newdir)
if err != nil {
log.Fatalf("Failed to chdir into %s: %s\n", newdir, err.Error())
}
// set up SEMMLE_PATH_TRANSFORMER to ensure paths in the source archive and the snapshot
// match the original source location, not the location we moved it to
pt, err := ioutil.TempFile("", "path-transformer")
if err != nil {
log.Fatalf("Unable to create path transformer file: %s.", err.Error())
}
defer os.Remove(pt.Name())
_, err = pt.WriteString("#" + srcdir + "\n" + newdir + "//\n")
if err != nil {
log.Fatalf("Unable to write path transformer file: %s.", err.Error())
}
err = pt.Close()
if err != nil {
log.Fatalf("Unable to close path transformer file: %s.", err.Error())
}
err = os.Setenv("SEMMLE_PATH_TRANSFORMER", pt.Name())
if err != nil {
log.Fatalf("Unable to set SEMMLE_PATH_TRANSFORMER environment variable: %s.\n", err.Error())
}
// set/extend GOPATH
oldGopath := os.Getenv("GOPATH")
var newGopath string
if oldGopath != "" {
newGopath = strings.Join(
[]string{root, oldGopath},
string(os.PathListSeparator),
)
} else {
newGopath = root
}
err = os.Setenv("GOPATH", newGopath)
if err != nil {
log.Fatalf("Unable to set GOPATH to %s: %s\n", newGopath, err.Error())
}
log.Printf("GOPATH set to %s.\n", newGopath)
}
// install dependencies
inst := os.Getenv("LGTM_INDEX_BUILD_COMMAND")
var install *exec.Cmd
if inst == "" {
// automatically determine command to install dependencies
if depMode == Dep {
// set up the dep cache if SEMMLE_CACHE is set
cacheDir := os.Getenv("SEMMLE_CACHE")
if cacheDir != "" {
depCacheDir := filepath.Join(cacheDir, "go", "dep")
log.Printf("Attempting to create dep cache dir %s\n", depCacheDir)
err := os.MkdirAll(depCacheDir, 0755)
if err != nil {
log.Printf("Failed to create dep cache directory: %s\n", err.Error())
} else {
log.Printf("Setting dep cache directory to %s\n", depCacheDir)
err = os.Setenv("DEPCACHEDIR", depCacheDir)
if err != nil {
log.Println("Failed to set dep cache directory")
} else {
err = os.Setenv("DEPCACHEAGE", "720h") // 30 days
if err != nil {
log.Println("Failed to set dep cache age")
}
}
}
}
if fileExists("Gopkg.lock") {
// if Gopkg.lock exists, don't update it and only vendor dependencies
install = exec.Command("dep", "ensure", "-v", "-vendor-only")
} else {
install = exec.Command("dep", "ensure", "-v")
}
log.Println("Installing dependencies using `dep ensure`.")
} else if depMode == Glide {
install = exec.Command("glide", "install")
log.Println("Installing dependencies using `glide install`")
} else {
if depMode == GoGetWithModules {
// enable go modules if used
os.Setenv("GO111MODULE", "on")
}
// get dependencies
install = exec.Command("go", "get", "-v", "./...")
log.Println("Installing dependencies using `go get -v ./...`.")
}
} else {
// write custom build commands into a script, then run it
var (
ext = ""
header = ""
footer = ""
)
if runtime.GOOS == "windows" {
ext = ".cmd"
header = "@echo on\n@prompt +$S\n"
footer = "\nIF %ERRORLEVEL% NEQ 0 EXIT"
} else {
ext = ".sh"
header = "#! /bin/bash\nset -xe +u\n"
}
script, err := ioutil.TempFile("", "go-build-command-*"+ext)
if err != nil {
log.Fatalf("Unable to create temporary script holding custom build commands: %s\n", err.Error())
}
defer os.Remove(script.Name())
_, err = script.WriteString(header + inst + footer)
if err != nil {
log.Fatalf("Unable to write to temporary script holding custom build commands: %s\n", err.Error())
}
err = script.Close()
if err != nil {
log.Fatalf("Unable to close temporary script holding custom build commands: %s\n", err.Error())
}
os.Chmod(script.Name(), 0700)
install = exec.Command(script.Name())
log.Println("Installing dependencies using custom build command.")
}
if install != nil {
install.Stdout = os.Stdout
install.Stderr = os.Stderr
err := install.Run()
if err != nil {
log.Printf("Installation of dependencies failed, continuing anyway: %s\n", err.Error())
}
}
// extract
mypath, err := os.Executable()
if err != nil {
log.Fatalf("Could not determine path of autobuilder: %v.\n", err)
}
extractor := filepath.Join(filepath.Dir(mypath), "go-extractor")
if runtime.GOOS == "windows" {
extractor = extractor + ".exe"
}
cwd, err := os.Getwd()
if err != nil {
log.Fatalf("Unable to determine current directory: %s\n", err.Error())
}
log.Printf("Running extractor command '%s ./...' from directory '%s'.\n", extractor, cwd)
cmd := exec.Command(extractor, "./...")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
err = cmd.Run()
if err != nil {
log.Fatalf("Extraction failed: %s\n", err.Error())
}
}

Просмотреть файл

@ -0,0 +1,19 @@
package main
import "testing"
func TestGetImportPathFromRepoURL(t *testing.T) {
tests := map[string]string{
"git@github.com:Semmle/go.git": "github.com/Semmle/go",
"git@github.com:Semmle/go": "github.com/Semmle/go",
"https://github.com/Semmle/go.git": "github.com/Semmle/go",
"https://github.com:12345/Semmle/go": "github.com/Semmle/go",
"gitolite@some.url:some/repo": "some.url/some/repo",
}
for input, expected := range tests {
actual := getImportPathFromRepoURL(input)
if actual != expected {
t.Errorf("Expected getImportPathFromRepoURL(\"%s\") to be \"%s\", but got \"%s\".", input, expected, actual)
}
}
}

Просмотреть файл

@ -0,0 +1,51 @@
package main
import (
"fmt"
"io/ioutil"
"log"
"os"
"regexp"
)
// A utility program for generating `project` and `variable` files for SemmleCore Go projects
//
// This program should not normally be run directly; it is usually executed as part of
// `odasa bootstrap`, and expects two files as arguments: a (partial) `variables` file and
// an empty file to be filled in with an `<autoupdate>` element containing build steps.
//
// The `variables` file is extended with a definition of `LGTM_SRC` and, if it defines the
// `repository` variable, `SEMMLE_REPO_URL`. The only build step is an invocation of the
// Go autobuilder.
func main() {
vars := os.Args[1]
buildSteps := os.Args[2]
haveRepo := false
content, err := ioutil.ReadFile(vars)
if err != nil {
log.Fatal(err)
}
re := regexp.MustCompile(`(^|\n)repository=`)
haveRepo = re.Find(content) != nil
additionalVars := "LGTM_SRC=${src}\n"
if haveRepo {
additionalVars += "SEMMLE_REPO_URL=${repository}\n"
}
content = append(content, []byte(additionalVars)...)
err = ioutil.WriteFile(vars, content, 0644)
if err != nil {
log.Fatal(err)
}
export := "LGTM_SRC"
if haveRepo {
export += ",SEMMLE_REPO_URL"
}
content = []byte(fmt.Sprintf(`<autoupdate>
<build export="%s">${semmle_dist}/language-packs/go/tools/platform/${semmle_platform}/bin/go-autobuilder</build>
</autoupdate>
`, export))
err = ioutil.WriteFile(buildSteps, content, 0644)
}

Просмотреть файл

@ -0,0 +1,71 @@
package main
import (
"fmt"
"log"
"os"
"strings"
"github.com/Semmle/go/extractor/dbscheme"
"github.com/Semmle/go/extractor"
)
func usage() {
fmt.Fprintf(os.Stderr, "%s is a program for building a snapshot of a Go code base.\n\n", os.Args[0])
fmt.Fprintf(os.Stderr, "Usage:\n\n %s [<flag>...] [<buildflag>...] [--] <file>...\n\n", os.Args[0])
fmt.Fprintf(os.Stderr, "Flags:\n\n")
fmt.Fprintf(os.Stderr, "--help Print this help.\n")
fmt.Fprintf(os.Stderr, "--dbscheme string Write dbscheme to this file.\n")
}
func parseFlags(args []string) ([]string, []string, string) {
i := 0
var dumpDbscheme string
buildFlags := []string{}
for i < len(args) && strings.HasPrefix(args[i], "-") {
if args[i] == "--" {
i += 1
break
}
if strings.HasPrefix(args[i], "--dbscheme=") {
dumpDbscheme = strings.TrimPrefix(args[i], "--dbscheme=")
} else if args[i] == "--dbscheme" {
i += 1
dumpDbscheme = args[i]
} else if args[i] == "--help" {
usage()
os.Exit(0)
} else {
buildFlags = append(buildFlags, args[i])
}
i += 1
}
return buildFlags, args[i:], dumpDbscheme
}
func main() {
buildFlags, patterns, dumpDbscheme := parseFlags(os.Args[1:])
if dumpDbscheme != "" {
f, err := os.Create(dumpDbscheme)
if err != nil {
log.Fatalf("Unable to open file %s for writing.", dumpDbscheme)
}
dbscheme.PrintDbScheme(f)
f.Close()
log.Printf("Dbscheme written to file %s.", dumpDbscheme)
}
if len(patterns) == 0 {
log.Println("Nothing to extract.")
} else {
err := extractor.ExtractWithFlags(buildFlags, patterns)
if err != nil {
log.Fatal(err)
}
}
}

Просмотреть файл

@ -0,0 +1,57 @@
package main
import (
"encoding/csv"
"flag"
"fmt"
"go/scanner"
"go/token"
"io/ioutil"
"log"
"os"
"strings"
)
func main() {
flag.Parse()
fs := token.NewFileSet()
csv := csv.NewWriter(os.Stdout)
defer csv.Flush()
for _, fileName := range flag.Args() {
src, err := ioutil.ReadFile(fileName)
if err != nil {
log.Fatalf("Unable to read file %s.", fileName)
}
f := fs.AddFile(fileName, -1, len(src))
var s scanner.Scanner
s.Init(f, src, nil, 0)
for {
beginPos, tok, text := s.Scan()
if strings.TrimSpace(text) != "" {
var fuzzyText string
if tok.IsLiteral() {
fuzzyText = tok.String()
} else {
fuzzyText = text
}
endPos := f.Pos(f.Offset(beginPos) + len(text))
beginLine := fmt.Sprintf("%d", f.Position(beginPos).Line)
beginColumn := fmt.Sprintf("%d", f.Position(beginPos).Column)
endLine := fmt.Sprintf("%d", f.Position(endPos).Line)
endColumn := fmt.Sprintf("%d", f.Position(endPos).Column)
err = csv.Write([]string{text, fuzzyText, beginLine, beginColumn, endLine, endColumn})
if err != nil {
log.Fatalf("Unable to write CSV data: %v", err)
}
}
if tok == token.EOF {
break
}
}
}
}

Просмотреть файл

@ -0,0 +1,412 @@
package dbscheme
import (
"fmt"
"io"
"log"
"reflect"
"strings"
"github.com/Semmle/go/extractor/trap"
)
// A Type represents a database type
type Type interface {
def() string
ref() string
repr() string
valid(val interface{}) bool
}
// A PrimitiveType represents a primitive dataase type
type PrimitiveType int
const (
// INT represents the primitive database type `int`
INT PrimitiveType = iota
// FLOAT represents the primitive database type `float`
FLOAT
// BOOLEAN represents the primitive database type `boolean`
BOOLEAN
// DATE represents the primitive database type `date`
DATE
// STRING represents the primitive database type `string`
STRING
)
// A PrimaryKeyType represents a database type defined by a primary key column
type PrimaryKeyType struct {
name string
}
// A UnionType represents a database type defined as the union of other database types
type UnionType struct {
name string
components []Type
}
// An AliasType represents a database type which is an alias of another database type
type AliasType struct {
name string
underlying Type
}
// A CaseType represents a database type defined by a primary key column with a supplementary kind column
type CaseType struct {
base Type
column string
branches []*BranchType
}
// A BranchType represents one branch of a case type
type BranchType struct {
idx int
name string
}
func (pt PrimitiveType) def() string {
return ""
}
func (pt PrimitiveType) ref() string {
switch pt {
case INT:
return "int"
case FLOAT:
return "float"
case BOOLEAN:
return "boolean"
case DATE:
return "date"
case STRING:
return "string"
default:
panic(fmt.Sprintf("Unexpected primitive type %d", pt))
}
}
func (pt PrimitiveType) repr() string {
switch pt {
case INT:
return "int"
case FLOAT:
return "float"
case BOOLEAN:
return "boolean"
case DATE:
return "date"
case STRING:
return "string"
default:
panic(fmt.Sprintf("Unexpected primitive type %d", pt))
}
}
func (pt PrimitiveType) valid(value interface{}) bool {
switch value.(type) {
case int:
return pt == INT
case float64:
return pt == FLOAT
case bool:
return pt == BOOLEAN
case string:
return pt == STRING
}
return false
}
func (pkt PrimaryKeyType) def() string {
return ""
}
func (pkt PrimaryKeyType) ref() string {
return pkt.name
}
func (pkt PrimaryKeyType) repr() string {
return "int"
}
func (pkt PrimaryKeyType) valid(value interface{}) bool {
_, ok := value.(trap.Label)
return ok
}
func (ut UnionType) def() string {
var b strings.Builder
nl := 0
fmt.Fprintf(&b, "%s = ", ut.name)
for i, comp := range ut.components {
if i > 0 {
if i < len(ut.components)-1 && b.Len()-nl > 100 {
fmt.Fprintf(&b, "\n%s", strings.Repeat(" ", len(ut.name)))
nl = b.Len()
}
fmt.Fprint(&b, " | ")
}
fmt.Fprint(&b, comp.ref())
}
fmt.Fprint(&b, ";")
return b.String()
}
func (ut UnionType) ref() string {
return ut.name
}
func (ut UnionType) repr() string {
return "int"
}
func (ut UnionType) valid(value interface{}) bool {
_, ok := value.(trap.Label)
return ok
}
func (at AliasType) def() string {
return at.name + " = " + at.underlying.ref() + ";"
}
func (at AliasType) ref() string {
return at.name
}
func (at AliasType) repr() string {
return at.underlying.repr()
}
func (at AliasType) valid(value interface{}) bool {
return at.underlying.valid(value)
}
func (ct CaseType) def() string {
var b strings.Builder
fmt.Fprintf(&b, "case %s.%s of", ct.base.ref(), ct.column)
sep := " "
for _, branch := range ct.branches {
fmt.Fprintf(&b, "\n%s%s", sep, branch.def())
sep = "| "
}
fmt.Fprint(&b, ";")
return b.String()
}
func (ct CaseType) ref() string {
panic("case types do not have a name")
}
func (ct CaseType) repr() string {
return "int"
}
func (ct CaseType) valid(value interface{}) bool {
_, ok := value.(trap.Label)
return ok
}
func (bt BranchType) def() string {
return fmt.Sprintf("%d = %s", bt.idx, bt.name)
}
func (bt BranchType) ref() string {
return bt.name
}
func (bt BranchType) repr() string {
return "int"
}
func (bt BranchType) valid(value interface{}) bool {
_, ok := value.(trap.Label)
return ok
}
// Index returns the numeric index of this branch type
func (bt BranchType) Index() int {
return bt.idx
}
// A Column represents a column in a database table
type Column struct {
columnName string
columnType Type
unique bool
ref bool
}
func (col Column) String() string {
var b strings.Builder
if col.unique {
fmt.Fprint(&b, "unique ")
}
fmt.Fprintf(&b, "%s %s: %s", col.columnType.repr(), col.columnName, col.columnType.ref())
if col.ref {
fmt.Fprint(&b, " ref")
}
return b.String()
}
// Key returns a new column that is the same as this column, but has the `key` flag set to `true`
func (col Column) Key() Column {
return Column{col.columnName, col.columnType, true, false}
}
// Unique returns a new column that is the same as this column, but has the `unique` flag set to `true`
func (col Column) Unique() Column {
return Column{col.columnName, col.columnType, true, col.ref}
}
// EntityColumn constructs a column with name `columnName` holding entities of type `columnType`
func EntityColumn(columnType Type, columnName string) Column {
return Column{columnName, columnType, false, true}
}
// StringColumn constructs a column with name `columnName` holding string values
func StringColumn(columnName string) Column {
return Column{columnName, STRING, false, true}
}
// IntColumn constructs a column with name `columnName` holding integer values
func IntColumn(columnName string) Column {
return Column{columnName, INT, false, true}
}
// A Table represents a database table
type Table struct {
name string
schema []Column
keysets [][]string
}
// KeySet adds `keys` as a keyset to this table
func (tbl *Table) KeySet(keys ...string) *Table {
tbl.keysets = append(tbl.keysets, keys)
return tbl
}
func (tbl Table) String() string {
var b strings.Builder
for _, keyset := range tbl.keysets {
fmt.Fprint(&b, "#keyset[")
sep := ""
for _, key := range keyset {
fmt.Fprintf(&b, "%s%s", sep, key)
sep = ", "
}
fmt.Fprint(&b, "]\n")
}
fmt.Fprint(&b, tbl.name)
fmt.Fprint(&b, "(")
nl := 0
for i, column := range tbl.schema {
if i > 0 {
// wrap >100 char lines
if i < len(tbl.schema)-1 && b.Len()-nl > 100 {
fmt.Fprintf(&b, ",\n%s", strings.Repeat(" ", len(tbl.name)+1))
nl = b.Len()
} else {
fmt.Fprint(&b, ", ")
}
}
fmt.Fprint(&b, column.String())
}
fmt.Fprint(&b, ");")
return b.String()
}
// Emit outputs a tuple of `values` for this table using trap writer `tw`
// and panicks if the tuple does not have the right schema
func (tbl Table) Emit(tw *trap.Writer, values ...interface{}) {
if ncol, nval := len(tbl.schema), len(values); ncol != nval {
log.Fatalf("wrong number of values for table %s; expected %d, but got %d", tbl.name, ncol, nval)
}
for i, col := range tbl.schema {
if !col.columnType.valid(values[i]) {
panic(fmt.Sprintf("Invalid value for column %d of table %s; expected a %s, but got %s which is a %s", i, tbl.name, col.columnType.ref(), values[i], reflect.TypeOf(values[i])))
}
}
tw.Emit(tbl.name, values)
}
var tables = []*Table{}
var types = []Type{}
var defaultSnippets = []string{}
// NewTable constructs a new table with the given `name` and `columns`
func NewTable(name string, columns ...Column) *Table {
tbl := &Table{name, columns, [][]string{}}
tables = append(tables, tbl)
return tbl
}
// NewPrimaryKeyType constructs a new primary key type with the given `name`,
// and adds it to the union types `parents` (if any)
func NewPrimaryKeyType(name string, parents ...*UnionType) *PrimaryKeyType {
tp := &PrimaryKeyType{name}
types = append(types, tp)
for _, parent := range parents {
parent.components = append(parent.components, tp)
}
return tp
}
// NewUnionType constructs a new union type with the given `name`,
// and adds it to the union types `parents` (if any)
func NewUnionType(name string, parents ...*UnionType) *UnionType {
tp := &UnionType{name, []Type{}}
types = append(types, tp)
for _, parent := range parents {
parent.components = append(parent.components, tp)
}
return tp
}
// NewAliasType constructs a new alias type with the given `name` that aliases `underlying`
func NewAliasType(name string, underlying Type) *AliasType {
tp := &AliasType{name, underlying}
types = append(types, tp)
return tp
}
// NewCaseType constructs a new case type on the given `base` type whose discriminator values
// come from `column`
func NewCaseType(base Type, column string) *CaseType {
tp := &CaseType{base, column, []*BranchType{}}
types = append(types, tp)
return tp
}
// NewBranch adds a new branch with the given `name` to this case type
// and adds it to the union types `parents` (if any)
func (ct *CaseType) NewBranch(name string, parents ...*UnionType) *BranchType {
tp := &BranchType{len(ct.branches), name}
ct.branches = append(ct.branches, tp)
for _, parent := range parents {
parent.components = append(parent.components, tp)
}
return tp
}
// AddDefaultSnippet adds the given text `snippet` to the schema of this database
func AddDefaultSnippet(snippet string) bool {
defaultSnippets = append(defaultSnippets, snippet)
return true
}
// PrintDbScheme prints the schema of this database to the writer `w`
func PrintDbScheme(w io.Writer) {
fmt.Fprintf(w, "/** Auto-generated dbscheme; do not edit. */\n\n")
for _, snippet := range defaultSnippets {
fmt.Fprintf(w, "%s\n", snippet)
}
for _, table := range tables {
fmt.Fprintf(w, "%s\n\n", table.String())
}
for _, tp := range types {
def := tp.def()
if def != "" {
fmt.Fprintf(w, "%s\n\n", def)
}
}
}

Просмотреть файл

@ -0,0 +1,872 @@
package dbscheme
import (
"go/ast"
"go/token"
gotypes "go/types"
)
var defaultSnippet = AddDefaultSnippet(`
/** Duplicate code **/
duplicateCode(
unique int id : @duplication,
varchar(900) relativePath : string ref,
int equivClass : int ref);
similarCode(
unique int id : @similarity,
varchar(900) relativePath : string ref,
int equivClass : int ref);
@duplication_or_similarity = @duplication | @similarity;
tokens(
int id : @duplication_or_similarity ref,
int offset : int ref,
int beginLine : int ref,
int beginColumn : int ref,
int endLine : int ref,
int endColumn : int ref);
/** External data **/
externalData(
int id : @externalDataElement,
varchar(900) path : string ref,
int column: int ref,
varchar(900) value : string ref
);
snapshotDate(unique date snapshotDate : date ref);
sourceLocationPrefix(varchar(900) prefix : string ref);
`)
// ContainerType is the type of files and folders
var ContainerType = NewUnionType("@container")
// LocatableType is the type of program entities that have locations
var LocatableType = NewUnionType("@locatable")
// NodeType is the type of AST nodes
var NodeType = NewUnionType("@node", LocatableType)
// DocumentableType is the type of AST nodes to which documentation can be attached
var DocumentableType = NewUnionType("@documentable", NodeType)
// ExprParentType is the type of AST nodes that can have expressions as children
var ExprParentType = NewUnionType("@exprparent", NodeType)
// FieldParentType is the type of AST nodes that can have fields as children
var FieldParentType = NewUnionType("@fieldparent", NodeType)
// StmtParentType is the type of AST nodes that can have statements as children
var StmtParentType = NewUnionType("@stmtparent", NodeType)
// DeclParentType is the type of AST nodes that can have declarations as children
var DeclParentType = NewUnionType("@declparent", NodeType)
// FuncDefType is the type of AST nodes that define functions, that is, function
// declarations and function literals
var FuncDefType = NewUnionType("@funcdef", StmtParentType, ExprParentType)
// ScopeNodeType is the type of AST nodes that mapy have a scope attached to them
var ScopeNodeType = NewUnionType("@scopenode", NodeType)
// LocationDefaultType is the type of source locations
var LocationDefaultType = NewPrimaryKeyType("@location_default")
// FileType is the type of file AST nodes
var FileType = NewPrimaryKeyType("@file", ContainerType, DocumentableType, ExprParentType, DeclParentType, ScopeNodeType)
// FolderType is the type of folders
var FolderType = NewPrimaryKeyType("@folder", ContainerType)
// CommentGroupType is the type of comment groups
var CommentGroupType = NewPrimaryKeyType("@comment_group", NodeType)
// CommentType is the type of comments
var CommentType = NewPrimaryKeyType("@comment", NodeType)
// ExprType is the type of expression AST nodes
var ExprType = NewPrimaryKeyType("@expr", ExprParentType)
// FieldType is the type of field AST nodes
var FieldType = NewPrimaryKeyType("@field", DocumentableType, ExprParentType)
// StmtType is the type of statement AST nodes
var StmtType = NewPrimaryKeyType("@stmt", ExprParentType, StmtParentType)
// DeclType is the type of declaration AST nodes
var DeclType = NewPrimaryKeyType("@decl", ExprParentType, StmtParentType, FieldParentType)
// SpecType is the type of spec AST nodes
var SpecType = NewPrimaryKeyType("@spec", ExprParentType, DocumentableType)
// TypeType is the type of types
var TypeType = NewPrimaryKeyType("@type")
// LocationType is an alias for LocationDefaultType
var LocationType = NewAliasType("@location", LocationDefaultType)
// SourceLineType is an alias for LocatableType
var SourceLineType = NewAliasType("@sourceline", LocatableType)
// CommentKind is a case type for distinguishing different kinds of comments
var CommentKind = NewCaseType(CommentType, "kind")
// SlashSlashComment is the type of single-line comments starting with a double slash
var SlashSlashComment = CommentKind.NewBranch("@slashslashcomment")
// SlashStarComment is the type of block comments delimited by stars and slashes
var SlashStarComment = CommentKind.NewBranch("@slashstarcomment")
// ExprKind is a case type for distinguishing different kinds of expression AST nodes
var ExprKind = NewCaseType(ExprType, "kind")
// BadExpr is type of bad (that is, unparseable) expression AST nodes
var BadExpr = ExprKind.NewBranch("@badexpr")
// IdentExpr is the type of identifier expression AST nodes
var IdentExpr = ExprKind.NewBranch("@ident")
// EllipsisExpr is the type of ellipsis expression AST nodes
var EllipsisExpr = ExprKind.NewBranch("@ellipsis")
// BasicLitExpr is the type of basic (that is, primitive) literal expression AST nodes
var BasicLitExpr = NewUnionType("@basiclit")
// IntLitExpr is a case type for dishinguishing different kinds of literal expression AST nodes
var IntLitExpr = ExprKind.NewBranch("@intlit", BasicLitExpr)
// FloatLitExpr is the type of floating-point literal expression AST nodes
var FloatLitExpr = ExprKind.NewBranch("@floatlit", BasicLitExpr)
// ImagLitExpr is the type of imaginary literal expression AST nodes
var ImagLitExpr = ExprKind.NewBranch("@imaglit", BasicLitExpr)
// CharLitExpr is the type of character literal expression AST nodes
var CharLitExpr = ExprKind.NewBranch("@charlit", BasicLitExpr)
// StringLitExpr is the type of string literal expression AST nodes
var StringLitExpr = ExprKind.NewBranch("@stringlit", BasicLitExpr)
// FuncLitExpr is the type of function literal expression AST nodes
var FuncLitExpr = ExprKind.NewBranch("@funclit", FuncDefType)
// CompositeLitExpr is the type of composite literal expression AST nodes
var CompositeLitExpr = ExprKind.NewBranch("@compositelit")
// ParenExpr is the type of parenthesis expression AST nodes
var ParenExpr = ExprKind.NewBranch("@parenexpr")
// SelectorExpr is the type of selector expression AST nodes
var SelectorExpr = ExprKind.NewBranch("@selectorexpr")
// IndexExpr is the type of index expression AST nodes
var IndexExpr = ExprKind.NewBranch("@indexexpr")
// SliceExpr is the type of slice expression AST nodes
var SliceExpr = ExprKind.NewBranch("@sliceexpr")
// TypeAssertExpr is the type of type assertion expression AST nodes
var TypeAssertExpr = ExprKind.NewBranch("@typeassertexpr")
// CallOrConversionExpr is the type of call and conversion expression AST nodes
// (which cannot be distinguished by purely syntactic criteria)
var CallOrConversionExpr = ExprKind.NewBranch("@callorconversionexpr")
// StarExpr is the type of star expression AST nodes
var StarExpr = ExprKind.NewBranch("@starexpr")
// OperatorExpr is the type of operator expression AST nodes
var OperatorExpr = NewUnionType("@operatorexpr")
// LogicalExpr is the type of logical operator expression AST nodes
var LogicalExpr = NewUnionType("@logicalexpr", OperatorExpr)
// ArithmeticExpr is the type of arithmetic operator expression AST nodes
var ArithmeticExpr = NewUnionType("@arithmeticexpr", OperatorExpr)
// BitwiseExpr is the type of bitwise operator expression AST nodes
var BitwiseExpr = NewUnionType("@bitwiseexpr", OperatorExpr)
// UnaryExpr is the type of unary operator expression AST nodes
var UnaryExpr = NewUnionType("@unaryexpr", OperatorExpr)
// LogicalUnaryExpr is the type of logical unary operator expression AST nodes
var LogicalUnaryExpr = NewUnionType("@logicalunaryexpr", UnaryExpr, LogicalExpr)
// BitwiseUnaryExpr is the type of bitwise unary operator expression AST nodes
var BitwiseUnaryExpr = NewUnionType("@bitwiseunaryexpr", UnaryExpr, BitwiseExpr)
// ArithmeticUnaryExpr is the type of arithmetic unary operator expression AST nodes
var ArithmeticUnaryExpr = NewUnionType("@arithmeticunaryexpr", UnaryExpr, ArithmeticExpr)
// BinaryExpr is the type of binary operator expression AST nodes
var BinaryExpr = NewUnionType("@binaryexpr", OperatorExpr)
// LogicalBinaryExpr is the type of logical binary operator expression AST nodes
var LogicalBinaryExpr = NewUnionType("@logicalbinaryexpr", BinaryExpr, LogicalExpr)
// BitwiseBinaryExpr is the type of bitwise binary operator expression AST nodes
var BitwiseBinaryExpr = NewUnionType("@bitwisebinaryexpr", BinaryExpr, BitwiseExpr)
// ArithmeticBinaryExpr is the type of arithmetic binary operator expression AST nodes
var ArithmeticBinaryExpr = NewUnionType("@arithmeticbinaryexpr", BinaryExpr, ArithmeticExpr)
// ShiftExpr is the type of shift operator expression AST nodes
var ShiftExpr = NewUnionType("@shiftexpr", BitwiseBinaryExpr)
// Comparison is the type of comparison operator expression AST nodes
var Comparison = NewUnionType("@comparison", BinaryExpr)
// EqualityTest is the type of equality operator expression AST nodes
var EqualityTest = NewUnionType("@equalitytest", Comparison)
// RelationalComparison is the type of relational operator expression AST nodes
var RelationalComparison = NewUnionType("@relationalcomparison", Comparison)
// KeyValueExpr is the type of key-value expression AST nodes
var KeyValueExpr = ExprKind.NewBranch("@keyvalueexpr")
// ArrayTypeExpr is the type of array type AST nodes
var ArrayTypeExpr = ExprKind.NewBranch("@arraytypeexpr")
// StructTypeExpr is the type of struct type AST nodes
var StructTypeExpr = ExprKind.NewBranch("@structtypeexpr", FieldParentType)
// FuncTypeExpr is the type of function type AST nodes
var FuncTypeExpr = ExprKind.NewBranch("@functypeexpr", FieldParentType, ScopeNodeType)
// InterfaceTypeExpr is the type of interface type AST nodes
var InterfaceTypeExpr = ExprKind.NewBranch("@interfacetypeexpr", FieldParentType)
// MapTypeExpr is the type of map type AST nodes
var MapTypeExpr = ExprKind.NewBranch("@maptypeexpr")
// ChanTypeExpr is the type of channel type AST nodes
var ChanTypeExpr = NewUnionType("@chantypeexpr")
// UnaryExprs is a map from unary operator tokens to the corresponding AST node type
var UnaryExprs = map[token.Token]*BranchType{
token.ADD: ExprKind.NewBranch("@plusexpr", ArithmeticUnaryExpr),
token.SUB: ExprKind.NewBranch("@minusexpr", ArithmeticUnaryExpr),
token.NOT: ExprKind.NewBranch("@notexpr", LogicalUnaryExpr),
token.XOR: ExprKind.NewBranch("@complementexpr", BitwiseUnaryExpr),
token.MUL: ExprKind.NewBranch("@derefexpr", UnaryExpr),
token.AND: ExprKind.NewBranch("@addressexpr", UnaryExpr),
token.ARROW: ExprKind.NewBranch("@arrowexpr", UnaryExpr),
}
// BinaryExprs is a map from binary operator tokens to the corresponding AST node type
var BinaryExprs = map[token.Token]*BranchType{
token.LOR: ExprKind.NewBranch("@lorexpr", LogicalBinaryExpr),
token.LAND: ExprKind.NewBranch("@landexpr", LogicalBinaryExpr),
token.EQL: ExprKind.NewBranch("@eqlexpr", EqualityTest),
token.NEQ: ExprKind.NewBranch("@neqexpr", EqualityTest),
token.LSS: ExprKind.NewBranch("@lssexpr", RelationalComparison),
token.LEQ: ExprKind.NewBranch("@leqexpr", RelationalComparison),
token.GTR: ExprKind.NewBranch("@gtrexpr", RelationalComparison),
token.GEQ: ExprKind.NewBranch("@geqexpr", RelationalComparison),
token.ADD: ExprKind.NewBranch("@addexpr", ArithmeticBinaryExpr),
token.SUB: ExprKind.NewBranch("@subexpr", ArithmeticBinaryExpr),
token.OR: ExprKind.NewBranch("@orexpr", BitwiseBinaryExpr),
token.XOR: ExprKind.NewBranch("@xorexpr", BitwiseBinaryExpr),
token.MUL: ExprKind.NewBranch("@mulexpr", ArithmeticBinaryExpr),
token.QUO: ExprKind.NewBranch("@quoexpr", ArithmeticBinaryExpr),
token.REM: ExprKind.NewBranch("@remexpr", ArithmeticBinaryExpr),
token.SHL: ExprKind.NewBranch("@shlexpr", ShiftExpr),
token.SHR: ExprKind.NewBranch("@shrexpr", ShiftExpr),
token.AND: ExprKind.NewBranch("@andexpr", BitwiseBinaryExpr),
token.AND_NOT: ExprKind.NewBranch("@andnotexpr", BitwiseBinaryExpr),
}
// ChanTypeExprs is a map from channel type expressions to the corresponding AST node type
var ChanTypeExprs = map[ast.ChanDir]*BranchType{
ast.SEND: ExprKind.NewBranch("@sendchantypeexpr", ChanTypeExpr),
ast.RECV: ExprKind.NewBranch("@recvchantypeexpr", ChanTypeExpr),
ast.SEND | ast.RECV: ExprKind.NewBranch("@sendrcvchantypeexpr", ChanTypeExpr),
}
// StmtKind is a case type for distinguishing different kinds of statement AST nodes
var StmtKind = NewCaseType(StmtType, "kind")
// BadStmtType is the type of bad (that is, unparseable) statement AST nodes
var BadStmtType = StmtKind.NewBranch("@badstmt")
// DeclStmtType is the type of declaration statement AST nodes
var DeclStmtType = StmtKind.NewBranch("@declstmt", DeclParentType)
// EmptyStmtType is the type of empty statement AST nodes
var EmptyStmtType = StmtKind.NewBranch("@emptystmt")
// LabeledStmtType is the type of labeled statement AST nodes
var LabeledStmtType = StmtKind.NewBranch("@labeledstmt")
// ExprStmtType is the type of expressio statemement AST nodes
var ExprStmtType = StmtKind.NewBranch("@exprstmt")
// SendStmtType is the type of send statement AST nodes
var SendStmtType = StmtKind.NewBranch("@sendstmt")
// IncDecStmtType is the type of increment/decrement statement AST nodes
var IncDecStmtType = NewUnionType("@incdecstmt")
// IncStmtType is the type of increment statement AST nodes
var IncStmtType = StmtKind.NewBranch("@incstmt", IncDecStmtType)
// DecStmtType is the type of decrement statement AST nodes
var DecStmtType = StmtKind.NewBranch("@decstmt", IncDecStmtType)
// AssignmentType is the type of assignment statement AST nodes
var AssignmentType = NewUnionType("@assignment")
// SimpleAssignStmtType is the type of simple (i.e., non-compound) assignment statement AST nodes
var SimpleAssignStmtType = NewUnionType("@simpleassignstmt", AssignmentType)
// CompoundAssignStmtType is the type of compound assignment statement AST nodes
var CompoundAssignStmtType = NewUnionType("@compoundassignstmt", AssignmentType)
// GoStmtType is the type of go statement AST nodes
var GoStmtType = StmtKind.NewBranch("@gostmt")
// DeferStmtType is the type of defer statement AST nodes
var DeferStmtType = StmtKind.NewBranch("@deferstmt")
// ReturnStmtType is the type of return statement AST nodes
var ReturnStmtType = StmtKind.NewBranch("@returnstmt")
// BranchStmtType is the type of branch statement AST nodes
var BranchStmtType = NewUnionType("@branchstmt")
// BreakStmtType is the type of break statement AST nodes
var BreakStmtType = StmtKind.NewBranch("@breakstmt", BranchStmtType)
// ContinueStmtType is the type of continue statement AST nodes
var ContinueStmtType = StmtKind.NewBranch("@continuestmt", BranchStmtType)
// GotoStmtType is the type of goto statement AST nodes
var GotoStmtType = StmtKind.NewBranch("@gotostmt", BranchStmtType)
// FallthroughStmtType is the type of fallthrough statement AST nodes
var FallthroughStmtType = StmtKind.NewBranch("@fallthroughstmt", BranchStmtType)
// BlockStmtType is the type of block statement AST nodes
var BlockStmtType = StmtKind.NewBranch("@blockstmt", ScopeNodeType)
// IfStmtType is the type of if statement AST nodes
var IfStmtType = StmtKind.NewBranch("@ifstmt", ScopeNodeType)
// CaseClauseType is the type of case clause AST nodes
var CaseClauseType = StmtKind.NewBranch("@caseclause", ScopeNodeType)
// SwitchStmtType is the type of switch statement AST nodes, covering both expression switch and type switch
var SwitchStmtType = NewUnionType("@switchstmt", ScopeNodeType)
// ExprSwitchStmtType is the type of expression-switch statement AST nodes
var ExprSwitchStmtType = StmtKind.NewBranch("@exprswitchstmt", SwitchStmtType)
// TypeSwitchStmtType is the type of type-switch statement AST nodes
var TypeSwitchStmtType = StmtKind.NewBranch("@typeswitchstmt", SwitchStmtType)
// CommClauseType is the type of comm clause AST ndoes
var CommClauseType = StmtKind.NewBranch("@commclause", ScopeNodeType)
// SelectStmtType is the type of select statement AST nodes
var SelectStmtType = StmtKind.NewBranch("@selectstmt")
// LoopStmtType is the type of loop statement AST nodes (including for statements and range statements)
var LoopStmtType = NewUnionType("@loopstmt", ScopeNodeType)
// ForStmtType is the type of for statement AST nodes
var ForStmtType = StmtKind.NewBranch("@forstmt", LoopStmtType)
// RangeStmtType is the type of range statement AST nodes
var RangeStmtType = StmtKind.NewBranch("@rangestmt", LoopStmtType)
// AssignStmtTypes is a map from assignmnt operator tokens to corresponding AST node types
var AssignStmtTypes = map[token.Token]*BranchType{
token.ASSIGN: StmtKind.NewBranch("@assignstmt", SimpleAssignStmtType),
token.DEFINE: StmtKind.NewBranch("@definestmt", SimpleAssignStmtType),
token.ADD_ASSIGN: StmtKind.NewBranch("@addassignstmt", CompoundAssignStmtType),
token.SUB_ASSIGN: StmtKind.NewBranch("@subassignstmt", CompoundAssignStmtType),
token.MUL_ASSIGN: StmtKind.NewBranch("@mulassignstmt", CompoundAssignStmtType),
token.QUO_ASSIGN: StmtKind.NewBranch("@quoassignstmt", CompoundAssignStmtType),
token.REM_ASSIGN: StmtKind.NewBranch("@remassignstmt", CompoundAssignStmtType),
token.AND_ASSIGN: StmtKind.NewBranch("@andassignstmt", CompoundAssignStmtType),
token.OR_ASSIGN: StmtKind.NewBranch("@orassignstmt", CompoundAssignStmtType),
token.XOR_ASSIGN: StmtKind.NewBranch("@xorassignstmt", CompoundAssignStmtType),
token.SHL_ASSIGN: StmtKind.NewBranch("@shlassignstmt", CompoundAssignStmtType),
token.SHR_ASSIGN: StmtKind.NewBranch("@shrassignstmt", CompoundAssignStmtType),
token.AND_NOT_ASSIGN: StmtKind.NewBranch("@andnotassignstmt", CompoundAssignStmtType),
}
// DeclKind is a case type for distinguishing different kinds of declaration AST nodes
var DeclKind = NewCaseType(DeclType, "kind")
// BadDeclType is the type of bad (that is, unparseable) declaration AST nodes
var BadDeclType = DeclKind.NewBranch("@baddecl")
// GenDeclType is the type of generic declaration AST nodes
var GenDeclType = NewUnionType("@gendecl", DocumentableType)
// ImportDeclType is the type of import declaration AST nodes
var ImportDeclType = DeclKind.NewBranch("@importdecl", GenDeclType)
// ConstDeclType is the type of constant declaration AST nodes
var ConstDeclType = DeclKind.NewBranch("@constdecl", GenDeclType)
// TypeDeclType is the type of type declaration AST nodes
var TypeDeclType = DeclKind.NewBranch("@typedecl", GenDeclType)
// VarDeclType is the type of variable declaration AST nodes
var VarDeclType = DeclKind.NewBranch("@vardecl", GenDeclType)
// FuncDeclType is the type of function declaration AST nodes
var FuncDeclType = DeclKind.NewBranch("@funcdecl", DocumentableType, FuncDefType)
// SpecKind is a case type for distinguishing different kinds of declaration specification nodes
var SpecKind = NewCaseType(SpecType, "kind")
// ImportSpecType is the type of import declaration specification nodes
var ImportSpecType = SpecKind.NewBranch("@importspec")
// ValueSpecType is the type of value declaration specification nodes
var ValueSpecType = SpecKind.NewBranch("@valuespec")
// TypeSpecType is the type of type declaration specification nodes
var TypeSpecType = SpecKind.NewBranch("@typespec")
// ObjectType is the type of objects (that is, declared entities)
var ObjectType = NewPrimaryKeyType("@object")
// ObjectKind is a case type for distinguishing different kinds of built-in and declared objects
var ObjectKind = NewCaseType(ObjectType, "kind")
// DeclObjectType is the type of declared objects
var DeclObjectType = NewUnionType("@declobject")
// BuiltinObjectType is the type of built-in objects
var BuiltinObjectType = NewUnionType("@builtinobject")
// PkgObjectType is the type of imported packages
var PkgObjectType = ObjectKind.NewBranch("@pkgobject")
// TypeObjectType is the type of declared or built-in named types
var TypeObjectType = NewUnionType("@typeobject")
// DeclTypeObjectType is the type of declared named types
var DeclTypeObjectType = ObjectKind.NewBranch("@decltypeobject", TypeObjectType, DeclObjectType)
// BuiltinTypeObjectType is the type of built-in named types
var BuiltinTypeObjectType = ObjectKind.NewBranch("@builtintypeobject", TypeObjectType, BuiltinObjectType)
// ValueObjectType is the type of declared or built-in variables or constants
var ValueObjectType = NewUnionType("@valueobject")
// ConstObjectType is the type of declared or built-in constants
var ConstObjectType = NewUnionType("@constobject", ValueObjectType)
// DeclConstObjectType is the type of declared constants
var DeclConstObjectType = ObjectKind.NewBranch("@declconstobject", ConstObjectType, DeclObjectType)
// BuiltinConstObjectType is the type of built-in constants
var BuiltinConstObjectType = ObjectKind.NewBranch("@builtinconstobject", ConstObjectType, BuiltinObjectType)
// VarObjectType is the type of declared or built-in variables (the latter do not currently exist)
var VarObjectType = NewUnionType("@varobject", ValueObjectType)
// DeclVarObjectType is the type of declared variables including function parameters, results and struct fields
var DeclVarObjectType = ObjectKind.NewBranch("@declvarobject", VarObjectType, DeclObjectType)
// FunctionObjectType is the type of declared or built-in functions
var FunctionObjectType = NewUnionType("@functionobject", ValueObjectType)
// DeclFuncObjectType is the type of declared functions, including (abstract and concrete) methods
var DeclFuncObjectType = ObjectKind.NewBranch("@declfunctionobject", FunctionObjectType, DeclObjectType)
// BuiltinFuncObjectType is the type of built-in functions
var BuiltinFuncObjectType = ObjectKind.NewBranch("@builtinfunctionobject", FunctionObjectType, BuiltinObjectType)
// LabelObjectType is the type of statement labels
var LabelObjectType = ObjectKind.NewBranch("@labelobject")
// ScopeType is the type of scopes
var ScopeType = NewPrimaryKeyType("@scope")
// ScopeKind is a case type for distinguishing different kinds of scopes
var ScopeKind = NewCaseType(ScopeType, "kind")
// UniverseScopeType is the type of the universe scope
var UniverseScopeType = ScopeKind.NewBranch("@universescope")
// PackageScopeType is the type of package scopes
var PackageScopeType = ScopeKind.NewBranch("@packagescope")
// LocalScopeType is the type of local (that is, non-universe, non-package) scopes
var LocalScopeType = ScopeKind.NewBranch("@localscope", LocatableType)
// TypeKind is a case type for distinguishing different kinds of types
var TypeKind = NewCaseType(TypeType, "kind")
// BasicType is the union of all basic types
var BasicType = NewUnionType("@basictype")
// BoolType is the union of the normal and literal bool types
var BoolType = NewUnionType("@booltype", BasicType)
// NumericType is the union of numeric types
var NumericType = NewUnionType("@numerictype", BasicType)
// IntegerType is the union of integer types
var IntegerType = NewUnionType("@integertype", NumericType)
// SignedIntegerType is the union of signed integer types
var SignedIntegerType = NewUnionType("@signedintegertype", IntegerType)
// UnsignedIntegerType is the union of unsigned integer types
var UnsignedIntegerType = NewUnionType("@unsignedintegertype", IntegerType)
// FloatType is the union of floating-point types
var FloatType = NewUnionType("@floattype", NumericType)
// ComplexType is the union of complex types
var ComplexType = NewUnionType("@complextype", NumericType)
// StringType is the union of the normal and literal string types
var StringType = NewUnionType("@stringtype", BasicType)
// LiteralType is the union of literal types
var LiteralType = NewUnionType("@literaltype", BasicType)
// BasicTypes is a map from basic type kinds to the corresponding entity types
var BasicTypes = map[gotypes.BasicKind]*BranchType{
gotypes.Invalid: TypeKind.NewBranch("@invalidtype", BasicType),
gotypes.Bool: TypeKind.NewBranch("@boolexprtype", BoolType),
gotypes.Int: TypeKind.NewBranch("@inttype", SignedIntegerType),
gotypes.Int8: TypeKind.NewBranch("@int8type", SignedIntegerType),
gotypes.Int16: TypeKind.NewBranch("@int16type", SignedIntegerType),
gotypes.Int32: TypeKind.NewBranch("@int32type", SignedIntegerType),
gotypes.Int64: TypeKind.NewBranch("@int64type", SignedIntegerType),
gotypes.Uint: TypeKind.NewBranch("@uinttype", UnsignedIntegerType),
gotypes.Uint8: TypeKind.NewBranch("@uint8type", UnsignedIntegerType),
gotypes.Uint16: TypeKind.NewBranch("@uint16type", UnsignedIntegerType),
gotypes.Uint32: TypeKind.NewBranch("@uint32type", UnsignedIntegerType),
gotypes.Uint64: TypeKind.NewBranch("@uint64type", UnsignedIntegerType),
gotypes.Uintptr: TypeKind.NewBranch("@uintptrtype", BasicType),
gotypes.Float32: TypeKind.NewBranch("@float32type", FloatType),
gotypes.Float64: TypeKind.NewBranch("@float64type", FloatType),
gotypes.Complex64: TypeKind.NewBranch("@complex64type", ComplexType),
gotypes.Complex128: TypeKind.NewBranch("@complex128type", ComplexType),
gotypes.String: TypeKind.NewBranch("@stringexprtype", StringType),
gotypes.UnsafePointer: TypeKind.NewBranch("@unsafepointertype", BasicType),
gotypes.UntypedBool: TypeKind.NewBranch("@boolliteraltype", LiteralType, BoolType),
gotypes.UntypedInt: TypeKind.NewBranch("@intliteraltype", LiteralType, SignedIntegerType),
gotypes.UntypedRune: TypeKind.NewBranch("@runeliteraltype", LiteralType, SignedIntegerType),
gotypes.UntypedFloat: TypeKind.NewBranch("@floatliteraltype", LiteralType, FloatType),
gotypes.UntypedComplex: TypeKind.NewBranch("@complexliteraltype", LiteralType, ComplexType),
gotypes.UntypedString: TypeKind.NewBranch("@stringliteraltype", LiteralType, StringType),
gotypes.UntypedNil: TypeKind.NewBranch("@nilliteraltype", LiteralType),
}
// CompositeType is the type of all composite (that is, non-basic) types
var CompositeType = NewUnionType("@compositetype")
// ElementContainerType is the type of types that have elements, such as arrays
// and channels
var ElementContainerType = NewUnionType("@containertype", CompositeType)
// ArrayType is the type of array types
var ArrayType = TypeKind.NewBranch("@arraytype", ElementContainerType)
// SliceType is the type of slice types
var SliceType = TypeKind.NewBranch("@slicetype", ElementContainerType)
// StructType is the type of struct types
var StructType = TypeKind.NewBranch("@structtype", CompositeType)
// PointerType is the type of pointer types
var PointerType = TypeKind.NewBranch("@pointertype", CompositeType)
// InterfaceType is the type of interface types
var InterfaceType = TypeKind.NewBranch("@interfacetype", CompositeType)
// TupleType is the type of tuple types
var TupleType = TypeKind.NewBranch("@tupletype", CompositeType)
// SignatureType is the type of signature types
var SignatureType = TypeKind.NewBranch("@signaturetype", CompositeType)
// MapType is the type of map types
var MapType = TypeKind.NewBranch("@maptype", ElementContainerType)
// ChanType is the type of channel types
var ChanType = NewUnionType("@chantype", ElementContainerType)
// ChanTypes is a map from channel type directions to the corresponding type
var ChanTypes = map[gotypes.ChanDir]*BranchType{
gotypes.SendOnly: TypeKind.NewBranch("@sendchantype", ChanType),
gotypes.RecvOnly: TypeKind.NewBranch("@recvchantype", ChanType),
gotypes.SendRecv: TypeKind.NewBranch("@sendrcvchantype", ChanType),
}
// NamedType is the type of named types
var NamedType = TypeKind.NewBranch("@namedtype", CompositeType)
// PackageType is the type of packages
var PackageType = NewPrimaryKeyType("@package")
// LocationsDefaultTable is the table defining location objects
var LocationsDefaultTable = NewTable("locations_default",
EntityColumn(LocationDefaultType, "id").Key(),
EntityColumn(FileType, "file"),
IntColumn("beginLine"),
IntColumn("beginColumn"),
IntColumn("endLine"),
IntColumn("endColumn"),
)
// NumlinesTable is the table containing LoC information
var NumlinesTable = NewTable("numlines",
EntityColumn(SourceLineType, "element_id"),
IntColumn("num_lines"),
IntColumn("num_code"),
IntColumn("num_comment"),
)
// FilesTable is the table defining file nodes
var FilesTable = NewTable("files",
EntityColumn(FileType, "id").Key(),
StringColumn("name"),
StringColumn("simple"),
StringColumn("ext"),
IntColumn("fromSource"),
)
// FoldersTable is the table defining folder entities
var FoldersTable = NewTable("folders",
EntityColumn(FolderType, "id").Key(),
StringColumn("name"),
StringColumn("simple"),
)
// ContainerParentTable is the table defining the parent-child relation among container entities
var ContainerParentTable = NewTable("containerparent",
EntityColumn(ContainerType, "parent"),
EntityColumn(ContainerType, "child").Unique(),
)
// HasLocationTable is the table associating entities with their locations
var HasLocationTable = NewTable("has_location",
EntityColumn(LocatableType, "locatable").Unique(),
EntityColumn(LocationType, "location"),
)
// CommentGroupsTable is the table defining comment group entities
var CommentGroupsTable = NewTable("comment_groups",
EntityColumn(CommentGroupType, "id").Key(),
)
// CommentsTable is the table defining comment entities
var CommentsTable = NewTable("comments",
EntityColumn(CommentType, "id").Key(),
IntColumn("kind"),
EntityColumn(CommentGroupType, "parent"),
IntColumn("idx"),
StringColumn("text"),
)
// DocCommentsTable is the table associating doc comments with the nodes they document
var DocCommentsTable = NewTable("doc_comments",
EntityColumn(DocumentableType, "node").Unique(),
EntityColumn(CommentGroupType, "comment"),
)
// ExprsTable is the table defininig expression AST nodes
var ExprsTable = NewTable("exprs",
EntityColumn(ExprType, "id").Key(),
IntColumn("kind"),
EntityColumn(ExprParentType, "parent"),
IntColumn("idx"),
).KeySet("parent", "idx")
// LiteralsTable is the table associating literal expression AST nodes with their values
var LiteralsTable = NewTable("literals",
EntityColumn(ExprType, "expr").Unique(),
StringColumn("value"),
StringColumn("raw"),
)
// ConstValuesTable is the table associating constant expressions with their values
var ConstValuesTable = NewTable("constvalues",
EntityColumn(ExprType, "expr").Unique(),
StringColumn("value"),
StringColumn("exact"),
)
// FieldsTable is the table defining field AST nodes
var FieldsTable = NewTable("fields",
EntityColumn(FieldType, "id").Key(),
EntityColumn(FieldParentType, "parent"),
IntColumn("idx"),
)
// StmtsTable is the table defining statement AST nodes
var StmtsTable = NewTable("stmts",
EntityColumn(StmtType, "id").Key(),
IntColumn("kind"),
EntityColumn(StmtParentType, "parent"),
IntColumn("idx"),
).KeySet("parent", "idx")
// DeclsTable is the table defining declaration AST nodes
var DeclsTable = NewTable("decls",
EntityColumn(DeclType, "id").Key(),
IntColumn("kind"),
EntityColumn(DeclParentType, "parent"),
IntColumn("idx"),
).KeySet("parent", "idx")
// SpecsTable is the table defining declaration specification AST nodes
var SpecsTable = NewTable("specs",
EntityColumn(SpecType, "id").Key(),
IntColumn("kind"),
EntityColumn(GenDeclType, "parent"),
IntColumn("idx"),
).KeySet("parent", "idx")
// ScopesTable is the table defining scopes
var ScopesTable = NewTable("scopes",
EntityColumn(ScopeType, "id").Key(),
IntColumn("kind"),
)
// ScopeNestingTable is the table describing scope nesting
var ScopeNestingTable = NewTable("scopenesting",
EntityColumn(ScopeType, "inner").Unique(),
EntityColumn(ScopeType, "outer"),
)
// ScopeNodesTable is the table associating local scopes with the AST nodes that induce them
var ScopeNodesTable = NewTable("scopenodes",
EntityColumn(ScopeNodeType, "node").Unique(),
EntityColumn(LocalScopeType, "scope"),
)
// ObjectsTable is the table describing objects (that is, declared entities)
var ObjectsTable = NewTable("objects",
EntityColumn(ObjectType, "id").Key(),
IntColumn("kind"),
StringColumn("name"),
)
// ObjectScopesTable is the table describing the scope to which an object belongs (if any)
var ObjectScopesTable = NewTable("objectscopes",
EntityColumn(ObjectType, "object").Unique(),
EntityColumn(ScopeType, "scope"),
)
// ObjectTypesTable is the table describing the type of an object (if any)
var ObjectTypesTable = NewTable("objecttypes",
EntityColumn(ObjectType, "object").Unique(),
EntityColumn(TypeType, "tp"),
)
// MethodReceiversTable maps methods to their receiver
var MethodReceiversTable = NewTable("methodreceivers",
EntityColumn(ObjectType, "method").Unique(),
EntityColumn(ObjectType, "receiver"),
)
// FieldStructsTable maps fields to the structs they are in
var FieldStructsTable = NewTable("fieldstructs",
EntityColumn(ObjectType, "field").Unique(),
EntityColumn(StructType, "struct"),
)
// DefsTable maps identifiers to the objects they define
var DefsTable = NewTable("defs",
EntityColumn(IdentExpr, "ident"),
EntityColumn(ObjectType, "object"),
)
// UsesTable maps identifiers to the objects they denote
var UsesTable = NewTable("uses",
EntityColumn(IdentExpr, "ident"),
EntityColumn(ObjectType, "object"),
)
// TypesTable is the table describing types
var TypesTable = NewTable("types",
EntityColumn(TypeType, "id").Key(),
IntColumn("kind"),
)
// TypeOfTable is the table associating expressions with their types (if known)
var TypeOfTable = NewTable("type_of",
EntityColumn(ExprType, "expr").Unique(),
EntityColumn(TypeType, "tp"),
)
// TypeNameTable is the table associating named types with their names
var TypeNameTable = NewTable("typename",
EntityColumn(TypeType, "tp").Unique(),
StringColumn("name"),
)
// KeyTypeTable is the table associating maps with their key type
var KeyTypeTable = NewTable("key_type",
EntityColumn(MapType, "map").Unique(),
EntityColumn(TypeType, "tp"),
)
// ElementTypeTable is the table associating container types with their element
// type
var ElementTypeTable = NewTable("element_type",
EntityColumn(ElementContainerType, "container").Unique(),
EntityColumn(TypeType, "tp"),
)
// BaseTypeTable is the table associating pointer types with their base type
var BaseTypeTable = NewTable("base_type",
EntityColumn(PointerType, "ptr").Unique(),
EntityColumn(TypeType, "tp"),
)
// UnderlyingTypeTable is the table associating named types with their
// underlying type
var UnderlyingTypeTable = NewTable("underlying_type",
EntityColumn(NamedType, "named").Unique(),
EntityColumn(TypeType, "tp"),
)
// ComponentTypesTable is the table associating composite types with their component types
var ComponentTypesTable = NewTable("component_types",
EntityColumn(CompositeType, "parent"),
IntColumn("index"),
StringColumn("name"),
EntityColumn(TypeType, "tp"),
).KeySet("parent", "index")
// ArrayLengthTable is the table associating array types with their length (represented as a string
// since Go array lengths are 64-bit and hence do not always fit into a QL integer)
var ArrayLengthTable = NewTable("array_length",
EntityColumn(ArrayType, "tp").Unique(),
StringColumn("len"),
)
// TypeObjectTable maps types to their corresponding objects, if any
var TypeObjectTable = NewTable("type_objects",
EntityColumn(TypeType, "tp").Unique(),
EntityColumn(ObjectType, "object"),
)
// PackagesTable is the table describing packages
var PackagesTable = NewTable("packages",
EntityColumn(PackageType, "id").Key(),
StringColumn("name"),
StringColumn("path"),
EntityColumn(PackageScopeType, "scope"),
)

1305
extractor/extractor.go Normal file

Разница между файлами не показана из-за своего большого размера Загрузить разницу

Просмотреть файл

@ -0,0 +1,13 @@
package net.sourceforge.pmd.cpd;
/*
* This is a stub definition for pmd's AbstractLanguage class
* including only the API used by the GoLanguage class.
*/
public abstract class AbstractLanguage {
public AbstractLanguage(String... extensions) {}
public abstract Tokenizer getTokenizer(boolean fuzzyMatch);
}

Просмотреть файл

@ -0,0 +1,68 @@
package net.sourceforge.pmd.cpd;
import java.io.IOException;
import java.io.InputStreamReader;
import java.lang.ProcessBuilder.Redirect;
import java.nio.charset.Charset;
import java.nio.file.Paths;
import java.util.List;
import opencsv.CSVReader;
public class GoLanguage extends AbstractLanguage {
public GoLanguage() {
super(".go");
}
@Override
public Tokenizer getTokenizer(final boolean fuzzyMatch) {
return new Tokenizer() {
@Override
public void tokenize(SourceCode tokens, List<TokenEntry> tokenEntries) {
String fileName = tokens.getFileName();
String platform = "linux", exe = "";
String osName = System.getProperty("os.name", "unknown");
if (osName.contains("Windows")) {
platform = "win";
exe = ".exe";
} else if (osName.contains("Mac OS X")) {
platform = "osx";
}
// get tools folder from SEMMLE_DIST
String toolsDir = null;
String dist = System.getenv("SEMMLE_DIST");
if (dist != null && !dist.isEmpty()) {
toolsDir = dist + "/language-packs/go/tools/platform/" + platform;
}
String goTokenizer = toolsDir == null ? "go-tokenizer" : toolsDir + "/bin/go-tokenizer";
goTokenizer += exe;
ProcessBuilder pb = new ProcessBuilder(Paths.get(goTokenizer).toString(), fileName);
pb.redirectError(Redirect.INHERIT);
try {
Process process = pb.start();
try (
CSVReader r = new CSVReader(new InputStreamReader(process.getInputStream(), Charset.forName("UTF-8")))
) {
String[] row;
while ((row = r.readNext()) != null) {
String text = row[0];
String fuzzyText = row[1];
int beginLine = Integer.parseInt(row[2]);
int beginColumn = Integer.parseInt(row[3]);
int endLine = Integer.parseInt(row[4]);
int endColumn = Integer.parseInt(row[5]);
tokenEntries.add(new TokenEntry(fuzzyMatch ? text : fuzzyText, fileName, beginLine, beginColumn, endLine, endColumn));
}
}
int exitCode = process.waitFor();
if (exitCode != 0)
throw new RuntimeException("Tokenizing " + fileName + " returned " + exitCode + ".");
} catch (IOException | InterruptedException e) {
throw new RuntimeException(e);
}
}
};
}
}

Просмотреть файл

@ -0,0 +1,12 @@
package net.sourceforge.pmd.cpd;
/*
* This is a stub definition for pmd's SourceCode class
* including only the API used by the GoLanguage class.
*/
public class SourceCode {
public String getFileName() {
return null;
}
}

Просмотреть файл

@ -0,0 +1,11 @@
package net.sourceforge.pmd.cpd;
/*
* This is a stub definition for pmd's TokenEntry class
* including only the API used by the GoLanguage class.
*/
public class TokenEntry {
public TokenEntry(String image, String tokenSrcID, int beginLine, int beginColumn, int endLine, int endColumn) {
}
}

Просмотреть файл

@ -0,0 +1,12 @@
package net.sourceforge.pmd.cpd;
/*
* This is a stub definition for pmd's Tokenizer interface
* including only the API used by the GoLanguage class.
*/
import java.util.List;
public interface Tokenizer {
void tokenize(SourceCode tokens, List<TokenEntry> tokenEntries);
}

Просмотреть файл

@ -0,0 +1,207 @@
/**
Copyright 2005 Bytecode Pty Ltd.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package opencsv;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* A very simple CSV parser released under a commercial-friendly license.
* This just implements splitting a single line into fields.
*
* @author Glen Smith
* @author Rainer Pruy
*
*/
public class CSVParser {
private final char separator;
private final char quotechar;
private final char escape;
private final boolean strictQuotes;
private StringBuilder buf = new StringBuilder(INITIAL_READ_SIZE);
/** The default separator to use if none is supplied to the constructor. */
public static final char DEFAULT_SEPARATOR = ',';
private static final int INITIAL_READ_SIZE = 128;
/**
* The default quote character to use if none is supplied to the
* constructor.
*/
public static final char DEFAULT_QUOTE_CHARACTER = '"';
/**
* The default escape character to use if none is supplied to the
* constructor.
*/
public static final char DEFAULT_ESCAPE_CHARACTER = '"';
/**
* The default strict quote behavior to use if none is supplied to the
* constructor
*/
public static final boolean DEFAULT_STRICT_QUOTES = false;
/**
* Constructs CSVReader with supplied separator and quote char.
* Allows setting the "strict quotes" flag
* @param separator
* the delimiter to use for separating entries
* @param quotechar
* the character to use for quoted elements
* @param escape
* the character to use for escaping a separator or quote
* @param strictQuotes
* if true, characters outside the quotes are ignored
*/
CSVParser(char separator, char quotechar, char escape, boolean strictQuotes) {
this.separator = separator;
this.quotechar = quotechar;
this.escape = escape;
this.strictQuotes = strictQuotes;
}
/**
*
* @return true if something was left over from last call(s)
*/
public boolean isPending() {
return buf.length() != 0;
}
public String[] parseLineMulti(String nextLine) throws IOException {
return parseLine(nextLine, true);
}
public String[] parseLine(String nextLine) throws IOException {
return parseLine(nextLine, false);
}
/**
* Parses an incoming String and returns an array of elements.
*
* @param nextLine
* the string to parse
* @return the comma-tokenized list of elements, or null if nextLine is null
* @throws IOException if bad things happen during the read
*/
private String[] parseLine(String nextLine, boolean multi) throws IOException {
if (!multi && isPending()) {
clear();
}
if (nextLine == null) {
if (isPending()) {
String s = buf.toString();
clear();
return new String[] {s};
} else {
return null;
}
}
List<String>tokensOnThisLine = new ArrayList<String>();
boolean inQuotes = isPending();
for (int i = 0; i < nextLine.length(); i++) {
char c = nextLine.charAt(i);
if (c == this.escape && isNextCharacterEscapable(nextLine, inQuotes, i)) {
buf.append(nextLine.charAt(i+1));
i++;
} else if (c == quotechar) {
if( isNextCharacterEscapedQuote(nextLine, inQuotes, i) ){
buf.append(nextLine.charAt(i+1));
i++;
}else{
inQuotes = !inQuotes;
// the tricky case of an embedded quote in the middle: a,bc"d"ef,g
if (!strictQuotes) {
if(i>2 //not on the beginning of the line
&& nextLine.charAt(i-1) != this.separator //not at the beginning of an escape sequence
&& nextLine.length()>(i+1) &&
nextLine.charAt(i+1) != this.separator //not at the end of an escape sequence
){
buf.append(c);
}
}
}
} else if (c == separator && !inQuotes) {
tokensOnThisLine.add(buf.toString());
clear(); // start work on next token
} else {
if (!strictQuotes || inQuotes)
buf.append(c);
}
}
// line is done - check status
if (inQuotes) {
if (multi) {
// continuing a quoted section, re-append newline
buf.append('\n');
// this partial content is not to be added to field list yet
} else {
throw new IOException("Un-terminated quoted field at end of CSV line");
}
} else {
tokensOnThisLine.add(buf.toString());
clear();
}
return tokensOnThisLine.toArray(new String[tokensOnThisLine.size()]);
}
/**
* precondition: the current character is a quote or an escape
* @param nextLine the current line
* @param inQuotes true if the current context is quoted
* @param i current index in line
* @return true if the following character is a quote
*/
private boolean isNextCharacterEscapedQuote(String nextLine, boolean inQuotes, int i) {
return inQuotes // we are in quotes, therefore there can be escaped quotes in here.
&& nextLine.length() > (i+1) // there is indeed another character to check.
&& nextLine.charAt(i+1) == quotechar;
}
/**
* precondition: the current character is an escape
* @param nextLine the current line
* @param inQuotes true if the current context is quoted
* @param i current index in line
* @return true if the following character is a quote
*/
protected boolean isNextCharacterEscapable(String nextLine, boolean inQuotes, int i) {
return inQuotes // we are in quotes, therefore there can be escaped quotes in here.
&& nextLine.length() > (i+1) // there is indeed another character to check.
&& ( nextLine.charAt(i+1) == quotechar || nextLine.charAt(i+1) == this.escape);
}
/**
* Reset the buffer used for storing the current field's value
*/
private void clear() {
buf.setLength(0);
}
}

Просмотреть файл

@ -0,0 +1,192 @@
/**
Copyright 2005 Bytecode Pty Ltd.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package opencsv;
import java.io.BufferedReader;
import java.io.Closeable;
import java.io.IOException;
import java.io.Reader;
import java.util.ArrayList;
import java.util.List;
/**
* A very simple CSV reader released under a commercial-friendly license.
*
* @author Glen Smith
*
*/
public class CSVReader implements Closeable {
private final BufferedReader br;
private boolean hasNext = true;
private final CSVParser parser;
private final int skipLines;
private boolean linesSkipped;
/** The line number of the last physical line read (one-based). */
private int curline = 0;
/** The physical line number at which the last logical line read started (one-based). */
private int startLine = 0;
/**
* The default line to start reading.
*/
private static final int DEFAULT_SKIP_LINES = 0;
/**
* Constructs CSVReader using a comma for the separator.
*
* @param reader
* the reader to an underlying CSV source.
*/
public CSVReader(Reader reader) {
this(reader,
CSVParser.DEFAULT_SEPARATOR, CSVParser.DEFAULT_QUOTE_CHARACTER,
CSVParser.DEFAULT_ESCAPE_CHARACTER, DEFAULT_SKIP_LINES,
CSVParser.DEFAULT_STRICT_QUOTES);
}
/**
* Constructs CSVReader with supplied separator and quote char.
*
* @param reader
* the reader to an underlying CSV source.
* @param separator
* the delimiter to use for separating entries
* @param quotechar
* the character to use for quoted elements
* @param escape
* the character to use for escaping a separator or quote
* @param line
* the line number to skip for start reading
* @param strictQuotes
* sets if characters outside the quotes are ignored
*/
private CSVReader(Reader reader, char separator, char quotechar, char escape, int line, boolean strictQuotes) {
this.br = new BufferedReader(reader);
this.parser = new CSVParser(separator, quotechar, escape, strictQuotes);
this.skipLines = line;
}
/**
* Reads the entire file into a List with each element being a String[] of
* tokens.
*
* @return a List of String[], with each String[] representing a line of the
* file.
*
* @throws IOException
* if bad things happen during the read
*/
public List<String[]> readAll() throws IOException {
List<String[]> allElements = new ArrayList<String[]>();
while (hasNext) {
String[] nextLineAsTokens = readNext();
if (nextLineAsTokens != null)
allElements.add(nextLineAsTokens);
}
return allElements;
}
/**
* Reads the next line from the buffer and converts to a string array.
*
* @return a string array with each comma-separated element as a separate
* entry, or null if there are no more lines to read.
*
* @throws IOException
* if bad things happen during the read
*/
public String[] readNext() throws IOException {
boolean first = true;
String[] result = null;
do {
String nextLine = getNextLine();
if (first) {
startLine = curline;
first = false;
}
if (!hasNext) {
return result; // should throw if still pending?
}
String[] r = parser.parseLineMulti(nextLine);
if (r.length > 0) {
if (result == null) {
result = r;
} else {
String[] t = new String[result.length+r.length];
System.arraycopy(result, 0, t, 0, result.length);
System.arraycopy(r, 0, t, result.length, r.length);
result = t;
}
}
} while (parser.isPending());
return result;
}
/**
* Reads the next line from the file.
*
* @return the next line from the file without trailing newline
* @throws IOException
* if bad things happen during the read
*/
private String getNextLine() throws IOException {
if (!this.linesSkipped) {
for (int i = 0; i < skipLines; i++) {
br.readLine();
++curline;
}
this.linesSkipped = true;
}
String nextLine = br.readLine();
if (nextLine == null) {
hasNext = false;
} else {
++curline;
}
return hasNext ? nextLine : null;
}
/**
* Closes the underlying reader.
*
* @throws IOException if the close fails
*/
@Override
public void close() throws IOException{
br.close();
}
/**
* Return the physical line number (one-based) at which the last logical line read started,
* or zero if no line has been read yet.
*/
public int getStartLine() {
return startLine;
}
}

42
extractor/semaphore.go Normal file
Просмотреть файл

@ -0,0 +1,42 @@
package extractor
import (
"log"
)
type Unit struct{}
var unit = Unit{}
type semaphore struct {
counter, lock chan Unit
}
func (s *semaphore) acquire(n int) {
if s != nil {
if cap(s.counter) < n {
log.Fatalf("Tried to acquire more resources than were available.")
}
s.lock <- unit
for i := 0; i < n; i++ {
s.counter <- unit
}
<-s.lock
}
}
func (s *semaphore) release(n int) {
if s != nil {
for i := 0; i < n; i++ {
<-s.counter
}
}
}
func newSemaphore(max int) *semaphore {
if max > 0 {
return &semaphore{make(chan Unit, max), make(chan Unit, 1)}
} else {
return nil
}
}

Просмотреть файл

@ -0,0 +1,105 @@
package srcarchive
import (
"bufio"
"errors"
"fmt"
"os"
"strings"
)
// ProjectLayout describes a very simple project layout rewriting paths starting
// with `from` to start with `to` instead.
//
// We currently only support project layouts of the form
//
// # to
// from//
type ProjectLayout struct {
from, to string
}
// normaliseSlashes adds an initial slash to `path` if there isn't one, and trims
// a final slash if there is one
func normaliseSlashes(path string) string {
if !strings.HasPrefix(path, "/") {
path = "/" + path
}
return strings.TrimSuffix(path, "/")
}
// LoadProjectLayout loads a project layout from the given file, returning an error
// if the file does not have the right format
func LoadProjectLayout(file *os.File) (*ProjectLayout, error) {
res := ProjectLayout{}
scanner := bufio.NewScanner(file)
line := ""
for ; line == "" && scanner.Scan(); line = strings.TrimSpace(scanner.Text()) {
}
if !strings.HasPrefix(line, "#") {
return nil, fmt.Errorf("first line of project layout should start with #, but got %s", line)
}
res.to = normaliseSlashes(strings.TrimSpace(strings.TrimPrefix(line, "#")))
if !scanner.Scan() {
return nil, errors.New("empty section in project-layout file")
}
line = strings.TrimSpace(scanner.Text())
if !strings.HasSuffix(line, "//") {
return nil, errors.New("unsupported project-layout feature")
}
line = strings.TrimSuffix(line, "//")
if strings.HasPrefix(line, "-") || strings.Contains(line, "*") || strings.Contains(line, "//") {
return nil, errors.New("unsupported project-layout feature")
}
res.from = normaliseSlashes(line)
for scanner.Scan() {
if strings.TrimSpace(scanner.Text()) != "" {
return nil, errors.New("only one section with one rewrite supported")
}
}
return &res, nil
}
// transformString transforms `str` as specified by the project layout: if it starts with the `from`
// prefix, that prefix is relaced by `to`; otherwise the string is returned unchanged
func (p *ProjectLayout) transformString(str string) string {
if str == p.from {
return p.to
}
if strings.HasPrefix(str, p.from+"/") {
return p.to + "/" + str[len(p.from)+1:]
}
return str
}
// isWindowsPath checks whether the substring of `path` starting at `idx` looks like a (slashified)
// Windows path, that is, starts with a drive letter followed by a colon and a slash
func isWindowsPath(path string, idx int) bool {
return len(path) >= 3+idx &&
path[idx] != '/' &&
path[idx+1] == ':' && path[idx+2] == '/'
}
// Transform transforms the given path according to the project layout: if it starts with the `from`
// prefix, that prefix is relaced by `to`; otherwise the path is returned unchanged.
//
// Unlike the (internal) method `transformString`, this method handles Windows paths sensibly.
func (p *ProjectLayout) Transform(path string) string {
if isWindowsPath(path, 0) {
result := p.transformString("/" + path)
if isWindowsPath(result, 1) && result[0] == '/' {
return result[1:]
}
return result
} else {
return p.transformString(path)
}
}

Просмотреть файл

@ -0,0 +1,136 @@
package srcarchive
import (
"io/ioutil"
"os"
"testing"
)
func mkProjectLayout(projectLayoutSource string, t *testing.T) (*ProjectLayout, error) {
pt, err := ioutil.TempFile("", "path-transformer")
if err != nil {
t.Fatalf("Unable to create temporary file for project layout: %s", err.Error())
}
defer os.Remove(pt.Name())
_, err = pt.WriteString(projectLayoutSource)
if err != nil {
t.Fatalf("Unable to write to temporary file for project layout: %s", err.Error())
}
err = pt.Close()
if err != nil {
t.Fatalf("Unable to close path transformer file: %s.", err.Error())
}
pt, err = os.Open(pt.Name())
if err != nil {
t.Fatalf("Unable to open path transformer file: %s.", err.Error())
}
return LoadProjectLayout(pt)
}
func testTransformation(projectLayout *ProjectLayout, t *testing.T, path string, expected string) {
actual := projectLayout.Transform(path)
if actual != expected {
t.Errorf("Expected %s to be transformed to %s, but got %s", path, expected, actual)
}
}
func TestValidProjectLayout(t *testing.T) {
p, err := mkProjectLayout(`
# /opt/src
/opt/src/root/src/org/repo//
`, t)
if err != nil {
t.Fatalf("Error loading project layout: %s", err.Error())
}
testTransformation(p, t, "/opt/src/root/src/org/repo", "/opt/src")
testTransformation(p, t, "/opt/src/root/src/org/repo/", "/opt/src/")
testTransformation(p, t, "/opt/src/root/src/org/repo/main.go", "/opt/src/main.go")
testTransformation(p, t, "/opt/not/in/src", "/opt/not/in/src")
testTransformation(p, t, "/opt/src/root/srcorg/repo", "/opt/src/root/srcorg/repo")
testTransformation(p, t, "opt/src/root/src/org/repo", "opt/src/root/src/org/repo")
}
func TestWindowsPaths(t *testing.T) {
p, err := mkProjectLayout(`
# /c:/virtual
/d://
`, t)
if err != nil {
t.Fatalf("Error loading project layout: %s", err.Error())
}
testTransformation(p, t, "d:/foo", "c:/virtual/foo")
}
func TestWindowsToUnixPaths(t *testing.T) {
p, err := mkProjectLayout(`
# /opt/src
/d://
`, t)
if err != nil {
t.Fatalf("Error loading project layout: %s", err.Error())
}
testTransformation(p, t, "d:/foo", "/opt/src/foo")
}
func TestEmptyProjectLayout(t *testing.T) {
_, err := mkProjectLayout("", t)
if err == nil {
t.Error("Expected error on empty project layout")
}
}
func TestEmptyProjectLayout2(t *testing.T) {
_, err := mkProjectLayout(`
`, t)
if err == nil {
t.Error("Expected error on empty project layout")
}
}
func TestExclusion(t *testing.T) {
_, err := mkProjectLayout(`
# /opt/src
-/foo//
`, t)
if err == nil {
t.Error("Expected error on exclusion")
}
}
func TestStar(t *testing.T) {
_, err := mkProjectLayout(`
# /opt/src
/foo/**/bar//
`, t)
if err == nil {
t.Error("Expected error on star")
}
}
func TestDoubleSlash(t *testing.T) {
_, err := mkProjectLayout(`
# /opt/src
/foo//bar//
`, t)
if err == nil {
t.Error("Expected error on multiple double slashes")
}
}
func TestInternalDoubleSlash(t *testing.T) {
_, err := mkProjectLayout(`
# /opt/src
/foo//bar
`, t)
if err == nil {
t.Error("Expected error on internal double slash")
}
}

Просмотреть файл

@ -0,0 +1,82 @@
package srcarchive
import (
"errors"
"io"
"log"
"os"
"path/filepath"
"strings"
)
var pathTransformer *ProjectLayout
func init() {
pt := os.Getenv("SEMMLE_PATH_TRANSFORMER")
if pt != "" {
ptf, err := os.Open(pt)
if err != nil {
log.Fatalf("Unable to open path transformer %s: %s.\n", pt, err.Error())
}
pathTransformer, err = LoadProjectLayout(ptf)
if err != nil {
log.Fatalf("Unable to initialize path transformer: %s.\n", err.Error())
}
}
}
// Add inserts the file with the given `path` into the source archive, returning a non-nil
// error value if it fails
func Add(path string) error {
srcArchive, err := srcArchive()
if err != nil {
return err
}
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
archiveFilePath := filepath.Join(srcArchive, AppendablePath(path))
err = os.MkdirAll(filepath.Dir(archiveFilePath), 0755)
if err != nil {
return err
}
archiveFile, err := os.Create(archiveFilePath)
if err != nil {
return err
}
defer archiveFile.Close()
_, err = io.Copy(archiveFile, file)
return err
}
func srcArchive() (string, error) {
srcArchive := os.Getenv("SOURCE_ARCHIVE")
if srcArchive == "" {
return "", errors.New("environment variable SOURCE_ARCHIVE not set")
}
err := os.MkdirAll(srcArchive, 0755)
if err != nil {
return "", err
}
return srcArchive, nil
}
// TransformPath applies the transformations specified by `SEMMLE_PATH_TRANSFORMER` (if any) to the
// given path
func TransformPath(path string) string {
if pathTransformer != nil {
return filepath.FromSlash(pathTransformer.Transform(filepath.ToSlash(path)))
}
return path
}
// AppendablePath transforms the given path and also replaces colons with underscores to make it
// possible to append it to a base path on Windows
func AppendablePath(path string) string {
return strings.ReplaceAll(TransformPath(path), ":", "_")
}

216
extractor/trap/labels.go Normal file
Просмотреть файл

@ -0,0 +1,216 @@
package trap
import (
"fmt"
"go/ast"
"go/types"
)
// Label represents a label
type Label struct {
id string
}
// InvalidLabel represents an uninitialized or otherwise invalid label
var InvalidLabel Label
func (lbl *Label) String() string {
return lbl.id
}
// Labeler is used to represent labels for a file. It is used to write
// associate objects with labels.
type Labeler struct {
tw *Writer
nextid int
fileLabel Label
nodeLabels map[ast.Node]Label // labels associated with AST nodes
scopeLabels map[*types.Scope]Label // labels associated with scopes
objectLabels map[types.Object]Label // labels associated with objects (that is, declared entities)
TypeLabels map[types.Type]Label // labels associated with types
keyLabels map[string]Label
}
func newLabeler(tw *Writer) *Labeler {
return &Labeler{
tw,
10000,
InvalidLabel,
make(map[ast.Node]Label),
make(map[*types.Scope]Label),
make(map[types.Object]Label),
make(map[types.Type]Label),
make(map[string]Label),
}
}
func (l *Labeler) nextID() string {
var id = l.nextid
l.nextid++
return fmt.Sprintf("#%d", id)
}
// GlobalID associates a label with the given `key` and returns it
func (l *Labeler) GlobalID(key string) Label {
label, exists := l.keyLabels[key]
if !exists {
id := l.nextID()
fmt.Fprintf(l.tw.w, "%s=@\"%s\"\n", id, escapeString(key))
label = Label{id}
l.keyLabels[key] = label
}
return label
}
// FileLabel returns the label for the file with which the trap writer is associated
func (l *Labeler) FileLabel() Label {
if l.fileLabel == InvalidLabel {
l.fileLabel = l.GlobalID(l.tw.path + ";sourcefile")
}
return l.fileLabel
}
// LocalID associates a label with the given AST node `nd` and returns it
func (l *Labeler) LocalID(nd ast.Node) Label {
label, exists := l.nodeLabels[nd]
if !exists {
label = l.FreshID()
l.nodeLabels[nd] = label
}
return label
}
// FreshID creates a fresh label and returns it
func (l *Labeler) FreshID() Label {
id := l.nextID()
fmt.Fprintf(l.tw.w, "%s=*\n", id)
return Label{id}
}
// ScopeID associates a label with the given scope and returns it
func (l *Labeler) ScopeID(scope *types.Scope, pkg *types.Package) Label {
label, exists := l.scopeLabels[scope]
if !exists {
if scope == types.Universe {
label = l.GlobalID("universe;scope")
} else {
if pkg != nil && pkg.Scope() == scope {
// if this scope is the package scope
pkgLabel := l.GlobalID(pkg.Path() + ";package")
label = l.GlobalID("{" + pkgLabel.String() + "};scope")
} else {
label = l.FreshID()
}
}
l.scopeLabels[scope] = label
}
return label
}
// LookupObjectID looks up the label associated with the given object and returns it; if the object does not have
// a label yet, it tries to construct one based on its scope and/or name, and otherwise returns InvalidLabel
func (l *Labeler) LookupObjectID(object types.Object, typelbl Label) (Label, bool) {
label, exists := l.objectLabels[object]
if !exists {
if object.Parent() == nil {
// blank identifiers and the pseudo-package `.` (from `import . "..."` imports) can only be referenced
// once, so we can use a fresh label for them
if object.Name() == "_" || object.Name() == "." {
label = l.FreshID()
l.objectLabels[object] = label
return label, false
}
label = InvalidLabel
} else {
label, exists = l.ScopedObjectID(object, typelbl)
}
}
return label, exists
}
// ScopedObjectID associates a label with the given object and returns it,
// together with a flag indicating whether the object already had a label
// associated with it; the object must have a scope, since the scope's label is
// used to construct the label of the object.
//
// There is a special case for variables that are method receivers. When this is
// detected, we must construct a special label, as the variable can be reached
// from several files via the method. As the type label is required to construct
// the receiver object id, it is also required here.
func (l *Labeler) ScopedObjectID(object types.Object, typelbl Label) (Label, bool) {
label, exists := l.objectLabels[object]
if !exists {
scope := object.Parent()
if scope == nil {
panic(fmt.Sprintf("Object has no scope: %v :: %v.\n", object,
l.tw.Package.Fset.Position(object.Pos())))
} else {
// associate method receiver objects to special keys, because those can be
// referenced from other files via their method
isRecv := false
if namedType, ok := object.Type().(*types.Named); ok {
for i := 0; i < namedType.NumMethods(); i++ {
meth := namedType.Method(i)
if object == meth.Type().(*types.Signature).Recv() {
isRecv = true
methlbl, _ := l.MethodID(meth, typelbl)
label, _ = l.ReceiverObjectID(object, methlbl)
}
}
}
if !isRecv {
scopeLbl := l.ScopeID(scope, object.Pkg())
label = l.GlobalID(fmt.Sprintf("{%s},%s;object", scopeLbl.String(), object.Name()))
}
}
l.objectLabels[object] = label
}
return label, exists
}
// ReceiverObjectID associates a label with the given object and returns it, together with a flag indicating whether
// the object already had a label associated with it; the object must be the receiver of `methlbl`, since that label
// is used to construct the label of the object
func (l *Labeler) ReceiverObjectID(object types.Object, methlbl Label) (Label, bool) {
label, exists := l.objectLabels[object]
if !exists {
// if we can't, construct a special label
label = l.GlobalID(fmt.Sprintf("{%s},%s;receiver", methlbl.String(), object.Name()))
l.objectLabels[object] = label
}
return label, exists
}
// FieldID associates a label with the given field and returns it, together with
// a flag indicating whether the field already had a label associated with it;
// the field must belong to `structlbl`, since that label is used to construct
// the label of the field. When the field name is the blank identifier `_`,
// `idx` is used to generate a unique name.
func (l *Labeler) FieldID(field *types.Var, idx int, structlbl Label) (Label, bool) {
label, exists := l.objectLabels[field]
if !exists {
name := field.Name()
// there can be multiple fields with the blank identifier, so use index to
// distinguish them
if field.Name() == "_" {
name = fmt.Sprintf("_%d", idx)
}
label = l.GlobalID(fmt.Sprintf("{%s},%s;field", structlbl.String(), name))
l.objectLabels[field] = label
}
return label, exists
}
// MethodID associates a label with the given method and returns it, together with a flag indicating whether
// the method already had a label associated with it; the method must belong to `recvlbl`, since that label
// is used to construct the label of the method
func (l *Labeler) MethodID(method types.Object, recvlbl Label) (Label, bool) {
label, exists := l.objectLabels[method]
if !exists {
label = l.GlobalID(fmt.Sprintf("{%s},%s;method", recvlbl, method.Name()))
l.objectLabels[method] = label
}
return label, exists
}

Просмотреть файл

@ -0,0 +1,106 @@
package trap
import (
"errors"
"fmt"
"go/types"
"io/ioutil"
"os"
"path/filepath"
"github.com/Semmle/go/extractor/srcarchive"
"golang.org/x/tools/go/packages"
)
// A Writer provides methods for writing data to a TRAP file
type Writer struct {
w *os.File
Labeler *Labeler
path string
trapFilePath string
Package *packages.Package
}
// NewWriter creates a TRAP file for the given path and returns a writer for
// writing to it
func NewWriter(path string, pkg *packages.Package) (*Writer, error) {
trapFolder, err := trapFolder()
if err != nil {
return nil, err
}
trapFilePath := filepath.Join(trapFolder, srcarchive.AppendablePath(path)+".trap")
trapFileDir := filepath.Dir(trapFilePath)
err = os.MkdirAll(trapFileDir, 0755)
if err != nil {
return nil, err
}
tmpFile, err := ioutil.TempFile(trapFileDir, filepath.Base(trapFilePath))
if err != nil {
return nil, err
}
tw := &Writer{
tmpFile,
nil,
path,
trapFilePath,
pkg,
}
tw.Labeler = newLabeler(tw)
return tw, nil
}
func trapFolder() (string, error) {
trapFolder := os.Getenv("TRAP_FOLDER")
if trapFolder == "" {
return "", errors.New("environment variable TRAP_FOLDER not set")
}
err := os.MkdirAll(trapFolder, 0755)
if err != nil {
return "", err
}
return trapFolder, nil
}
// Close the underlying file writer
func (tw *Writer) Close() error {
err := tw.w.Sync()
if err != nil {
return err
}
err = tw.w.Close()
if err != nil {
return err
}
return os.Rename(tw.w.Name(), tw.trapFilePath)
}
// ForEachObject iterates over all objects labeled by this labeler, and invokes
// the provided callback with a writer for the trap file, the object, and its
// label.
func (tw *Writer) ForEachObject(cb func(*Writer, types.Object, Label)) {
for object, lbl := range tw.Labeler.objectLabels {
cb(tw, object, lbl)
}
}
// Emit writes out a tuple of values for the given `table`
func (tw *Writer) Emit(table string, values []interface{}) error {
fmt.Fprintf(tw.w, "%s(", table)
for i, value := range values {
if i > 0 {
fmt.Fprint(tw.w, ", ")
}
switch value := value.(type) {
case Label:
fmt.Fprint(tw.w, value.id)
case string:
fmt.Fprintf(tw.w, "\"%s\"", escapeString(value))
case int:
fmt.Fprintf(tw.w, "%d", value)
default:
return errors.New("Cannot emit value")
}
}
fmt.Fprintf(tw.w, ")\n")
return nil
}

9
extractor/trap/util.go Normal file
Просмотреть файл

@ -0,0 +1,9 @@
package trap
import (
"strings"
)
func escapeString(s string) string {
return strings.Replace(s, "\"", "\"\"", -1)
}

5
go.mod Normal file
Просмотреть файл

@ -0,0 +1,5 @@
module github.com/Semmle/go
go 1.13
require golang.org/x/tools v0.0.0-20191030225452-7871c2d76733

9
go.sum Normal file
Просмотреть файл

@ -0,0 +1,9 @@
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/tools v0.0.0-20191030225452-7871c2d76733 h1:wtYExk7epHk5WDdLiCO92FIXY5eiMtZqV1RMSLiR/3M=
golang.org/x/tools v0.0.0-20191030225452-7871c2d76733/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7 h1:9zdDQZ7Thm29KFXgAX/+yaf3eVbP7djjWp/dXAppNCc=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

Просмотреть файл

@ -0,0 +1,3 @@
name: legacy-libraries-go
version: 0.0.0
libraryPathDependencies: codeql-go

Просмотреть файл

@ -0,0 +1,3 @@
# DO NOT EDIT
# This is a stub file. The actual suite of queries to run is generated
# automatically based on query precision and severity.

Просмотреть файл

@ -0,0 +1,3 @@
@import "go-alerts-lgtm"
@import "go-metrics-lgtm"
@import "go-util-lgtm"

Просмотреть файл

@ -0,0 +1,2 @@
+ go-queries/Metrics/FLinesOfCode.ql: /Metrics/Files
@_namespace com.lgtm/go-queries

Просмотреть файл

@ -0,0 +1,6 @@
+ go-queries/definitions.ql
@_namespace com.lgtm/go-queries
+ go-queries/AlertSuppression.ql
@_namespace com.lgtm/go-queries
+ go-queries/filters/ClassifyFiles.ql
@_namespace com.lgtm/go-queries

12
ql/src/.project Normal file
Просмотреть файл

@ -0,0 +1,12 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>go-queries</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
</buildSpec>
<natures>
<nature>com.semmle.plugin.qdt.core.qlnature</nature>
</natures>
</projectDescription>

5
ql/src/.qlpath Normal file
Просмотреть файл

@ -0,0 +1,5 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ns:qlpath xmlns:ns="https://semmle.com/schemas/qlpath">
<librarypath/>
<dbscheme>/go-queries/go.dbscheme</dbscheme>
</ns:qlpath>

Просмотреть файл

@ -0,0 +1,78 @@
/**
* @name Alert suppression
* @description Generates information about alert suppressions.
* @kind alert-suppression
* @id go/alert-suppression
*/
import go
/**
* An alert suppression comment.
*/
class SuppressionComment extends Locatable {
string text;
string annotation;
SuppressionComment() {
text = this.(LineComment).getText() and
(
// match `lgtm[...]` anywhere in the comment
annotation = text.regexpFind("(?i)\\blgtm\\s*\\[[^\\]]*\\]", _, _)
or
// match `lgtm` at the start of the comment and after semicolon
annotation = text.regexpFind("(?i)(?<=^|;)\\s*lgtm(?!\\B|\\s*\\[)", _, _).trim()
)
}
/** Gets the text of this suppression comment, not including delimiters. */
string getText() { result = text }
/** Gets the suppression annotation in this comment. */
string getAnnotation() { result = annotation }
/**
* Holds if this comment applies to the range from column `startcolumn` of line `startline`
* to column `endcolumn` of line `endline` in file `filepath`.
*/
predicate covers(string filepath, int startline, int startcolumn, int endline, int endcolumn) {
this.getLocation().hasLocationInfo(filepath, startline, _, endline, endcolumn) and
startcolumn = 1
}
/** Gets the scope of this suppression. */
SuppressionScope getScope() { this = result.getSuppressionComment() }
}
/**
* The scope of an alert suppression comment.
*/
class SuppressionScope extends @locatable {
SuppressionScope() { this instanceof SuppressionComment }
/** Gets a suppression comment with this scope. */
SuppressionComment getSuppressionComment() { result = this }
/**
* Holds if this element is at the specified location.
* The location spans column `startcolumn` of line `startline` to
* column `endcolumn` of line `endline` in file `filepath`.
* For more information, see
* [Locations](https://help.semmle.com/QL/learn-ql/ql/locations.html).
*/
predicate hasLocationInfo(
string filepath, int startline, int startcolumn, int endline, int endcolumn
) {
this.(SuppressionComment).covers(filepath, startline, startcolumn, endline, endcolumn)
}
/** Gets a textual representation of this element. */
string toString() { result = "suppression range" }
}
from SuppressionComment c
select c, // suppression comment
c.getText(), // text of suppression comment (excluding delimiters)
c.getAnnotation(), // text of suppression annotation
c.getScope() // scope of suppression

12
ql/src/Customizations.qll Normal file
Просмотреть файл

@ -0,0 +1,12 @@
/**
* Contains customizations to the standard library.
*
* This module is imported by `go.qll`, so any customizations defined here automatically
* apply to all queries.
*
* Typical examples of customizations include adding new subclasses of abstract classes such as
* `FileSystemAccess`, or the `Source` and `Sink` classes associated with the security queries
* to model frameworks that are not covered by the standard library.
*/
import go

Просмотреть файл

@ -0,0 +1,13 @@
package main
func zeroOutExcept(a []int, lower int, upper int) {
// zero out everything below index `lower`
for i := lower - 1; i >= 0; i-- {
a[i] = 0
}
// zero out everything above index `upper`
for i := upper + 1; i < len(a); i-- {
a[i] = 0
}
}

Просмотреть файл

@ -0,0 +1,53 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
Most <code>for</code> loops either increment a variable until an upper bound is reached,
or decrement a variable until a lower bound is reached. If, instead, the variable is
incremented but checked against a lower bound, or decremented but checked against an
upper bound, then the loop will usually either terminate immediately and never execute
its body, or it will keep iterating indefinitely. Neither is likely to be intentional,
and is most likely the result of a typo.
</p>
<p>
The only exception to this are loops whose loop variable is an unsigned integer. In this
case, initializing the variable with an upper bound and then decrementing it while
checking against the same upper bound is unproblematic: the variable is counted down
from the upper bound to zero, and then wraps around to a large positive value, causing
the loop to terminate. This is usually intentional, and hence is not flagged by the query.
</p>
</overview>
<recommendation>
<p>
Examine the loop carefully to check whether its test expression or update statement
are erroneous.
</p>
</recommendation>
<example>
<p>
In the following example, two loops are used to set all elements of a slice <code>a</code>
outside a range <code>lower</code>..<code>upper</code> to zero. However, the second loop
contains a typo: the loop variable <code>i</code> is decremented instead of incremented,
so <code>i</code> is counted downwards from <code>upper+1</code> to <code>0</code>, <code>-1</code>,
<code>-2</code> and so on.
</p>
<sample src="InconsistentLoopOrientation.go" />
<p>
To correct this issue, change the second loop to increment its loop variable instead:
</p>
<sample src="InconsistentLoopOrientationGood.go" />
</example>
<references>
<li>The Go Programming Language Specification: <a href="https://golang.org/ref/spec#For_statements">For statements</a>.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,55 @@
/**
* @name Inconsistent direction of for loop
* @description A 'for' loop that increments its loop variable but checks it
* against a lower bound, or decrements its loop variable but
* checks it against an upper bound, will either stop iterating
* immediately or keep iterating indefinitely, and is usually
* indicative of a typo.
* @kind problem
* @problem.severity error
* @id go/inconsistent-loop-direction
* @tags correctness
* external/cwe/cwe-835
* @precision very-high
*/
import go
/**
* Holds if `test` bounds `v` in `direction`, which is either `"upward"`
* or `"downward"`.
*
* For example, `x < 42` bounds `x` upward, while `y >= 0` bounds `y`
* downward.
*/
predicate bounds(RelationalComparisonExpr test, Variable v, string direction) {
test.getLesserOperand() = v.getAUse() and direction = "upward"
or
test.getGreaterOperand() = v.getAUse() and direction = "downward"
}
/**
* Holds if `upd` updates `v` in `direction`, which is either `"upward"`
* or `"downward"`.
*
* For example, `++x` updates `x` upward, while `y--` updates `y`
* downward.
*/
predicate updates(IncDecStmt upd, Variable v, string direction) {
upd.getExpr() = v.getAUse() and
(
upd instanceof IncStmt and direction = "upward"
or
upd instanceof DecStmt and direction = "downward"
)
}
from ForStmt l, Variable v, string d1, string d2
where
bounds(l.getCond(), v, d1) and
updates(l.getPost(), v, d2) and
d1 != d2 and
// `for u = n; u <= n; u--` is a somewhat common idiom
not (v.getType().getUnderlyingType() instanceof UnsignedIntegerType and d2 = "downward")
select l.getPost(), "This loop counts " + d2 + ", but its variable is $@ " + d1 + ".", l.getCond(),
"bounded"

Просмотреть файл

@ -0,0 +1,13 @@
package main
func zeroOutExcept(a []int, lower int, upper int) {
// zero out everything below index `lower`
for i := lower - 1; i >= 0; i-- {
a[i] = 0
}
// zero out everything above index `upper`
for i := upper + 1; i < len(a); i++ {
a[i] = 0
}
}

Просмотреть файл

@ -0,0 +1,15 @@
package main
import "strings"
func containsBad(searchName string, names string) bool {
values := strings.Split(names, ",")
// BAD: index could be equal to length
for i := 0; i <= len(values); i++ {
// When i = length, this access will be out of bounds
if values[i] == searchName {
return true
}
}
return false
}

Просмотреть файл

@ -0,0 +1,47 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
Indexing operations on arrays, slices or strings should use an index at most one less
than the length. If the index to be accessed is checked for being less than or equal to the length
(<code>&lt;=</code>), instead of less than the length (<code>&lt;</code>), the index could be out
of bounds.
</p>
</overview>
<recommendation>
<p>
Use less than (<code>&lt;</code>) rather than less than or equals (<code>&lt;=</code>) when
comparing a potential index against a length. For loops that iterate over every element, a better
solution is to use a <code>range</code> loop instead of looping over explicit indexes.
</p>
</recommendation>
<example>
<p>
The following example shows a method which checks whether a value appears in a comma-separated
list of values:
</p>
<sample src="LengthComparisonOffByOne.go" />
<p>
A loop using an index variable <code>i</code> is used to iterate over the elements in the comma-separated
list. However, the terminating condition of the loop is incorrectly specified as <code>i &lt;= len(values)</code>.
This condition holds when <code>i</code> is equal to <code>len(values)</code>, but the access <code>values[i]</code>
in the body of the loop will be out of bounds in this case.
</p>
<p>
One potential solution would be to replace <code>i &lt;= len(values)</code> with
<code>i &lt; len(values)</code>. A better solution is to use a <code>range</code> loop instead, which
avoids the need for explicitly manipulating the index variable:
</p>
<sample src="LengthComparisonOffByOneGood.go" />
</example>
<references>
<li>The Go Programming Language Specification: <a href="https://golang.org/ref/spec#For_statements">For statements</a>.</li>
<li>The Go Programming Language Specification: <a href="https://golang.org/ref/spec#Index_expressions">Index expressions</a>.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,100 @@
/**
* @name Off-by-one comparison against length
* @description An array index is compared with the length of the array,
* and then used in an indexing operation that could be out of bounds.
* @kind problem
* @problem.severity error
* @id go/index-out-of-bounds
* @tags reliability
* correctness
* logic
* external/cwe/cwe-193
* @precision high
*/
import go
newtype TIndex =
VariableIndex(DataFlow::SsaNode v) { v.getAUse() = any(DataFlow::ElementReadNode e).getIndex() } or
ConstantIndex(int v) { v = any(DataFlow::ElementReadNode e).getIndex().getIntValue() + [-1 .. 1] }
class Index extends TIndex {
string toString() {
exists(DataFlow::SsaNode v | this = VariableIndex(v) | result = v.getSourceVariable().getName())
or
exists(int v | this = ConstantIndex(v) | result = v.toString())
}
}
DataFlow::Node getAUse(Index i) {
i = VariableIndex(any(DataFlow::SsaNode v | result = v.getAUse()))
or
i = ConstantIndex(any(int v | v = result.getIntValue()))
}
/**
* Gets a call to `len(array)`.
*/
DataFlow::CallNode arrayLen(DataFlow::SsaNode array) {
result = Builtin::len().getACall() and
result.getArgument(0) = array.getAUse()
}
/**
* Gets a condition that checks that `index` is less than or equal to `array.length`.
*/
ControlFlow::ConditionGuardNode getLengthLEGuard(Index index, DataFlow::SsaNode array) {
result.ensuresLeq(getAUse(index), arrayLen(array), 0)
or
exists(int i, int bias | index = ConstantIndex(i) |
result.ensuresLeq(getAUse(ConstantIndex(i + bias)), arrayLen(array), bias)
)
}
/**
* Gets a condition that checks that `index` is not equal to `array.length`.
*/
ControlFlow::ConditionGuardNode getLengthNEGuard(Index index, DataFlow::SsaNode array) {
result.ensuresNeq(getAUse(index), arrayLen(array))
}
/**
* Holds if `ea` is a read from `array[index]` in basic block `bb`.
*/
predicate elementRead(
DataFlow::ElementReadNode ea, DataFlow::SsaNode array, Index index, BasicBlock bb
) {
ea.reads(array.getAUse(), getAUse(index)) and
not array.getType().getUnderlyingType() instanceof MapType and
bb = ea.asInstruction().getBasicBlock()
}
predicate isRegexpMethodCall(DataFlow::MethodCallNode c) {
exists(NamedType regexp, Type recvtp |
regexp.getName() = "Regexp" and recvtp = c.getReceiver().getType()
|
recvtp = regexp or recvtp.(PointerType).getBaseType() = regexp
)
}
from
ControlFlow::ConditionGuardNode cond, DataFlow::SsaNode array, Index index,
DataFlow::ElementReadNode ea, BasicBlock bb
where
// there is a comparison `index <= len(array)`
cond = getLengthLEGuard(index, array) and
// there is a read from `array[index]`
elementRead(ea, array, index, bb) and
// and the read is guarded by the comparison
cond.dominates(bb) and
// but the read is not guarded by another check that `index != len(array)`
not getLengthNEGuard(index, array).dominates(bb) and
// and it is not additionally guarded by a stronger index check
not exists(Index index2, int i, int i2 |
index = ConstantIndex(i) and index2 = ConstantIndex(i2) and i < i2
|
getLengthLEGuard(index2, array).dominates(bb)
) and
not isRegexpMethodCall(array.getInit())
select cond.getCondition(),
"Off-by-one index comparison against length may lead to out-of-bounds $@.", ea, "read"

Просмотреть файл

@ -0,0 +1,14 @@
package main
import "strings"
func containsGood(searchName string, names string) bool {
values := strings.Split(names, ",")
// GOOD: Avoid using indexes, use range loop instead
for _, name := range values {
if name == searchName {
return true
}
}
return true
}

Просмотреть файл

@ -0,0 +1,7 @@
package main
import "fmt"
func main() {
fmt.Println(2 ^ 32) // should be 1 << 32
}

Просмотреть файл

@ -0,0 +1,36 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
The caret symbol (<code>^</code>) is sometimes used to represent
exponentiation but in Go, as in many C-like languages, it represents the
bitwise exclusive-or operation. The expression as <code>2^32</code> thus
evaluates the number 34, not 2<sup>32</sup>, and it is likely that patterns
such as this are mistakes.
</p>
</overview>
<recommendation>
<p>
To compute 2<sup><code>EXP</code></sup>, <code>1 &lt;&lt; EXP</code> can be
used. For constant exponents, <code>1eEXP</code> can be used to find
10<sup><code>EXP</code></sup>. In other cases, there is <code>math.Pow</code>
in the Go standard library which provides this functionality.
</p>
</recommendation>
<example>
<p>
The example below prints 34 and not 2<sup>32</sup> (4294967296).
</p>
<sample src="MistypedExponentiation.go" />
</example>
<references>
<li>GCC Bugzilla: <a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90885">GCC should warn about 2^16 and 2^32 and 2^64</a></li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,37 @@
/**
* @name Bitwise exclusive-or used like exponentiation
* @description Using ^ as exponentiation is a mistake, as it is the bitwise exclusive-or operator.
* @kind problem
* @problem.severity warning
* @id go/mistyped-exponentiation
* @precision high
*/
import go
/** Holds if `e` is not 0 and is either an octal or hexadecimal literal, or the number one. */
predicate maybeXorBitPattern(Expr e) {
// 0 makes no sense as an xor bit pattern
not e.getNumericValue() = 0 and
// include octal and hex literals
e.(IntLit).getText().matches("0%")
or
e.getNumericValue() = 1
}
from XorExpr xe, Expr lhs, Expr rhs
where
lhs = xe.getLeftOperand() and
rhs = xe.getRightOperand() and
exists(lhs.getNumericValue()) and
not maybeXorBitPattern(lhs) and
(
not maybeXorBitPattern(rhs) and
rhs.getIntValue() >= 0
or
exists(Ident id | id = xe.getRightOperand() |
id.getName().regexpMatch("(?i)_*((exp(onent)?)|pow(er)?)")
)
)
select xe,
"This expression uses the bitwise exclusive-or operator when exponentiation was likely meant."

Просмотреть файл

@ -0,0 +1,5 @@
package main
func isBitSetBad(x int, pos uint) bool {
return x&1<<pos != 0
}

Просмотреть файл

@ -0,0 +1,44 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
Nested expressions where the spacing around operators suggests a different
grouping than that imposed by the Go operator precedence rules are problematic:
they could indicate a bug where the author of the code misunderstood the precedence
rules. Even if there is no a bug, the spacing could be confusing to people who
read the code.
</p>
</overview>
<recommendation>
<p>
Make sure that the spacing around operators reflects operator precedence, or use parentheses to
clarify grouping.
</p>
</recommendation>
<example>
<p>
Consider the following function intended for checking whether the bit at position `pos` of the variable `x`
is set:
</p>
<sample src="WhitespaceContradictsPrecedence.go" />
<p>
Here, the spacing around <code>&amp;</code> and <code>&lt;&lt;</code> suggests the grouping
<code>x &amp; (1&lt;&lt;pos)</code>. However, in Go <code>&amp;</code> and <code>&lt;&lt;</code> have
the same precedence and hence are evaluated left to right, so the expression is actually equivalent to
<code>(x &amp; 1) &lt;&lt; pos</code>.
</p>
<p>
To fix this issue and give the expression its intended semantics, parentheses should be used like this:
</p>
<sample src="WhitespaceContradictsPrecedenceGood.go" />
</example>
<references>
<li>The Go Programming Language Specification: <a href="https://golang.org/ref/spec#Operator_precedence">Operator precedence</a>.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,88 @@
/**
* @name Whitespace contradicts operator precedence
* @description Nested expressions where the formatting contradicts the grouping enforced by operator precedence
* are difficult to read and may even indicate a bug.
* @kind problem
* @problem.severity warning
* @id go/whitespace-contradicts-precedence
* @tags maintainability
* correctness
* external/cwe/cwe-783
* @precision very-high
*/
import go
/**
* A nested associative expression.
*
* That is, a binary expression of the form `x op y`, which is itself an operand
* (say, the left) of another binary expression `(x op y) op' z` such that
* `(x op y) op' z = x op (y op' z)`, disregarding overflow.
*/
class AssocNestedExpr extends BinaryExpr {
AssocNestedExpr() {
exists(BinaryExpr parent, int idx | this = parent.getChildExpr(idx) |
// +, *, &&, || and the bitwise operations are associative
(
this instanceof AddExpr or
this instanceof MulExpr or
this instanceof BitwiseExpr or
this instanceof LogicalBinaryExpr
) and
parent.getOperator() = this.getOperator()
or
// (x*y)/z = x*(y/z)
this instanceof MulExpr and parent instanceof DivExpr and idx = 0
or
// (x/y)%z = x/(y%z)
this instanceof DivExpr and parent instanceof ModExpr and idx = 0
or
// (x+y)-z = x+(y-z)
this instanceof AddExpr and parent instanceof SubExpr and idx = 0
)
}
}
/**
* A binary expression nested inside another binary expression where the relative
* precedence of the two operators is unlikely to cause confusion.
*/
class HarmlessNestedExpr extends BinaryExpr {
HarmlessNestedExpr() {
exists(BinaryExpr parent | this = parent.getAChildExpr() |
parent instanceof Comparison and
(this instanceof ArithmeticExpr or this instanceof ShiftExpr)
or
parent instanceof LogicalExpr and this instanceof Comparison
)
}
}
/**
* Holds if `inner` is an operand of `outer`, and the relative precedence
* may not be immediately clear, but is important for the semantics of
* the expression (that is, the operators are not associative).
*/
predicate interestingNesting(BinaryExpr inner, BinaryExpr outer) {
inner = outer.getAChildExpr() and
not inner instanceof AssocNestedExpr and
not inner instanceof HarmlessNestedExpr
}
/** Gets the number of whitespace characters around the operator `op` of `be`. */
int getWhitespaceAroundOperator(BinaryExpr be, string op) {
exists(string file, int line, int left, int right |
be.getLeftOperand().hasLocationInfo(file, _, _, line, left) and
be.getRightOperand().hasLocationInfo(file, line, right, _, _) and
op = be.getOperator() and
result = (right - left - op.length() - 1) / 2
)
}
from BinaryExpr inner, BinaryExpr outer, string innerOp, string outerOp
where
interestingNesting(inner, outer) and
getWhitespaceAroundOperator(inner, innerOp) > getWhitespaceAroundOperator(outer, outerOp)
select outer,
innerOp + " is evaluated before " + outerOp + ", but whitespace suggests the opposite."

Просмотреть файл

@ -0,0 +1,5 @@
package main
func isBitSetGood(x int, pos uint) bool {
return x&(1<<pos) != 0
}

Просмотреть файл

@ -0,0 +1,38 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
There are a number of problems associated with a high number of lines of code:
</p>
<ul>
<li>It can be difficult to understand and maintain, even with good tool support.</li>
<li>It increases the likelihood of multiple developers needing to work on the same
file at once, and it therefore increases the likelihood of merge conflicts.</li>
<li>It may increase network traffic if you use a version control system that requires
the whole file to be transmitted even for a tiny change.</li>
<li>It may arise as a result of bundling many unrelated things into the same file,
and so it can indicate weak code organization.</li>
</ul>
</overview>
<recommendation>
<p>
The solution depends on the reason for the high number of lines:
</p>
<ul>
<li>If the file contains one or more very large functions, you should decompose them
into smaller functions by means of the Extract Function refactoring.</li>
<li>If the file contains many smaller functions, you should try to split up the file
into multiple smaller files.</li>
<li>If the file has been automatically generated by a tool, no changes are required
because the file will not be maintained by a programmer.</li>
</ul>
</recommendation>
<references>
<li>M. Fowler, <em>Refactoring</em>. Addison-Wesley, 1999.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,18 @@
/**
* @name Lines of code in files
* @description Measures the number of lines of code in each file, ignoring lines that
* contain only comments or whitespace.
* @kind metric
* @treemap.warnOn highValues
* @metricType file
* @metricAggregate avg sum max
* @precision very-high
* @id go/lines-of-code-in-files
* @tags maintainability
*/
import go
from File f, int n
where n = f.getNumberOfLinesOfCode()
select f, n order by n desc

Просмотреть файл

@ -0,0 +1,24 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
This metric measures the number of comment lines per file. A low number of comments
may indicate files that are difficult to understand due to poor documentation.
</p>
</overview>
<recommendation>
<p>
Consider if the file needs more documentation. Most files should have at least a comment
explaining their purpose.
</p>
</recommendation>
<references>
<li>Jeff Atwood. <a href="http://www.codinghorror.com/blog/2005/11/avoiding-undocumentation.html">Avoiding Undocumentation</a>. 2005.</li>
<li>Steve McConnell. <em>Code Complete</em>. 2nd Edition. Microsoft Press. 2004.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,17 @@
/**
* @name Lines of comments in files
* @description Files with few lines of comment might not have sufficient documentation
* to make them understandable.
* @kind metric
* @treemap.warnOn lowValues
* @metricType file
* @metricAggregate avg sum max
* @precision very-high
* @id go/lines-of-comments-in-files
* @tags documentation
*/
import go
from File f
select f, f.getNumberOfLinesOfComments() as n order by n desc

Просмотреть файл

@ -0,0 +1,166 @@
/**
* Provides predicates for hashing AST nodes by structure.
*/
import go
/**
* The root of a sub-AST that should be hashed.
*/
abstract class HashRoot extends AstNode { }
/**
* An AST node that can be hashed.
*/
class HashableNode extends AstNode {
HashableNode() {
this instanceof HashRoot or
getParent() instanceof HashableNode
}
/**
* An opaque integer describing the type of this AST node.
*/
int getKind() {
exists(int baseKind |
// map expression kinds to even positive numbers
baseKind = this.(Expr).getKind() and
result = (baseKind + 1) * 2
or
// map statement kinds to odd positive numbers
baseKind = this.(Stmt).getKind() and
result = baseKind * 2 + 1
or
// map declaration kinds to even negative numbers
baseKind = this.(Decl).getKind() and
result = -(baseKind + 1) * 2
or
// map declaration specifier kinds to odd negative numbers
baseKind = this.(Spec).getKind() and
result = -(baseKind * 2 + 1)
or
// give files kind zero
this instanceof File and
baseKind = 0 and
result = 0
)
}
/**
* Gets the value of this AST node, or the empty string if it does not have one.
*/
string getValue() {
literals(this, result, _)
or
not literals(this, _, _) and
result = ""
}
/**
* Computes a hash for this AST node based on its structure.
*/
abstract HashedNode hash();
}
/**
* An AST node without any children.
*/
class HashableNullaryNode extends HashableNode {
HashableNullaryNode() { not exists(getAChild()) }
predicate unpack(int kind, string value) { kind = getKind() and value = getValue() }
override HashedNode hash() {
exists(int kind, string value | unpack(kind, value) | result = MkHashedNullaryNode(kind, value))
}
}
/**
* An AST node with exactly one child, which is at position zero.
*/
class HashableUnaryNode extends HashableNode {
HashableUnaryNode() { getNumChild() = 1 and exists(getChild(0)) }
predicate unpack(int kind, string value, HashedNode child) {
kind = getKind() and value = getValue() and child = getChild(0).(HashableNode).hash()
}
override HashedNode hash() {
exists(int kind, string value, HashedNode child | unpack(kind, value, child) |
result = MkHashedUnaryNode(kind, value, child)
)
}
}
/**
* An AST node with exactly two children, which are at positions zero and one.
*/
class HashableBinaryNode extends HashableNode {
HashableBinaryNode() { getNumChild() = 2 and exists(getChild(0)) and exists(getChild(1)) }
predicate unpack(int kind, string value, HashedNode left, HashedNode right) {
kind = getKind() and
value = getValue() and
left = getChild(0).(HashableNode).hash() and
right = getChild(1).(HashableNode).hash()
}
override HashedNode hash() {
exists(int kind, string value, HashedNode left, HashedNode right |
unpack(kind, value, left, right)
|
result = MkHashedBinaryNode(kind, value, left, right)
)
}
}
/**
* An AST node with more than two children, or with non-consecutive children.
*/
class HashableNAryNode extends HashableNode {
HashableNAryNode() {
exists(int n | n = strictcount(getAChild()) | n > 2 or not exists(getChild([0 .. n - 1])))
}
predicate unpack(int kind, string value, HashedChildren children) {
kind = getKind() and value = getValue() and children = hashChildren()
}
predicate childAt(int i, HashedNode child, HashedChildren rest) {
child = getChild(i).(HashableNode).hash() and rest = hashChildren(i + 1)
}
override HashedNode hash() {
exists(int kind, string value, HashedChildren children | unpack(kind, value, children) |
result = MkHashedNAryNode(kind, value, children)
)
}
HashedChildren hashChildren() { result = hashChildren(0) }
HashedChildren hashChildren(int i) {
i = max(int n | exists(getChild(n))) + 1 and result = Nil()
or
exists(HashedNode child, HashedChildren rest | childAt(i, child, rest) |
result = AHashedChild(i, child, rest)
)
}
}
newtype HashedNode =
MkHashedNullaryNode(int kind, string value) { any(HashableNullaryNode nd).unpack(kind, value) } or
MkHashedUnaryNode(int kind, string value, HashedNode child) {
any(HashableUnaryNode nd).unpack(kind, value, child)
} or
MkHashedBinaryNode(int kind, string value, HashedNode left, HashedNode right) {
any(HashableBinaryNode nd).unpack(kind, value, left, right)
} or
MkHashedNAryNode(int kind, string value, HashedChildren children) {
any(HashableNAryNode nd).unpack(kind, value, children)
}
newtype HashedChildren =
Nil() or
AHashedChild(int i, HashedNode child, HashedChildren rest) {
exists(HashableNAryNode nd | nd.childAt(i, child, rest))
}

Просмотреть файл

@ -0,0 +1,12 @@
package main
type Rectangle struct {
x, y, width, height float64
}
func (r *Rectangle) containsBad(x, y float64) bool {
return r.x <= x &&
y <= y &&
x <= r.x+r.width &&
y <= r.y+r.height
}

Просмотреть файл

@ -0,0 +1,37 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
Comparing two identical expressions typically indicates a mistake such as a missing qualifier or a
misspelled variable name.
</p>
</overview>
<recommendation>
<p>
Carefully inspect the comparison to determine whether it is a symptom of a bug.
</p>
</recommendation>
<example>
<p>
In the example below, the method <code>Rectangle.contains</code> is intended to check whether a
point <code>(x, y)</code> lies inside a rectangle <code>r</code> given by its origin
<code>(r.x, r.y)</code>, its width <code>r.width</code>, and its height <code>r.height</code>.
</p>
<sample src="CompareIdenticalValues.go" />
<p>
Note, however, that on line 9 the programmer forgot to qualify <code>r.y</code>,
thus ending up comparing the argument <code>y</code> against itself. The comparison
should be fixed accordingly:
</p>
<sample src="CompareIdenticalValuesGood.go" />
</example>
</qhelp>

Просмотреть файл

@ -0,0 +1,27 @@
/**
* @name Comparison of identical values
* @description If the same expression occurs on both sides of a comparison
* operator, the operator is redundant, and probably indicates a mistake.
* @kind problem
* @problem.severity warning
* @id go/comparison-of-identical-expressions
* @tags correctness
* external/cwe/cwe-570
* external/cwe/cwe-571
* @precision very-high
*/
import go
from Comparison cmp, Expr l
where
l = cmp.getLeftOperand() and
l.getGlobalValueNumber() = cmp.getRightOperand().getGlobalValueNumber() and
// whitelist floats, where self-comparison may be used for NaN checks
not l.getType().getUnderlyingType() instanceof FloatType and
// whitelist comparisons of symbolic constants to literal constants; these are often feature flags
not exists(DeclaredConstant decl |
cmp.getAnOperand() = decl.getAReference() and
cmp.getAnOperand() instanceof BasicLit
)
select cmp, "This expression compares $@ to itself.", cmp.getLeftOperand(), "an expression"

Просмотреть файл

@ -0,0 +1,8 @@
package main
func (r *Rectangle) containsGood(x, y float64) bool {
return r.x <= x &&
r.y <= y &&
x <= r.x+r.width &&
y <= r.y+r.height
}

Просмотреть файл

@ -0,0 +1,9 @@
package main
type counter struct {
val int
}
func (c counter) reset() {
c.val = 0
}

Просмотреть файл

@ -0,0 +1,42 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
A value is assigned to a field, but its value is never read. This means that the assignment
has no effect, and could indicate a logic error or incomplete code.
</p>
</overview>
<recommendation>
<p>
Examine the assignment closely to determine whether it is redundant, or whether it is perhaps
a symptom of another bug.
</p>
</recommendation>
<example>
<p>
The following example shows a simple <code>struct</code> type wrapping an integer counter with a
method <code>reset</code> that sets the counter to zero.
</p>
<sample src="DeadStoreOfField.go" />
<p>
However, the receiver variable of <code>reset</code> is declared to be of type
<code>counter</code>, not <code>*counter</code>, so the receiver value is passed into the method
by value, not by reference. Consequently, the method does not actually mutate its receiver as
intended.
</p>
<p>
To fix this, change the type of the receiver variable to <code>*counter</code>:
</p>
<sample src="DeadStoreOfFieldGood.go" />
</example>
<references>
<li>Go Frequently Asked Questions: <a href="https://golang.org/doc/faq#methods_on_values_or_pointers">Should I define methods on values or pointers?</a></li>
<li>The Go Programming Language Specification: <a href="https://golang.org/ref/spec#Method_declarations">Method declarations</a>.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,66 @@
/**
* @name Useless assignment to field
* @description An assignment to a field that is not used later on has no effect.
* @kind problem
* @problem.severity warning
* @id go/useless-assignment-to-field
* @tags maintainability
* external/cwe/cwe-563
* @precision very-high
*/
import go
/**
* Holds if `nd` escapes, that is, its value flows into the heap or across
* function boundaries.
*/
predicate escapes(DataFlow::Node nd) {
// if `nd` is written to something that is not an SSA variable (such as
// a global variable, a field or an array element), then it escapes
exists(Write w |
nd = w.getRhs() and
not w.definesSsaVariable(_, _)
)
or
// if `nd` is used as an index into an array or similar, then it escapes
exists(IndexExpr idx | nd.asExpr() = idx.getIndex())
or
// if `nd` is used in an (in-)equality comparison, then it escapes
exists(EqualityTestExpr eq | nd.asExpr() = eq.getAnOperand())
or
// if `nd` is returned from a function, then it escapes
nd instanceof DataFlow::ResultNode
or
// if `nd` is sent over a channel, then it escapes
exists(SendStmt s | nd.asExpr() = s.getValue())
or
// if `nd` is passed to a function, then it escapes
nd instanceof DataFlow::ArgumentNode
or
// if `nd` has its address taken, then it escapes
exists(AddressExpr ae | nd.asExpr() = ae.getOperand())
or
// if `nd` is used as to look up a method with a pointer receiver, then it escapes
exists(SelectorExpr sel | nd.asExpr() = sel.getBase() |
exists(Method m | sel = m.getAReference() | m.getReceiverType() instanceof PointerType)
or
// if we cannot resolve a reference, we make worst-case assumptions
not exists(sel.(Name).getTarget())
)
or
// if `nd` flows into something that escapes, then it escapes
escapes(nd.getASuccessor())
}
from Write w, LocalVariable v, Field f
where
// `w` writes `f` on `v`
w.writesField(v.getARead(), f, _) and
// but `f` is never read on `v`
not exists(Read r | r.readsField(v.getARead(), f)) and
// exclude pointer-typed `v`; there may be reads through an alias
not v.getType().getUnderlyingType() instanceof PointerType and
// exclude escaping `v`; there may be reads in other functions
not exists(Read r | r.reads(v) | escapes(r))
select w, "This assignment to " + f + " is useless since its value is never read."

Просмотреть файл

@ -0,0 +1,5 @@
package main
func (c *counter) resetGood() {
c.val = 0
}

Просмотреть файл

@ -0,0 +1,45 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
A value is assigned to a variable, but either it is never read, or its value is
always overwritten before being read. This means that the original assignment
has no effect, and could indicate a logic error or incomplete code.
</p>
</overview>
<recommendation>
<p>
Remove assignments to variables that are immediately overwritten, or use the
blank identifier <code>_</code> as a placeholder for return values that are
never used.
</p>
</recommendation>
<example>
<p>
In the following example, a value is assigned to <code>a</code>, but then
immediately overwritten, a value is assigned to <code>b</code> and never used,
and finally, the results of a call to <code>fmt.Println</code> are assigned to
two temporary variables, which are then immediately overwritten by a call to
<code>function</code>.
</p>
<sample src="DeadStoreOfLocalBad.go" />
<p>
The result of <code>calculateValue</code> is never used, and
if <code>calculateValue</code> is a side-effect free function, those assignments
can be removed. To ignore all the return values of <code>fmt.Println</code>, you
can simply not assign it to any variables. To ignore only certain return values,
use <code>_</code>.
</p>
<sample src="DeadStoreOfLocalGood.go" />
</example>
<references>
<li>Wikipedia: <a href="http://en.wikipedia.org/wiki/Dead_store">Dead store</a>.</li>
<li>The Go Programming Language Specification: <a href="https://golang.org/ref/spec#Blank_identifier">Blank identifier</a>.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,42 @@
/**
* @name Useless assignment to local variable
* @description An assignment to a local variable that is not used later on, or whose value is always
* overwritten, has no effect.
* @kind problem
* @problem.severity warning
* @id go/useless-assignment-to-local
* @tags maintainability
* external/cwe/cwe-563
* @precision very-high
*/
import go
/** Holds if `nd` is an initializer that we do not want to flag for this query. */
predicate isSimple(IR::Instruction nd) {
exists(Expr e |
e.isConst() or
e.(CompositeLit).getNumElement() = 0
|
nd = IR::evalExprInstruction(e)
)
or
nd = IR::implicitInitInstruction(_)
or
// don't flag parameters
nd instanceof IR::ReadArgumentInstruction
}
from IR::Instruction def, SsaSourceVariable target, IR::Instruction rhs
where
def.writes(target, rhs) and
not exists(SsaExplicitDefinition ssa | ssa.getInstruction() = def) and
// exclude assignments in dead code
def.getBasicBlock() instanceof ReachableBasicBlock and
// exclude assignments with default values or simple expressions
not isSimple(rhs) and
// exclude variables that are not used at all
exists(target.getAUse()) and
// exclude variables with indirect references
not target.mayHaveIndirectReferences()
select def, "This definition of " + target + " is never used."

Просмотреть файл

@ -0,0 +1,19 @@
package main
import "fmt"
func main() {
a := calculateValue()
a = 2
b := calculateValue()
ignore, ignore1 := fmt.Println(a)
ignore, ignore1, err := function()
if err != nil {
panic(err)
}
fmt.Println(a)
}

Просмотреть файл

@ -0,0 +1,16 @@
package main
import "fmt"
func main() {
a := 2
fmt.Println(a)
_, _, err := function()
if err != nil {
panic(err)
}
fmt.Println(a)
}

Просмотреть файл

@ -0,0 +1,9 @@
package main
func abs(x int) int {
if x >= 0 {
return x
} else {
return x
}
}

Просмотреть файл

@ -0,0 +1,36 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
If the 'then' and 'else' branches of an 'if' statement are identical, this suggests a copy-paste
error where the first branch was copied and then not properly adjusted.
</p>
</overview>
<recommendation>
<p>
Examine the two branches to find out what operations were meant to perform. If both the branches
and the conditions that they check are identical, then the second branch is duplicate code
that can be deleted. If the branches are really meant to perform the same operations, it may be clearer to just have a single branch that checks the disjunction of both conditions.
</p>
</recommendation>
<example>
<p>
The example below shows a buggy implementation of the absolute-value function which checks the sign
of its argument, but then returns the same value regardless of the outcome of the check:
</p>
<sample src="DuplicateBranches.go" />
<p>
Clearly, the 'else' branch should return <code>-x</code> instead:
</p>
<sample src="DuplicateBranchesGood.go" />
</example>
<references>
<li>The Go Programming Language Specification: <a href="https://golang.org/ref/spec#If_statements">If statements</a>.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,25 @@
/**
* @name Duplicate 'if' branches
* @description If the 'then' and 'else' branches of an 'if' statement are identical, the
* conditional may be superfluous, or it may indicate a mistake.
* @kind problem
* @problem.severity warning
* @precision very-high
* @id go/duplicate-branches
* @tags maintainability
* correctness
* external/cwe/cwe-561
*/
import Clones
class HashedBranch extends HashRoot, Stmt {
HashedBranch() { exists(IfStmt is | this = is.getThen() or this = is.getElse()) }
}
from IfStmt is, HashableNode thenBranch, HashableNode elseBranch
where
thenBranch = is.getThen() and
elseBranch = is.getElse() and
thenBranch.hash() = elseBranch.hash()
select is.getCond(), "The 'then' and 'else' branches of this if statement are identical."

Просмотреть файл

@ -0,0 +1,9 @@
package main
func absGood(x int) int {
if x >= 0 {
return x
} else {
return -x
}
}

Просмотреть файл

@ -0,0 +1,11 @@
package main
func controller(msg string) {
if msg == "start" {
start()
} else if msg == "start" {
stop()
} else {
panic("Message not understood.")
}
}

Просмотреть файл

@ -0,0 +1,38 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
If two conditions in an 'if'-'else if' chain are identical, the second condition will never
hold. This most likely indicates a copy-paste error where the first condition was copied
and then not properly adjusted.
</p>
</overview>
<recommendation>
<p>
Examine the two conditions to find out what they were meant to check. If both the conditions
and the branches that depend on them are identical, then the second branch is duplicate code
that can be deleted. Otherwise, the second condition needs to be adjusted.
</p>
</recommendation>
<example>
<p>
In the example below, the function <code>controller</code> checks its parameter <code>msg</code>
to determine what operation it is meant to perform. However, the comparison in the 'else if' is
identical to the comparison in the 'if', so this branch will never be taken.
</p>
<sample src="DuplicateCondition.go" />
<p>
Most likely, the 'else if' branch should compare <code>msg</code> to <code>"stop"</code>:
</p>
<sample src="DuplicateConditionGood.go" />
</example>
<references>
<li>The Go Programming Language Specification: <a href="https://golang.org/ref/spec#If_statements">If statements</a>.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,33 @@
/**
* @name Duplicate 'if' condition
* @description If two conditions in an 'if'-'else if' chain are identical, the
* second condition will never hold.
* @kind problem
* @problem.severity error
* @id go/duplicate-condition
* @tags maintainability
* correctness
* external/cwe/cwe-561
* @precision very-high
*/
import go
/** Gets the `i`th condition in the `if`-`else if` chain starting at `stmt`. */
Expr getCondition(IfStmt stmt, int i) {
i = 0 and result = stmt.getCond()
or
exists(IfStmt elsif | elsif = stmt.getElse() |
not exists(elsif.getInit()) and
result = getCondition(stmt.getElse(), i - 1)
)
}
/** Gets the global value number of `e`, which is the `i`th condition of `is`. */
GVN conditionGVN(IfStmt is, int i, Expr e) {
e = getCondition(is, i) and result = e.getGlobalValueNumber()
}
from IfStmt is, Expr e, Expr f, int i, int j
where conditionGVN(is, i, e) = conditionGVN(is, j, f) and i < j
select f, "This condition is a duplicate of $@.", e, "an earlier condition"

Просмотреть файл

@ -0,0 +1,11 @@
package main
func controllerGood(msg string) {
if msg == "start" {
start()
} else if msg == "stop" {
stop()
} else {
panic("Message not understood.")
}
}

Просмотреть файл

@ -0,0 +1,12 @@
package main
func controller(msg string) {
switch {
case msg == "start":
start()
case msg == "start":
stop()
default:
panic("Message not understood.")
}
}

Просмотреть файл

@ -0,0 +1,38 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
If two cases in a 'switch' statement are identical, the second case will never be executed.
This most likely indicates a copy-paste error where the first case was copied and then not properly
adjusted.
</p>
</overview>
<recommendation>
<p>
Examine the two cases to find out what they were meant to check. If both the conditions
and the bodies are identical, then the second case is duplicate code that can be deleted. Otherwise,
the second case needs to be adjusted.
</p>
</recommendation>
<example>
<p>
In the example below, the function <code>controller</code> checks its parameter <code>msg</code>
to determine what operation it is meant to perform. However, the condition of the second case is
identical to that of the first, so this case will never be executed.
</p>
<sample src="DuplicateSwitchCase.go" />
<p>
Most likely, the second case should compare <code>msg</code> to <code>"stop"</code>:
</p>
<sample src="DuplicateSwitchCaseGood.go" />
</example>
<references>
<li>The Go Programming Language Specification: <a href="https://golang.org/ref/spec#Switch_statements">Switch statements</a>.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,23 @@
/**
* @name Duplicate switch case
* @description If two cases in a switch statement have the same label, the second case
* will never be executed.
* @kind problem
* @problem.severity error
* @id go/duplicate-switch-case
* @tags maintainability
* correctness
* external/cwe/cwe-561
* @precision very-high
*/
import go
/** Gets the global value number of of `e`, which is the `i`th case label of `switch`. */
GVN switchCaseGVN(SwitchStmt switch, int i, Expr e) {
e = switch.getCase(i).getExpr(0) and result = e.getGlobalValueNumber()
}
from SwitchStmt switch, int i, Expr e, int j, Expr f
where switchCaseGVN(switch, i, e) = switchCaseGVN(switch, j, f) and i < j
select f, "This case is a duplicate of $@.", e, "an earlier case"

Просмотреть файл

@ -0,0 +1,12 @@
package main
func controllerGood(msg string) {
switch {
case msg == "start":
start()
case msg == "stop":
stop()
default:
panic("Message not understood.")
}
}

Просмотреть файл

@ -0,0 +1,15 @@
package main
import "fmt"
type Timestamp int
func (t Timestamp) addDays(d int) Timestamp {
return Timestamp(int(t) + d*24*3600)
}
func test(t Timestamp) {
fmt.Printf("Before: %s\n", t)
t.addDays(7)
fmt.Printf("After: %s\n", t)
}

Просмотреть файл

@ -0,0 +1,37 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
An expression that has no effects (such as changing variable values or producing output) and
occurs in a context where its value is ignored possibly indicates missing code or a latent bug.
</p>
</overview>
<recommendation>
<p>
Carefully inspect the expression to ensure it is not a symptom of a bug.
</p>
</recommendation>
<example>
<p>
The following example shows a named type <code>Timestamp</code> that is an alias for
<code>int</code>, representing time stamps expressed as the number of seconds elapsed since some
epoch. The <code>addDays</code> method returns a time stamp that is a given number of days after
another time stamp, without modifying that time stamp.
</p>
<p>
However, when <code>addDays</code> is used in function <code>test</code>, its result is discarded,
perhaps because the programmer mistakenly assumed that <code>addDays</code> updates the time stamp
in place.
</p>
<sample src="ExprHasNoEffect.go"/>
<p>
Instead, the result of <code>addDays</code> should be assigned back into <code>t</code>:
</p>
<sample src="ExprHasNoEffectGood.go" />
</example>
</qhelp>

Просмотреть файл

@ -0,0 +1,39 @@
/**
* @name Expression has no effect
* @description An expression that has no effect and is used in a void context is most
* likely redundant and may indicate a bug.
* @kind problem
* @problem.severity warning
* @id go/useless-expression
* @tags maintainability
* correctness
* external/cwe/cwe-480
* external/cwe/cwe-561
* @precision very-high
*/
import go
/**
* Holds if `e` appears in a syntactic context where its value is discarded.
*/
predicate inVoidContext(Expr e) {
e = any(ExprStmt es).getExpr()
or
exists(LogicalBinaryExpr logical | e = logical.getRightOperand() and inVoidContext(logical))
}
/**
* Holds if `ce` is a call to a stub function with an empty body.
*/
predicate callToStubFunction(CallExpr ce) {
ce.getTarget().getBody().getNumStmt() = 0
}
from Expr e
where
not e.mayHaveOwnSideEffects() and
inVoidContext(e) and
// don't flag calls to functions with an empty body
not callToStubFunction(e)
select e, "This expression has no effect."

Просмотреть файл

@ -0,0 +1,9 @@
package main
import "fmt"
func testGood(t Timestamp) {
fmt.Printf("Before: %s\n", t)
t = t.addDays(7)
fmt.Printf("After: %s\n", t)
}

Просмотреть файл

@ -0,0 +1,8 @@
package main
func getFirst(xs []int) int {
if len(xs) < 0 {
panic("No elements provided")
}
return xs[0]
}

Просмотреть файл

@ -0,0 +1,45 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
The built-in <code>len</code> function returns the length of an array, slice or similar, which is
never less than zero. Hence, checking whether the result of a call to <code>len</code> is negative
is either redundant or indicates a logic mistake.
</p>
<p>
The same applies to the built-in function <code>cap</code>.
</p>
</overview>
<recommendation>
<p>
Examine the length check to see whether it is redundant and can be removed, or a mistake that
should be fixed.
</p>
</recommendation>
<example>
<p>
The example below shows a function that returns the first element of an array, triggering a panic
if the array is empty:
</p>
<sample src="NegativeLengthCheck.go" />
<p>
However, the emptiness check is ineffective: since <code>len(xs)</code> is never less than zero,
the condition will never hold and no panic will be triggered. Instead, the index expression
<code>xs[0]</code> will cause a panic.
</p>
<p>
The check should be rewritten like this:
</p>
<sample src="NegativeLengthCheckGood.go" />
</example>
<references>
<li>Package builtin: <a href="https://golang.org/pkg/builtin/#cap">func cap</a>.</li>
<li>Package builtin: <a href="https://golang.org/pkg/builtin/#len">func len</a>.</li>
</references>
</qhelp>

Просмотреть файл

@ -0,0 +1,36 @@
/**
* @name Check for negative length
* @description Checking whether the result of 'len' or 'cap' is negative is pointless,
* since these functions always returns a non-negative number.
* @kind problem
* @problem.severity warning
* @precision very-high
* @id go/negative-length-check
* @tags correctness
*/
import go
from Comparison cmp, BuiltinFunction len, int ub, string r
where
(len = Builtin::len() or len = Builtin::cap()) and
(
exists(RelationalComparisonExpr rel | rel = cmp |
rel.getLesserOperand() = len.getACallExpr() and
rel.getGreaterOperand().getIntValue() = ub and
(
ub < 0
or
ub = 0 and rel.isStrict()
) and
r = "be less than"
)
or
exists(EqualityTestExpr eq | eq = cmp |
eq.getAnOperand() = len.getACallExpr() and
eq.getAnOperand().getIntValue() = ub and
ub < 0 and
r = "equal"
)
)
select cmp, "'" + len.getName() + "' is always non-negative, and hence cannot " + r + " " + ub + "."

Просмотреть файл

@ -0,0 +1,8 @@
package main
func getFirstGood(xs []int) int {
if len(xs) == 0 {
panic("No elements provided")
}
return xs[0]
}

Просмотреть файл

@ -0,0 +1,5 @@
package main
func avg(x, y float64) float64 {
return (x + x) / 2
}

Некоторые файлы не были показаны из-за слишком большого количества измененных файлов Показать больше