hub/script/coverage

76 строки
1.9 KiB
Plaintext
Исходник Обычный вид История

Measure code coverage between tests Go has code coverage tooling for test mode, which temporarily rewrites the source code to insert annotations which will activate during the test run and track progress of executed code. Then, upon process completion, that information is dumped into a coverage report. We can't use this approach for hub, at least not without substantial changes. First of all, hub's test coverage is mostly "from the outside", utilizing Cucumber to invoke the binary with different arguments and inspect the outputs and result. There are some tests in go, but they are minimal compared to the cukes. Second, hub frequently aborts the process on errors via `os.Exit(1)`, and those scenarios need to be tested too. However, if the process exits prematurely, the code coverage report will never be generated. To work around this, I first used the go tool that annotates the source: go tool cover -mode=set -var=LiveCoverage myfile.go This injects `LiveCoverage.Count[pos] = 1` lines at appropriate places all over the source code, and generates a mapping of line/column positions in the original source. Then I rewrite those lines to become a method invocation: coverage.Record(LiveCoverage, pos) The new `Record` method will immediately append the information to a code coverage report file as soon as it's invoked. This ensures that there is coverage information even if the process gets aborted. This approach works the same for go tests as well as for cukes. They all append to the same file. Finally, the rest of Go tooling is used to generate an HTML report of code coverage: go tool cover -html=cover.out
2016-09-12 22:18:52 +03:00
#!/bin/bash
set -e
source_files() {
script/build files | grep -vE '^\./(coverage|fixtures)/'
}
prepare() {
local changed_files="$(source_files | xargs git diff --name-only --)"
if [ -n "$changed_files" ]; then
echo "Aborted: please commit the following files before continuing" >&2
cat <<<"$changed_files" >&2
Measure code coverage between tests Go has code coverage tooling for test mode, which temporarily rewrites the source code to insert annotations which will activate during the test run and track progress of executed code. Then, upon process completion, that information is dumped into a coverage report. We can't use this approach for hub, at least not without substantial changes. First of all, hub's test coverage is mostly "from the outside", utilizing Cucumber to invoke the binary with different arguments and inspect the outputs and result. There are some tests in go, but they are minimal compared to the cukes. Second, hub frequently aborts the process on errors via `os.Exit(1)`, and those scenarios need to be tested too. However, if the process exits prematurely, the code coverage report will never be generated. To work around this, I first used the go tool that annotates the source: go tool cover -mode=set -var=LiveCoverage myfile.go This injects `LiveCoverage.Count[pos] = 1` lines at appropriate places all over the source code, and generates a mapping of line/column positions in the original source. Then I rewrite those lines to become a method invocation: coverage.Record(LiveCoverage, pos) The new `Record` method will immediately append the information to a code coverage report file as soon as it's invoked. This ensures that there is coverage information even if the process gets aborted. This approach works the same for go tests as well as for cukes. They all append to the same file. Finally, the rest of Go tooling is used to generate an HTML report of code coverage: go tool cover -html=cover.out
2016-09-12 22:18:52 +03:00
exit 1
fi
local n=0
for f in $(source_files); do
go tool cover -mode=set -var="LiveCoverage$((++n))" "$f" > "$f"~
sed -E '
/^package /a\
import "github.com/github/hub/coverage"
s/(LiveCoverage[0-9]+)\.Count\[([0-9]+)\][^;]+/coverage.Record(\1, \2)/g
Measure code coverage between tests Go has code coverage tooling for test mode, which temporarily rewrites the source code to insert annotations which will activate during the test run and track progress of executed code. Then, upon process completion, that information is dumped into a coverage report. We can't use this approach for hub, at least not without substantial changes. First of all, hub's test coverage is mostly "from the outside", utilizing Cucumber to invoke the binary with different arguments and inspect the outputs and result. There are some tests in go, but they are minimal compared to the cukes. Second, hub frequently aborts the process on errors via `os.Exit(1)`, and those scenarios need to be tested too. However, if the process exits prematurely, the code coverage report will never be generated. To work around this, I first used the go tool that annotates the source: go tool cover -mode=set -var=LiveCoverage myfile.go This injects `LiveCoverage.Count[pos] = 1` lines at appropriate places all over the source code, and generates a mapping of line/column positions in the original source. Then I rewrite those lines to become a method invocation: coverage.Record(LiveCoverage, pos) The new `Record` method will immediately append the information to a code coverage report file as soon as it's invoked. This ensures that there is coverage information even if the process gets aborted. This approach works the same for go tests as well as for cukes. They all append to the same file. Finally, the rest of Go tooling is used to generate an HTML report of code coverage: go tool cover -html=cover.out
2016-09-12 22:18:52 +03:00
' < "$f"~ > "$f"
rm "$f"~
done
rm -rf "$HUB_COVERAGE"
mkdir -p "${HUB_COVERAGE%/*}"
}
generate() {
source_files | xargs git checkout --
echo 'mode: count' > "$HUB_COVERAGE"~
sed -E 's!^.+/(github.com/github/hub/)!\1!' "$HUB_COVERAGE" | awk '
{ a[substr($0, 0, length()-2)] += $(NF) }
END { for (k in a) print k, a[k] }
' >> "$HUB_COVERAGE"~
go tool cover -func="$HUB_COVERAGE"~ > "${HUB_COVERAGE%.out}.func"
if [ -z "$CI" ]; then
go tool cover -html="$HUB_COVERAGE"~ -o "${HUB_COVERAGE%.out}.html"
fi
awk '/^total:/ { print $(NF) }' "${HUB_COVERAGE%.out}.func"
}
summarize() {
local total_coverage
local min_coverage="${1?}"
total_coverage="$(generate)"
echo "Code coverage: $total_coverage"
local result="$(bc <<<"${total_coverage%\%} < $min_coverage")"
if [ "$result" -eq 1 ]; then
echo "Error: coverage dropped below the minimum treshold of ${min_coverage}%!"
if [ -n "$CI" ]; then
html_result="${HUB_COVERAGE%.out}.html"
html_result="${html_result#$PWD/}"
printf 'Please run `script/test --coverage` locally and open `%s` to analyze the results.\n' "$html_result"
fi
return 1
fi
}
cmd="${1?}"
shift 1
case "$cmd" in
prepare | generate | summarize )
"$cmd" "$@"
Measure code coverage between tests Go has code coverage tooling for test mode, which temporarily rewrites the source code to insert annotations which will activate during the test run and track progress of executed code. Then, upon process completion, that information is dumped into a coverage report. We can't use this approach for hub, at least not without substantial changes. First of all, hub's test coverage is mostly "from the outside", utilizing Cucumber to invoke the binary with different arguments and inspect the outputs and result. There are some tests in go, but they are minimal compared to the cukes. Second, hub frequently aborts the process on errors via `os.Exit(1)`, and those scenarios need to be tested too. However, if the process exits prematurely, the code coverage report will never be generated. To work around this, I first used the go tool that annotates the source: go tool cover -mode=set -var=LiveCoverage myfile.go This injects `LiveCoverage.Count[pos] = 1` lines at appropriate places all over the source code, and generates a mapping of line/column positions in the original source. Then I rewrite those lines to become a method invocation: coverage.Record(LiveCoverage, pos) The new `Record` method will immediately append the information to a code coverage report file as soon as it's invoked. This ensures that there is coverage information even if the process gets aborted. This approach works the same for go tests as well as for cukes. They all append to the same file. Finally, the rest of Go tooling is used to generate an HTML report of code coverage: go tool cover -html=cover.out
2016-09-12 22:18:52 +03:00
;;
* )
exit 1
;;
esac