xamarin-macios/jenkins/compare.sh

92 строки
4.2 KiB
Bash
Исходник Обычный вид История

#!/bin/bash -e
#
# How to run API comparison locally:
#
# 1. To compare a specific commit to the previous one, just run this script (./compare.sh).
# 2. Replicating what happens for a specific PR is a bit more complicated:
# a. Fetch the source code for that PR, using "git fetch --no-tags --progress https://github.com/xamarin/xamarin-macios +refs/pull/<PR>/*:refs/remotes/origin/pr/<PR>/*",
# so for example this for PR #4614:
#
# git fetch --no-tags --progress https://github.com/xamarin/xamarin-macios +refs/pull/4614/*:refs/remotes/origin/pr/4614/*
#
# b. Checkout the commit you want to compare (typically the branch you created the PR from if it's your own PR).
# i) This isn't 100% accurate, since GitHub will build a merged version
# of your branch + the target branch. If you want to test _exactly_
# the same code as GitHub, checkout the 'origin/pr/<merge>/merge'
# branch the previous git fetch command created.
#
# c. Execute 'ghprbPullId=4614 ./compare.sh'
#
cd "$(dirname "${BASH_SOURCE[0]}")/.."
WORKSPACE=$(pwd)
report_error ()
{
printf "🔥 [Failed to compare API and create generator diff](%s/console) 🔥\\n" "$BUILD_URL" >> "$WORKSPACE/jenkins/pr-comments.md"
touch "$WORKSPACE/jenkins/failure-stamp"
exit 0
}
trap report_error ERR
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
# SC2154: ghprbPullId is referenced but not assigned.
# shellcheck disable=SC2154
if test -n "$ghprbPullId"; then
if ./jenkins/fetch-pr-labels.sh --check=skip-api-comparison; then
printf "❎ Skipped API comparison because the PR has the label 'skip-api-comparison'\\n" >> "$WORKSPACE/jenkins/pr-comments.md"
exit 0
fi
if ./jenkins/fetch-pr-labels.sh --check=skip-public-jenkins; then
echo "Skipping API comparison because the label 'skip-public-jenkins' was found."
exit 0
fi
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
fi
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
if test -z "$ghprbPullId"; then
BASE=HEAD
else
BASE="origin/pr/$ghprbPullId/merge"
fi
if ! git rev-parse "$BASE" >/dev/null 2>&1; then
echo "Can't compare API and create generator diff because the pull request has conflicts that must be resolved first (the branch '$BASE' doesn't exist)."
printf "🔥 [Failed to compare API and create generator diff because the pull request has conflicts that must be resolved first](%s/console) 🔥\\n" "$BUILD_URL" >> "$WORKSPACE/jenkins/pr-comments.md"
exit 0
fi
./tools/compare-commits.sh --base="$BASE^1"
mkdir -p jenkins-results/apicomparison
cp -R tools/comparison/apidiff/diff jenkins-results/apicomparison/
cp tools/comparison/apidiff/*.html jenkins-results/apicomparison/
cp -R tools/comparison/generator-diff jenkins-results/generator-diff
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
if [[ "x$1" == "x--publish" ]]; then
URL_PREFIX=$(./jenkins/publish-results.sh | grep "^Url Prefix: " | sed 's/^Url Prefix: //')
URL_API="$URL_PREFIX/apicomparison/api-diff.html"
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
URL_GENERATOR="$URL_PREFIX/generator-diff/index.html"
else
URL_API="$BUILD_URL/API_20diff_20_28PR_20only_29"
URL_GENERATOR="$BUILD_URL/Generator_20Diff"
fi
if ! grep "href=" jenkins-results/apicomparison/api-diff.html >/dev/null 2>&1; then
printf "✅ [API Diff (from PR only)](%s) (no change)" "$URL_API" >> "$WORKSPACE/jenkins/pr-comments.md"
elif perl -0777 -pe 's/<script type="text\/javascript">.*?<.script>/script removed/gs' jenkins-results/apicomparison/*.html | grep data-is-breaking; then
printf "⚠️ [API Diff (from PR only)](%s) (🔥 breaking changes 🔥)" "$URL_API" >> "$WORKSPACE/jenkins/pr-comments.md"
else
printf " [API Diff (from PR only)](%s) (please review changes)" "$URL_API" >> "$WORKSPACE/jenkins/pr-comments.md"
fi
printf "\\n" >> "$WORKSPACE/jenkins/pr-comments.md"
if ! test -s jenkins-results/generator-diff/generator.diff; then
printf "✅ [Generator Diff](%s) (no change)" "$URL_GENERATOR" >> "$WORKSPACE/jenkins/pr-comments.md"
elif grep "^[+-][^+-]" jenkins-results/generator-diff/generator.diff | grep -v "^.[[]assembly: AssemblyInformationalVersion" | grep -v "^[+-][[:space:]]*internal const string Revision =" >/dev/null 2>&1; then
printf " [Generator Diff](%s) (please review changes)" "$URL_GENERATOR" >> "$WORKSPACE/jenkins/pr-comments.md"
else
printf "✅ [Generator Diff](%s) (only version changes)" "$URL_GENERATOR" >> "$WORKSPACE/jenkins/pr-comments.md"
fi
printf "\\n" >> "$WORKSPACE/jenkins/pr-comments.md"