xamarin-macios/jenkins/run-tests.sh

110 строки
2.7 KiB
Bash
Исходник Обычный вид История

#!/bin/bash -e
cd "$(dirname "${BASH_SOURCE[0]}")/.."
WORKSPACE=$(pwd)
# Print out environment for debug purposes
env | sort
report_error ()
{
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
printf "🔥 [Test run failed](%s) 🔥\\n" "$URL" >> "$WORKSPACE/jenkins/pr-comments.md"
if test -f "$WORKSPACE/tests/TestSummary.md"; then
printf "\\n" >> "$WORKSPACE/jenkins/pr-comments.md"
cat "$WORKSPACE/tests/TestSummary.md" >> "$WORKSPACE/jenkins/pr-comments.md"
fi
touch "$WORKSPACE/jenkins/failure-stamp"
}
trap report_error ERR
# SC2154: ghprbPullId is referenced but not assigned.
# shellcheck disable=SC2154
if test -n "$ghprbPullId" && ./jenkins/fetch-pr-labels.sh --check=skip-public-jenkins; then
echo "Skipping tests because the label 'skip-public-jenkins' was found."
exit 0
fi
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
TARGET=jenkins
PUBLISH=
KEYCHAIN=builder
KEYCHAIN_PWD_FILE=~/.config/keychain
while ! test -z "$1"; do
case "$1" in
--target=*)
TARGET="${1:9}"
shift
;;
--keychain=*)
KEYCHAIN="${1:11}"
KEYCHAIN_PWD_FILE=~/.config/$KEYCHAIN-keychain
shift
;;
--publish)
PUBLISH=1
shift
;;
*)
echo "Unknown argument: $1"
exit 1
;;
esac
done
if test -n "$PUBLISH"; then
PUBLISH_OUTPUT=$(./jenkins/publish-results.sh)
URL_PREFIX=$(echo "$PUBLISH_OUTPUT" | grep "^Url Prefix: " | sed 's/^Url Prefix: //')
URL="$URL_PREFIX/tests/index.html"
TESTS_PERIODIC_COMMAND=$(echo "$PUBLISH_OUTPUT" | grep "^Periodic Command: " | sed 's/^Periodic Command: //')
export TESTS_PERIODIC_COMMAND
else
URL="$BUILD_URL/Test_20Report"
fi
export BUILD_REVISION=jenkins
# Unlock
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
if ! test -f ~/Library/Keychains/"$KEYCHAIN".keychain-db; then
echo "The '$KEYCHAIN' keychain is not available."
exit 1
fi
security default-keychain -s "$KEYCHAIN.keychain"
security list-keychains -s "$KEYCHAIN.keychain"
echo "Unlock keychain"
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
security unlock-keychain -p "$(cat "$KEYCHAIN_PWD_FILE")"
echo "Increase keychain unlock timeout to 6 hours"
security set-keychain-settings -lut 21600
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
security -v find-identity "$KEYCHAIN.keychain"
# Prevent dialogs from asking for permissions.
# http://stackoverflow.com/a/40039594/183422
# Discard output since there can be a *lot* of it.
security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k "$(cat "$KEYCHAIN_PWD_FILE")" "$KEYCHAIN.keychain" >/dev/null 2>&1
# clean mono keypairs (used in tests)
rm -rf ~/.config/.mono/keypairs/
# Run tests
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
RC=0
make -C tests "$TARGET" || RC=$?
# upload of the final html report
if test -n "$PUBLISH"; then
./jenkins/publish-results.sh
fi
if [[ x$RC != x0 ]]; then
report_error
exit $RC
fi
Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`: ![screenshot 2018-06-01 11 12 57](https://user-images.githubusercontent.com/249268/40832932-c3b05eb0-658c-11e8-9670-8de5fcc23407.png) * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`) ![screenshot 2018-06-01 11 14 33](https://user-images.githubusercontent.com/249268/40833014-fd8772f4-658c-11e8-8a35-5df46bfb16c7.png) Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
2018-06-05 02:40:16 +03:00
printf "✅ [Test run succeeded](%s)\\n" "$URL" >> "$WORKSPACE/jenkins/pr-comments.md"
if test -f "$WORKSPACE/jenkins/failure-stamp"; then
echo "Something went wrong:"
cat "$WORKSPACE/jenkins/pr-comments.md"
exit 1
fi