Ever since we started building and testing Git on Travis CI (522354d70
(Add Travis CI support, 2015-11-27)), we build Git in the
'before_script' phase and run the test suite in the 'script' phase
(except in the later introduced 32 bit Linux and Windows build jobs,
where we build in the 'script' phase').
Contrarily, the Travis CI practice is to build and test in the
'script' phase; indeed Travis CI's default build command for the
'script' phase of C/C++ projects is:
./configure && make && make test
The reason why Travis CI does it this way and why it's a better
approach than ours lies in how unsuccessful build jobs are
categorized. After something went wrong in a build job, its state can
be:
- 'failed', if a command in the 'script' phase returned an error.
This is indicated by a red 'X' on the Travis CI web interface.
- 'errored', if a command in the 'before_install', 'install', or
'before_script' phase returned an error, or the build job exceeded
the time limit. This is shown as a red '!' on the web interface.
This makes it easier, both for humans looking at the Travis CI web
interface and for automated tools querying the Travis CI API, to
decide when an unsuccessful build is our responsibility requiring
human attention, i.e. when a build job 'failed' because of a compiler
error or a test failure, and when it's caused by something beyond our
control and might be fixed by restarting the build job, e.g. when a
build job 'errored' because a dependency couldn't be installed due to
a temporary network error or because the OSX build job exceeded its
time limit.
The drawback of building Git in the 'before_script' phase is that one
has to check the trace log of all 'errored' build jobs, too, to see
what caused the error, as it might have been caused by a compiler
error. This requires additional clicks and page loads on the web
interface and additional complexity and API requests in automated
tools.
Therefore, move building Git from the 'before_script' phase to the
'script' phase, updating the script's name accordingly as well.
'ci/run-builds.sh' now becomes basically empty, remove it. Several of
our build job configurations override our default 'before_script' to
do nothing; with this change our default 'before_script' won't do
anything, either, so remove those overriding directives as well.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Every once in a while our explicit .gitignore files get out of sync
when our build process learns to create new artifacts, like test
helper executables, but the .gitignore files are not updated
accordingly.
Use Travis CI to help catch such issues earlier: check that there are
no untracked files at the end of any build jobs building Git (i.e. the
64 bit Clang and GCC Linux and OSX build jobs, plus the GETTEXT_POISON
and 32 bit Linux build jobs) or its documentation, and fail the build
job if there are any present.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The Clang and GCC 64 bit Linux build jobs download and store the P4
and Git LFS executables under the current directory, which is the
working tree that we are about to build and test. This means that Git
commands like 'status' or 'ls-files' would list these files as
untracked. The next commit is about to make sure that there are no
untracked files present after the build, and the downloaded
executables in the working tree are interfering with those upcoming
checks.
Therefore, let's download P4 and Git LFS in the home directory,
outside of the working tree.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Travis CI dutifully builds and tests each new branch tip, even if its
tree has previously been successfully built and tested. This happens
often enough in contributors' workflows, when a work-in-progress
branch is rebased changing e.g. only commit messages or the order or
number of commits while leaving the resulting code intact, and is then
pushed to a Travis CI-enabled GitHub fork.
This is wasting Travis CI's resources and is sometimes scary-annoying
when the new tip commit with a tree identical to the previous,
successfully tested one is suddenly reported in red, because one of
the OSX build jobs happened to exceed the time limit yet again.
So extend our Travis CI build scripts to skip building commits whose
trees have previously been successfully built and tested. Use the
Travis CI cache feature to keep a record of the object names of trees
that tested successfully, in a plain and simple flat text file, one
line per tree object name. Append the current tree's object name at
the end of every successful build job to this file, along with a bit
of additional info about the build job (commit object name, Travis CI
job number and id). Limit the size of this file to 1000 records, to
prevent it from growing too large for git/git's forever living
integration branches. Check, using a simple grep invocation, in each
build job whether the current commit's tree is already in there, and
skip the build if it is. Include a message in the skipped build job's
trace log, containing the URL to the build job successfully testing
that tree for the first time and instructions on how to force a
re-build. Catch the case when a build job, which successfully built
and tested a particular tree for the first time, is restarted and omit
the URL of the previous build job's trace log, as in this case it's
the same build job and the trace log has just been overwritten.
Note: this won't kick in if two identical trees are on two different
branches, because Travis CI caches are not shared between build jobs
of different branches.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It seems that Travis CI creates the cache directory for us anyway,
even when a previous cache doesn't exist for the current build job.
Alas, this behavior is not explicitly documented, therefore we don't
rely on it and create the cache directory ourselves in those build
jobs that read/write cached data (currently only the prove state).
In the following commit we'll start to cache additional data in every
build job, and will access the cache much earlier in the build
process.
Therefore move creating the cache directory to 'ci/lib-travisci.sh' to
make sure that it exists at the very beginning of every build job.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To make this info message stand out from the regular build job trace
output.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a build job running the test suite fails, our
'ci/print-test-failures.sh' script scans all 't/test-results/*.exit'
files to find failed tests and prints their verbose output. However,
if a build job were to fail before it ever gets to run the test suite,
then there will be no files to match the above pattern and the shell
will take the pattern literally, resulting in errors like this in the
trace log:
cat: t/test-results/*.exit: No such file or directory
------------------------------------------------------------------------
t/test-results/*.out...
------------------------------------------------------------------------
cat: t/test-results/*.out: No such file or directory
Check upfront and proceed only if there are any such files present.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This change follows suit of 6272ed319 (travis-ci: run previously
failed tests first, then slowest to fastest, 2016-01-26), which did
this for the Linux and OSX build jobs. Travis CI build jobs run the
tests parallel, which is sligtly faster when tests are run in slowest
to fastest order, shortening the overall runtime of this build job by
about a minute / 10%.
Note, that the 32 bit Linux build job runs the tests suite in a Docker
container and we have to share the Travis CI cache directory with the
container as a second volume. Otherwise we couldn't use a symlink
pointing to the prove state file in the cache directory, because
that's outside of the directory hierarchy accessible from within the
container.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The change in commit 4f2636667 (travis-ci: use 'set -x' in 'ci/*'
scripts for extra tracing output, 2017-12-12) left a couple of rough
edges:
- 'ci/run-linux32-build.sh' is executed in a Docker container and
therefore doesn't source 'ci/lib-travisci.sh', which would enable
tracing executed commands. Enable 'set -x' in this script, too.
- 'ci/print-test-failures.sh' iterates over all the files containing
the exit codes of all the executed test scripts. Since there are
over 800 such files, the loop produces way too much noise with
tracing executed commands enabled, so disable 'set -x' for this
script.
- 'ci/run-windows-build.sh' busily waits in a loop for the result of
the Windows build, producing too much noise with tracing executed
commands enabled as well. Disable 'set -x' for the duration of
that loop.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While the build logic was embedded in our '.travis.yml', Travis CI
used to produce a nice trace log including all commands executed in
those embedded scriptlets. Since 657343a60 (travis-ci: move Travis CI
code into dedicated scripts, 2017-09-10), however, we only see the
name of the dedicated scripts, but not what those scripts are actually
doing, resulting in a less useful trace log. A patch later in this
series will move setting environment variables from '.travis.yml' to
the 'ci/*' scripts, so not even those will be included in the trace
log.
Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other
'ci/*' scripts, so we get trace log about the commands executed in all
of those scripts.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 657343a60 (travis-ci: move Travis CI code into dedicated
scripts, 2017-09-10) converted '.travis.yml's default 'before_install'
scriptlet to the 'ci/install-dependencies.sh' script, and while doing
so moved setting GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang
Linux build jobs to that script. This is wrong for two reasons:
- The purpose of that script is, as its name suggests, to install
dependencies, not to set any environment variables influencing
which tests should be run (though, arguably, this was already an
issue with the original 'before_install' scriptlet).
- Setting the variable has no effect anymore, because that script is
run in a separate shell process, and the variable won't be visible
in any of the other scripts, notably in 'ci/run-tests.sh'
responsible for, well, running the tests.
Luckily, this didn't have a negative effect on our Travis CI build
jobs, because GIT_TEST_HTTPD is a tri-state variable defaulting to
"auto" and a functioning web server was installed in those Linux build
jobs, so the httpd tests were run anyway.
Apparently the httpd tests run just fine without GIT_TEST_HTTPD being
set, therefore we could simply remove this environment variable.
However, if a bug were to creep in to change the Travis CI build
environment to run the tests as root or to not install Apache, then
the httpd tests would be skipped and the build job would still
succeed. We would only notice if someone actually were to look
through the build job's trace log; but who would look at the trace log
of a successful build job?!
Since httpd tests are important, we do want to run them and we want to
be loudly reminded if they can't be run. Therefore, move setting
GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang Linux build jobs
to 'ci/lib-travisci.sh' to ensure that the build job fails when the
httpd tests can't be run. (We could set it in 'ci/run-tests.sh' just
as well, but it's better to keep all environment variables in one
place in 'ci/lib-travisci.sh'.)
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Our '.travis.yml's 'env.global' section sets a bunch of environment
variables for all build jobs, though none of them actually affects all
build jobs. It's convenient for us, and in most cases it works just
fine, because irrelevant environment variables are simply ignored.
However, $GIT_SKIP_TESTS is an exception: it tells the test harness to
skip the two test scripts that are prone to occasional failures on
OSX, but as it's set for all build jobs those tests are not run in any
of the build jobs that are capable to run them reliably, either.
Therefore $GIT_SKIP_TESTS should only be set in the OSX build jobs,
but those build jobs are included in the build matrix implicitly (i.e.
by combining the matrix keys 'os' and 'compiler'), and there is no way
to set an environment variable only for a subset of those implicit
build jobs. (Unless we were to add new scriptlets to '.travis.yml',
which is exactly the opposite direction that we took with commit
657343a60 (travis-ci: move Travis CI code into dedicated scripts,
2017-09-10)).
So move setting $GIT_SKIP_TESTS to 'ci/lib-travisci.sh', where it can
trivially be set only for the OSX build jobs.
Furthermore, move setting all other environment variables from
'.travis.yml' to 'ci/lib-travisci.sh', too, because a couple of
environment variables are already set there, and this way all
environment variables will be set in the same place. All the logic
controlling our builds is already in the 'ci/*' scripts anyway, so
there is really no good reason to keep the environment variables
separately.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A couple of 'ci/*' scripts are shared between different build jobs:
'ci/lib-travisci.sh', being a common library, is sourced from almost
every script, while 'ci/install-dependencies.sh', 'ci/run-build.sh'
and 'ci/run-tests.sh' are shared between the "regular" GCC and Clang
Linux and OSX build jobs, and the latter two scripts are used in the
GETTEXT_POISON Linux build job as well.
Our builds could benefit from these shared scripts being able to
easily tell which build job they are taking part in. Now, it's
already quite easy to tell apart Linux vs OSX and GCC vs Clang build
jobs, but it gets trickier with all the additional Linux-based build
jobs included explicitly in the build matrix.
Unfortunately, Travis CI doesn't provide much help in this regard.
The closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of
which is two dot-separated integers, where the second integer
indicates a particular build job. While it would be possible to use
that second number to identify the build job in our shared scripts, it
doesn't seem like a good idea to rely on that:
- Though the build job numbering sequence seems to be stable so far,
Travis CI's documentation doesn't explicitly states that it is
indeed stable and will remain so in the future. And even if it
were stable,
- if we were to remove or insert a build job in the middle, then the
job numbers of all subsequent build jobs would change accordingly.
So roll our own means of simple build job identification and introduce
the $jobname environment variable in our builds, setting it in the
environments of the explicitly included jobs in '.travis.yml', while
constructing one in 'ci/lib-travisci.sh' as the combination of the OS
and compiler name for the GCC and Clang Linux and OSX build jobs. Use
$jobname instead of $TRAVIS_OS_NAME in scripts taking different
actions based on the OS and build job (when installing P4 and Git LFS
dependencies and including them in $PATH). The following two patches
will also rely on $jobname.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Linux build jobs on Travis CI skip the P4 and Git LFS tests since
commit 657343a60 (travis-ci: move Travis CI code into dedicated
scripts, 2017-09-10), claiming there are no P4 or Git LFS installed.
The reason is that P4 and Git LFS binaries are not installed to a
directory in the default $PATH, but their directories are prepended to
$PATH. This worked just fine before said commit, because $PATH was
set in a scriptlet embedded in our '.travis.yml', thus its new value
was visible during the rest of the build job. However, after these
embedded scriptlets were moved into dedicated scripts executed in
separate shell processes, any variable set in one of those scripts is
only visible in that single script but not in any of the others. In
this case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and
modifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees
Travis CI's default $PATH.
Move adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all
other 'ci/' scripts, so all those scripts will see the updated $PATH
value.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
09f5e97 ("travis-ci: skip a branch build if equal tag is present",
2017-09-17) introduced the "skip_branch_tip_with_tag" function with
a broken string comparison. Fix it!
Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If we push a branch and a tag pointing to the HEAD of this branch,
then Travis CI would run the build twice. This wastes resources and
slows the testing.
Add a function to detect this situation and skip the build the branch
if appropriate. Invoke this function on every build.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most of the Travis CI commands are in the '.travis.yml'. The yml format
does not support functions and therefore code duplication is necessary
to run commands across all builds.
To fix this, add a library for common CI functions. Move all Travis CI
code into dedicated scripts and make them call the library first.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ls/travis-relays-for-windows-ci:
travis-ci: retry if Git for Windows CI returns HTTP error 502 or 503
travis-ci: handle Git for Windows CI status "failed" explicitly
Travis CI gained a task to format the documentation with both
AsciiDoc and AsciiDoctor.
* ls/travis-doc-asciidoctor:
travis-ci: check AsciiDoc/AsciiDoctor stderr output
travis-ci: unset compiler for jobs that do not need one
travis-ci: parallelize documentation build
travis-ci: build documentation with AsciiDoc and Asciidoctor
The Git for Windows CI web app sometimes returns HTTP errors of
"502 bad gateway" or "503 service unavailable" [1]. We also need to
check the HTTP content because the GfW web app seems to pass through
(error) results from other Azure calls with HTTP code 200.
Wait a little and retry the request if this happens.
[1] https://docs.microsoft.com/en-in/azure/app-service-web/app-service-web-troubleshoot-http-502-http-503
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git for Windows CI returns "completed: failed" if a build or test
failure happened. This case was processed as "Unhandled status".
Handle the case explicitly.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the $STATUS variable contains a "%" character then printf will
interpret that as invalid format string. Fix this by formatting $STATUS
as string.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
`make` does not necessarily fail with an error code if
Asciidoc/AsciiDoctor encounters problems. Anything written to stderr
might be a better indicator for problems.
Ensure that nothing is written to stderr during a documentation build.
The redirects do not work in `sh`, therefore the script uses `bash`.
This shouldn't be a problem as the script is only executed on TravisCI.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The documentation job without parallelization takes ~10min on TravisCI.
With parallelization ("--jobs=2") it takes ~6min.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
ec3366e introduced a knob to enable the use of Asciidoctor in addition
to AsciiDoc. Build the documentation on TravisCI with this knob to
reduce the likeliness of breaking Asciidoctor support in the future.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most Git developers work on Linux and they have no way to know if their
changes would break the Git for Windows build. Let's fix that by adding
a job to TravisCI that builds and tests Git on Windows. Unfortunately,
TravisCI does not support Windows.
Therefore, we did the following:
* Johannes Schindelin set up a Visual Studio Team Services build
sponsored by Microsoft and made it accessible via an Azure Function
that speaks a super-simple API. We made TravisCI use this API to
trigger a build, wait until its completion, and print the build and
test results.
* A Windows build and test run takes up to 3h and TravisCI has a timeout
after 50min for Open Source projects. Since the TravisCI job does not
use heavy CPU/memory/etc. resources, the friendly TravisCI folks
extended the job timeout for git/git to 3h.
Things, that would need to be done:
* Someone with write access to https://travis-ci.org/git/git would need
to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI
repository setting [1]. Afterwards the build should just work.
Things, that might need to be done:
* The Windows box can only process a single build at a time. A second
Windows build would need to wait until the first finishes. This
waiting time and the build time after the wait could exceed the 3h
threshold. If this is a problem, then it is likely to happen every day
as usually multiple branches are pushed at the same time (pu/next/
master/maint). I cannot test this as my TravisCI account has the 50min
timeout. One solution could be to limit the number of concurrent
TravisCI jobs [2].
[1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings
[2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When Git v2.9.1 was released, it had a bug that showed only on Windows
and on 32-bit systems: our assumption that `unsigned long` can hold
64-bit values turned out to be wrong.
This could have been caught earlier if we had a Continuous Testing
set up that includes a build and test run on 32-bit Linux.
Let's do this (and take care of the Windows build later). This patch
asks Travis CI to install a Docker image with 32-bit libraries and then
goes on to build and test Git using this 32-bit setup.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Build documentation as separate Travis CI job to check for
documentation errors.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
SZEDER Gábor
Lars Schneider
Junio C Hamano
René Genz
Johannes Schindelin