Running a test script with '-x' turns on 'set -x' tracing, the output
of which is normally sent to stderr. This causes a lot of
test failures, because many tests redirect and verify the stderr
of shell functions, most frequently that of 'test_must_fail'.
These issues were worked around somewhat in d88785e424 (test-lib: set
BASH_XTRACEFD automatically, 2016-05-11), so at least we could
reliably run tests with '-x' tracing under a Bash version supporting
BASH_XTRACEFD, i.e. v4.1 and later.
Futhermore, redirecting the stderr of test helper functions like
'test_must_fail' or 'test_expect_code' is the cause of a different
issue as well. If these functions detect something unexpected, they
will write their error messages intended to the user to thier stderr.
However, if their stderr is redirected in order to save and verify the
stderr of the tested git command invoked in the function, then the
function's error messages will be redirected as well. Consequently,
those messages won't reach the user, making the test's verbose output
less useful.
This patch makes it safe to redirect and verify the stderr of those
test helper functions which are meant to run the tested command given
as argument, even when running tests with '-x' and /bin/sh. This is
achieved through a couple of file descriptor redirections:
- Duplicate stderr of the tested command executed in the test helper
function from the function's fd 7 (see next point), to ensure that
the tested command's error messages go to a different fd than the
'-x' trace of the commands executed in the function or the
function's error messages.
- Duplicate the test helper function's fd 7 from the function's
original stderr, meaning that, after taking a detour through fd 7,
the error messages of the tested command do end up on the
function's original stderr.
- Duplicate stderr of the test helper function from fd 4, i.e. the
fd connected to the test script's original stderr and the fd used
for BASH_XTRACEFD. This ensures that the '-x' trace of the
commands executed in the function
- doesn't go to the function's original stderr, so it won't mess
with callers who want to save and verify the tested command's
stderr.
- does go to the same fd independently from the shell running
the test script, be it /bin/sh, an older Bash without
BASH_XTRACEFD, or a more recent Bash already supporting
BASH_XTRACEFD.
Furthermore, this also makes sure that the function's error
messages go to this fd 4, meaning that the user will be able to
see them even if the function's stderr is redirected in the test.
- Specify the latter two redirections above in the test helper
function's definition, so they are performed every time the
function is invoked, without the need to modify the callsites of
the function.
Perform these redirections in those test helper functions which can be
expected to have their stderr redirected, i.e. in the functions
'test_must_fail', 'test_might_fail', 'test_expect_code', 'test_env',
'nongit', 'test_terminal' and 'perl'. Note that 'test_might_fail',
'test_env', and 'nongit' are not involved in any test failures when
running tests with '-x' and /bin/sh.
The other test helper functions are left unchanged, because they
either don't run commands specified as their arguments, or redirecting
their stderr wouldn't make sense, or both.
With this change the number of failures when running the test suite
with '-x' tracing and /bin/sh goes down from 340 failed tests in 43
test scripts to 22 failed tests in 6 scripts (or 23 in 7, if the
system (OSX) uses an older Bash version without BASH_XTRACEFD to run
't9903-bash-prompt.sh').
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test helper lib-terminal always run an actual test_expect_* when
included, which screwed up with the use of skil-all that may have to
be done later.
* jk/lib-terminal-lazy:
t/lib-terminal: make TTY a lazy prerequisite
When lib-terminal.sh is sourced by a test script, we
immediately set up the TTY prerequisite. We do so inside a
test_expect_success, because that nicely isolates any
generated output.
However, this early test can interfere with a script that
later wants to skip all tests (e.g., t5541 then goes on to
set up the httpd server, and wants to skip_all if that
fails). TAP output doesn't let us skip everything after we
have already run at least one test.
We could fix this by reordering the inclusion of
lib-terminal.sh in t5541 to go after the httpd setup. That
solves this case, but we might eventually hit a case with
circular dependencies, where either lib-*.sh include might
want to skip_all after the other has run a test. So
instead, let's just remove the ordering constraint entirely
by doing the setup inside a test_lazy_prereq construct,
rather than in a regular test. We never cared about the
test outcome anyway (it was written to always succeed).
Note that in addition to setting up the prerequisite, the
current test also defines test_terminal. Since we can't
affect the environment from a lazy_prereq, we have to hoist
that out. We previously depended on it _not_ being defined
when the TTY prereq isn't set as a way to ensure that tests
properly declare their dependency on TTY. However, we still
cover the case (see the in-code comment for details).
Reported-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A #! line in these files is misleading, since these scriptlets are
meant to be sourced with '.' (using whatever shell sources them)
instead of run directly using the interpreter named on the #! line.
Removing the #! line shouldn't hurt syntax highlighting since
these files have filenames ending with '.sh'. For documentation,
add a brief description of how the files are meant to be used in
place of the shebang line.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As of the last commit, we can use "perl" instead of
"$PERL_PATH" when running tests, as the former is now a
function which uses the latter. As the shorter "perl" is
easier on the eyes, let's switch to using it everywhere.
This is not quite a mechanical s/$PERL_PATH/perl/
replacement, though. There are some places where we invoke
perl from a script we generate on the fly, and those scripts
do not have access to our internal shell functions. The
result can be double-checked by running:
ln -s /bin/false bin-wrappers/perl
make test
which continues to pass even after this patch.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Mac OS X 10.5.0, test_terminal gets stuck reading from the pty
master every once in a while. To reproduce the problem:
perl -MIO::Pty -MFile::Copy -e '
for (my $i = 0;; $i++) {
my $master = new IO::Pty;
my $slave = $master->slave;
if (fork == 0) {
close $master or die "close: $!";
open STDOUT, ">&", $slave or die "dup2: $!";
close $slave or die "close: $!";
exec("echo", "hi", $i) or die "exec: $!";
}
close $slave or die "close: $!";
copy($master, \*STDOUT) or die "copy: $!";
close $master or die "close: $!";
wait;
}
'
It blocks after 7000 iterations or so in sysread(). The relevant
sysread() call is the second call by the parent, which presumably
executes before the child dies but after the parent has read all
output from there.
Since this is an intermitent problem, the quick check of terminal
support in lib-terminal doesn't catch it. Skip these tests on the Mac
for now.
Noticed-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For terminal tests that capture output/stderr, the TTY prerequisite
warning does not quite work for commands like
test_terminal foo >out 2>err
because the warning gets "swallowed" up by the redirection that's
supposed only to be done by the subcommand.
Even worse, the outcome depends on whether stdout was already a
terminal (in which case test_terminal is a noop) or not (in which case
test_terminal introduces a pseudo-tty in the middle of the pipeline).
$ test_terminal.perl sh -c 'test -t 1 && echo >&2 YES' >out
YES
$ sh -c 'test -t 1 && echo >&2 YES' >out
$
So:
- use the test_terminal script even when running with "-v".
- skip tests that require a terminal when the test_terminal
script is unusable because IO::Pty is not installed.
- write the "need to declare TTY prerequisite" message to fd 4,
where it will be printed when running tests with -v, rather
than being swallowed up by an unrelated redireciton.
Noticed-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is easy to forget to declare the TTY prerequisite when
writing tests on a system where it would always be satisfied
(because IO::Pty is installed; see v1.7.3-rc0~33^2, 2010-08-16
for example). Automatically detect this problem so there is
no need to remember.
test_terminal: need to declare TTY prerequisite
test_must_fail: command not found: test_terminal echo hi
test_terminal returns status 127 in this case to simulate
not being available.
Also replace the SIMPLEPAGERTTY prerequisite on one test with
"SIMPLEPAGER,TTY", since (1) the latter is supported now and
(2) the prerequisite detection relies on the TTY prereq being
explicitly declared.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some outputs (like the pager) care whether stdout is a
terminal. Others (like progress meters) care about stderr.
This patch sets up both. Technically speaking, we could go
further and set up just one (because either the other goes
to a terminal, or because our tests are only interested in
one). This patch does both to keep the interface to
lib-terminal simple.
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Other tests besides the pager ones may want to check how we handle
output to a terminal. This patch makes the code reusable.
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
SZEDER Gábor
Alexander Kuleshov
Junio C Hamano
Jeff King
Jonathan Nieder