Граф коммитов

58 Коммитов

Автор SHA1 Сообщение Дата
Ævar Arnfjörð Bjarmason d24fbca7a5 Remove Git's support for smoke testing
I'm no longer running the Git smoke testing service at
smoke.git.nix.is due to Smolder being a fragile piece of software not
having time to follow through on making it easy for third parties to
run and submit their own smoke tests.

So remove the support in Git for sending smoke tests to
smoke.git.nix.is, it's still easy to modify the test suite to submit
smokes somewhere else.

This reverts the following commits:

    Revert "t/README: Add SMOKE_{COMMENT,TAGS}= to smoke_report target" -- e38efac87d
    Revert "t/README: Document the Smoke testing" -- d15e9ebc5c
    Revert "t/Makefile: Create test-results dir for smoke target" -- 617344d77b
    Revert "tests: Infrastructure for Git smoke testing" -- b6b84d1b74

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-13 02:29:07 -08:00
Junio C Hamano be653d6cb8 Merge branch 'mk/grep-pcre'
* mk/grep-pcre:
  git-grep: Fix problems with recently added tests
  git-grep: Update tests (mainly for -P)
  Makefile: Pass USE_LIBPCRE down in GIT-BUILD-OPTIONS
  git-grep: update tests now regexp type is "last one wins"
  git-grep: do not die upon -F/-P when grep.extendedRegexp is set.
  git-grep: Bail out when -P is used with -F or -E
  grep: Add basic tests
  configure: Check for libpcre
  git-grep: Learn PCRE
  grep: Extract compile_regexp_failed() from compile_regexp()
  grep: Fix a typo in a comment
  grep: Put calls to fixmatch() and regmatch() into patmatch()
  contrib/completion: --line-number to git grep
  Documentation: Add --line-number to git-grep synopsis
2011-05-30 00:00:07 -07:00
Michał Kiedrowicz 8f852ce613 grep: Add basic tests
This modest patch adds simple tests for git grep -P/--perl-regexp and
its interoperation with -i and -w.

Tests are only enabled when prerequisite LIBPCRE is defined (it's
automatically set based on USE_LIBPCRE in test-lib.sh).

Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-05-09 16:29:54 -07:00
Mathias Lafeldt 72942a617c t/README: unify documentation of test function args
Document all test function arguments in the same way.

While at it, tweak the description of test_path_is_* (thanks to Junio),
and correct some grammatical errors.

Signed-off-by: Mathias Lafeldt <misfire@debugon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-04-26 11:45:56 -07:00
Junio C Hamano 17a0299807 Merge branch 'maint'
* maint:
  contrib/thunderbird-patch-inline: do not require bash to run the script
  t8001: check the exit status of the command being tested
  strbuf.h: remove a tad stale docs-in-comment and reference api-doc instead
  Typos: t/README
  Documentation/config.txt: make truth value of numbers more explicit
  git-pack-objects.txt: fix grammatical errors
  parse-remote: replace unnecessary sed invocation
2011-03-30 14:10:41 -07:00
Michael Witten 63d3294593 Typos: t/README
Signed-off-by: Michael Witten <mfwitten@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-30 11:59:52 -07:00
Junio C Hamano cdc34664d4 Merge branch 'maint'
* maint:
  Update draft release notes to 1.7.4.2
  Work around broken ln on solaris as used in t8006
  t/README: Add a note about running commands under valgrind
2011-03-20 22:14:47 -07:00
Carlos Martín Nieto 9aec68d3ea t/README: Add a note about running commands under valgrind
The test suite runs valgrind with certain options activated. Add a
note saying how to run commands under the same conditions as the test
suite does.

Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-20 21:19:38 -07:00
Piotr Krukowiecki 0986de94f9 Documentation: running test with --debug keeps "trash" directory
Signed-off-by: Piotr Krukowiecki <piotr.krukowiecki@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-15 14:18:34 -07:00
Mathias Lafeldt 681186ae3a Fix typo in t/README
Signed-off-by: Mathias Lafeldt <misfire@debugon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09 12:53:13 -08:00
Johannes Sixt 95b104c830 t/README: hint about using $(pwd) rather than $PWD in tests
This adds just a "do it this way" instruction without a lot of explanation,
because the details are too complex to be explained at this point.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-01-11 10:51:25 -08:00
Junio C Hamano b3ff808b71 Merge branch 'en/and-cascade-tests'
* en/and-cascade-tests: (25 commits)
  t4124 (apply --whitespace): use test_might_fail
  t3404: do not use 'describe' to implement test_cmp_rev
  t3404 (rebase -i): introduce helper to check position of HEAD
  t3404 (rebase -i): move comment to description
  t3404 (rebase -i): unroll test_commit loops
  t3301 (notes): use test_expect_code for clarity
  t1400 (update-ref): use test_must_fail
  t1502 (rev-parse --parseopt): test exit code from "-h"
  t6022 (renaming merge): chain test commands with &&
  test-lib: introduce test_line_count to measure files
  tests: add missing &&, batch 2
  tests: add missing &&
  Introduce sane_unset and use it to ensure proper && chaining
  t7800 (difftool): add missing &&
  t7601 (merge-pull-config): add missing &&
  t7001 (mv): add missing &&
  t6016 (rev-list-graph-simplify-history): add missing &&
  t5602 (clone-remote-exec): add missing &&
  t4026 (color): remove unneeded and unchained command
  t4019 (diff-wserror): add lots of missing &&
  ...

Conflicts:
	t/t7006-pager.sh
2010-11-24 15:51:49 -08:00
Jonathan Nieder fb3340a6a7 test-lib: introduce test_line_count to measure files
Some tests check their output with code like the following:

	test "$(git ls-files -u B | wc -l)" -eq 3 || {
		echo "BAD: should have left stages for B"
		return 1
	}

The verbose failure condition is used because test, unlike
diff, does not print any useful information about the
nature of the failure when it fails.

Introduce a test_line_count function to help. If used like

	git ls-files -u B >output &&
	test_line_count -eq 3 output

it will produce output like

	test_line_count: line count for output !-eq 3
	100644 b023018cabc396e7692c70bbf5784a93d3f738ab 2	hi.c
	100644 45b983be36b73c0788dc9cbcb76cbb80fc7bb057 3	hi.c

on failure.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-09 14:47:17 -08:00
Michael J Gruber 28d836c815 test: allow running the tests under "prove"
You can run "make DEFAULT_TEST_TARGET=prove test" to run the test under
"prove" (or $(PROVE) if set).  The output is a bit easier to read when
running many tests in parallel.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Liked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Liked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-26 11:25:48 -07:00
Elijah Newren 00648ba050 Introduce sane_unset and use it to ensure proper && chaining
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-06 13:26:33 -07:00
Ævar Arnfjörð Bjarmason 892e6f7ea6 test-lib: make test_expect_code a test command
Change test_expect_code to be a normal test command instead of a
top-level command.

As a top-level command it would fail in cases like:

    test_expect_code 1 'phoney' '
        foo && bar && (exit 1)
    '

Here the test might incorrectly succeed if "foo" or "bar" happened to
fail with exit status 1. Instead we now do:

    test_expect_success 'phoney' '
        foo && bar && test_expect_code 1 "(exit 1)"
    '

Which will only succeed if "foo" and "bar" return status 0, and "(exit
1)" returns status 1.  Note that test_expect_code has been made slightly
noisier, as it reports the exit code it receives even upon success.

Some test code in t0000-basic.sh relied on the old semantics of
test_expect_code to test the test_when_finished command. I've
converted that code to use an external test similar to the TODO test I
added in v1.7.3-rc0~2^2~3.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-06 13:26:11 -07:00
Junio C Hamano a2c6726417 Merge branch 'ab/test-2'
* ab/test-2: (51 commits)
  tests: factor HOME=$(pwd) in test-lib.sh
  test-lib: use subshell instead of cd $new && .. && cd $old
  tests: simplify "missing PREREQ" message
  t/t0000-basic.sh: Run the passing TODO test inside its own test-lib
  test-lib: Allow overriding of TEST_DIRECTORY
  test-lib: Use "$GIT_BUILD_DIR" instead of "$TEST_DIRECTORY"/../
  test-lib: Use $TEST_DIRECTORY or $GIT_BUILD_DIR instead of $(pwd) and ../
  test: Introduce $GIT_BUILD_DIR
  cvs tests: do not touch test CVS repositories shipped with source
  t/t9602-cvsimport-branches-tags.sh: Add a PERL prerequisite
  t/t9601-cvsimport-vendor-branch.sh: Add a PERL prerequisite
  t/t7105-reset-patch.sh: Add a PERL prerequisite
  t/t9001-send-email.sh: convert setup code to tests
  t/t9001-send-email.sh: change from skip_all=* to prereq skip
  t/t9001-send-email.sh: Remove needless PROG=* assignment
  t/t9600-cvsimport.sh: change from skip_all=* to prereq skip
  lib-patch-mode tests: change from skip_all=* to prereq skip
  t/t3701-add-interactive.sh: change from skip_all=* to prereq skip
  tests: Move FILEMODE prerequisite to lib-prereq-FILEMODE.sh
  t/Makefile: Create test-results dir for smoke target
  ...

Conflicts:
	t/t6035-merge-dir-to-symlink.sh
2010-09-04 08:15:36 -07:00
Ævar Arnfjörð Bjarmason 99d9050d25 t/README: Update "Skipping tests" to align with best practices
The example I initially added to "Skipping tests" wasn't very
good. We'd rather skip tests using the three-arg prereq form to the
test_* functions, not bail out with a skip message.

Change the documentation to reflect that, but retain the bailout
example under a disclaimer which explains that it's probably not a
good idea to use it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:45 -07:00
Ævar Arnfjörð Bjarmason e8b55f5c58 t/README: Add a note about the dangers of coverage chasing
Having no coverage at all is almost always a bad sign, but trying to
attain 100% coverage everywhere is usually a waste of time. Add a
paragraph to explain this to future test writers.

Inspired-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:37 -07:00
Ævar Arnfjörð Bjarmason 0c357544b0 t/README: A new section about test coverage
Document how test writers can generate coverage reports, to ensure
that their tests are really testing the code they think they're
testing.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:37 -07:00
Ævar Arnfjörð Bjarmason e38efac87d t/README: Add SMOKE_{COMMENT,TAGS}= to smoke_report target
The smoke server supports a free form text field with comments about a
report, and a comma delimited list of tags. Change the smoke_report
target to expose this functionality. Now smokers can send more data
that explains and categorizes the reports they're submitting.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:14 -07:00
Ævar Arnfjörð Bjarmason d15e9ebc5c t/README: Document the Smoke testing
Git now has a smoke testing service at http://smoke.git.nix.is that
anyone can send reports to. Change the t/README file to mention this.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:14 -07:00
Ævar Arnfjörð Bjarmason c91cfd1916 tests: A SANITY test prereq for testing if we're root
Some tests depend on not being able to write to files after chmod
-w. This doesn't work when running the tests as root.

Change test-lib.sh to test if this works, and if so it sets a new
SANITY test prerequisite. The tests that use this previously failed
when run under root.

There was already a test for this in t3600-rm.sh, added by Junio C
Hamano in 2283645 in 2006. That check now uses the new SANITY
prerequisite.

Some of this was resurrected from the "Tests in Cygwin" thread in May
2009:

    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:04 -07:00
Ævar Arnfjörð Bjarmason be53deef0d t/README: Document the predefined test prerequisites
The README for the test library suggested that you grep the
test-lib.sh for test_set_prereq to see what the preset prerequisites
were.

Remove that bit, and write a section explaining all the preset
prerequisites. Most of the text was lifted from from Junio C Hamano
and Johannes Sixt, See the "Tests in Cygwin" thread in May 2009 for
the originals:

    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385
    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:04 -07:00
Ævar Arnfjörð Bjarmason 93a5724613 test-lib: Add support for multiple test prerequisites
Change the test_have_prereq function in test-lib.sh to support a
comma-separated list of prerequisites. This is useful for tests that
need e.g. both POSIXPERM and SANITY.

The implementation was stolen from Junio C Hamano and Johannes Sixt,
the tests and documentation were not. See the "Tests in Cygwin" thread
in May 2009 for the originals:

    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385
    http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:04 -07:00
Matthieu Moy 2caf20c52b test-lib: user-friendly alternatives to test [-d|-f|-e]
The helper functions are implemented, documented, and used in a few
places to validate them, but not everywhere to avoid useless code churn.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-11 10:21:36 -07:00
Jonathan Nieder bdcaa325b4 t/README: correct an exception when breaking a && chain in tests
The correct advice should have been taken from c289c31 (t/t7006: ignore
return status of shell's unset builtin, 2010-06-02).  A real-life issue
we experienced was with "unset", not with "export" (exporting an
unset variable may have similar portability issues, though).

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-21 11:52:26 -07:00
Junio C Hamano 5f7baac1e3 Merge branch 'maint'
* maint:
  t/README: clarify test_must_fail description
  Check size of path buffer before writing into it

Conflicts:
	t/README
2010-07-20 11:29:30 -07:00
Brandon Casey 971ecbd1f8 t/README: clarify test_must_fail description
Some have found the wording of the description to be somewhat ambiguous
with respect to when it is desirable to use test_must_fail instead of
"! <git-command>".  Tweak the wording somewhat to hopefully clarify that
it is _because_ test_must_fail can detect segmentation fault that it is
desirable to use it instead of "! <git-command>".

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-20 11:26:39 -07:00
Jonathan Nieder c9667456d2 t/README: document more test helpers
There is no documentation in t/README for test_must_fail,
test_might_fail, test_cmp, or test_when_finished.

Reported-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-06 21:26:11 -07:00
Junio C Hamano 6fd45295ae t/README: proposed rewording...
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05 11:37:30 -07:00
Ævar Arnfjörð Bjarmason 20873f45e7 t/README: Document the do's and don'ts of tests
Add a "Do's, don'ts & things to keep in mind" subsection to the
"Writing Tests" documentation. Much of this is based on Junio C
Hamano's "Test your stuff" section in
<7vhbkj2kcr.fsf@alter.siamese.dyndns.org>.

I turned it into a list of do's and don'ts to make it easier to skim
it, and integrated my note that a TAP harness will get confused if you
print "ok" or "not ok" at the beginning of a line.

Thad had to be fixed in 335f87871f when
TAP support was introduced.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05 11:23:47 -07:00
Ævar Arnfjörð Bjarmason b5500d16cd t/README: Add a section about skipping tests
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05 11:23:45 -07:00
Ævar Arnfjörð Bjarmason 97d9fd925b t/README: Document test_expect_code
test_expect_code (which was introduced in d3bfdb75) never had any
documentation.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05 11:23:43 -07:00
Ævar Arnfjörð Bjarmason 2fac6a4b93 t/README: Document test_external*
There was do documentation for the test_external_without_stderr and
test_external functions.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05 11:23:41 -07:00
Ævar Arnfjörð Bjarmason 9a897893a7 t/README: Document the prereq functions, and 3-arg test_*
There was no documentation for the test_set_prereq and
test_have_prereq functions, or the three-arg form of
test_expect_success and test_expect_failure.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05 11:23:40 -07:00
Ævar Arnfjörð Bjarmason 85b0b34ea4 t/README: Typo: paralell -> parallel
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05 11:23:37 -07:00
Ævar Arnfjörð Bjarmason e1ca1c9d9b t/README: The trash is in 't/trash directory.$name'
There's a unique trash directory for each test, not a single directory
as the previous documentation suggested.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05 11:23:31 -07:00
Ævar Arnfjörð Bjarmason 5099b99d25 test-lib: Adjust output to be valid TAP format
TAP, the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there. Before:

    $ ./t0005-signals.sh
    *   ok 1: sigchain works
    * passed all 1 test(s)

And after:

    $ ./t0005-signals.sh
    ok 1 - sigchain works
    # passed all 1 test(s)
    1..1

The advantage of using TAP is that any program that reads the format
(a "test harness") can run the tests. The most popular of these is the
prove(1) utility that comes with Perl. It can run tests in parallel,
display colored output, format the output to console, file, HTML etc.,
and much more. An example:

    $ prove ./t0005-signals.sh
    ./t0005-signals.sh .. ok
    All tests successful.
    Files=1, Tests=1,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.01 cusr  0.02 csys =  0.06 CPU)
    Result: PASS

prove(1) gives you human readable output without being too
verbose. Running the test suite in parallel with `make test -j15`
produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
makes it easy to follow what's going on.

All this patch does is re-arrange the output a bit so that it conforms
with the TAP spec, everything that the test suite did before continues
to work. That includes aggregating results in t/test-results/, the
--verbose, --debug and other options for tests, and the test color
output.

TAP harnesses ignore everything that they don't know about, so running
the tests with --verbose works:

    $ prove ./t0005-signals.sh :: --verbose --debug
    ./t0005-signals.sh .. Terminated
    ./t0005-signals.sh .. ok
    All tests successful.
    Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.01 cusr  0.01 csys =  0.05 CPU)
    Result: PASS

Just supply the -v option to prove itself to get all the verbose
output that it suppresses:

    $ prove -v ./t0005-signals.sh :: --verbose --debug
    ./t0005-signals.sh ..
    Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/
    expecting success:
            test-sigchain >actual
            case "$?" in
            143) true ;; # POSIX w/ SIGTERM=15
              3) true ;; # Windows
              *) false ;;
            esac &&
            test_cmp expect actual
    Terminated
    ok 1 - sigchain works
    # passed all 1 test(s)
    1..1
    ok
    All tests successful.
    Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.01 cusr  0.01 csys =  0.04 CPU)
    Result: PASS

As a further example, consider this test script that uses a lot of
test-lib.sh features by Jakub Narebski:

    #!/bin/sh

    test_description='this is a sample test.

    This test is here to see various test outputs.'

    . ./test-lib.sh

    say 'diagnostic message'

    test_expect_success 'true  test' 'true'
    test_expect_success 'false test' 'false'

    test_expect_failure 'true  test (todo)' 'true'
    test_expect_failure 'false test (todo)' 'false'

    test_debug 'echo "debug message"'

    test_done

The output of that was previously:

    * diagnostic message                      # yellow
    *   ok 1: true  test
    * FAIL 2: false test                      # bold red
            false
    *   FIXED 3: true  test (todo)
    *   still broken 4: false test (todo)     # bold green
    * fixed 1 known breakage(s)               # green
    * still have 1 known breakage(s)          # bold red
    * failed 1 among remaining 3 test(s)      # bold red

But is now:

    diagnostic message                                    # yellow
    ok 1 - true  test
    not ok - 2 false test                                 # bold red
    #       false
    ok 3 - true  test (todo) # TODO known breakage
    not ok 4 - false test (todo) # TODO known breakage    # bold green
    # fixed 1 known breakage(s)                           # green
    # still have 1 known breakage(s)                      # bold red
    # failed 1 among remaining 3 test(s)                  # bold red
    1..4

All the coloring is preserved when the test is run manually. Under
prove(1) the test performs as expected, even with --debug and
--verbose options:

    $ prove ./example.sh :: --debug --verbose
    ./example.sh .. Dubious, test returned 1 (wstat 256, 0x100)
    Failed 1/4 subtests
            (1 TODO test unexpectedly succeeded)

    Test Summary Report
    -------------------
    ./example.sh (Wstat: 256 Tests: 4 Failed: 1)
      Failed test:  2
      TODO passed:   3
      Non-zero exit status: 1
    Files=1, Tests=4,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.00 cusr  0.01 csys =  0.03 CPU)
    Result: FAIL

The TAP harness itself doesn't get confused by the color output, they
aren't used by test-lib.sh stdout isn't open to a terminal (test -t 1).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-25 10:03:19 -07:00
Thomas Rast 0d4dbcd35e t/README: document --root option
We've had this option since f423ef5 (tests: allow user to specify
trash directory location, 2009-08-09).  Make it easier to look up :-)

Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-11 13:49:28 -07:00
Matthew Ogilvie e4597aae65 run test suite without dashed git-commands in PATH
Only put bin-wrappers in the PATH (not GIT_EXEC_PATH), to emulate the
default installed user environment, and ensure all the programs run
correctly in such an environment.  This is now the default, although
it can be overridden with a --with-dashes test option when running
tests.

Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-03 11:38:00 -08:00
Matthew Ogilvie e160da7f60 t/README: Document GIT_TEST_INSTALLED and GIT_TEST_EXEC_PATH
These were added without documentation in 2009-03-16 (6720721).

Signed-off-by: Matthew Ogilvie <mmogilvi_git@miniinfo.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-29 23:09:56 -08:00
Johannes Schindelin 3da9365234 Tests: let --valgrind imply --verbose and --tee
It does not make much sense to run the (expensive) valgrind tests and
not look at the output.

To prevent output from scrolling out of reach, the parameter --tee is
implied, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-03 22:01:23 -08:00
Johannes Schindelin 44138559e8 test-lib.sh: optionally output to test-results/$TEST.out, too
When tests are run in parallel and a few tests fail, it does not help
that the output of the terminal is totally confusing, as you rarely know
which test which line came from.

So introduce the option '--tee' which triggers that the output of the
tests will be written to t/test-results/$TEST.out in addition to the
terminal, where $TEST is the basename of the script.

Unfortunately, there seems to be no way to redirect a given file
descriptor to a specified subprocess in POSIX shell, only redirection
to a file is supported via 'exec > $FILE'.

At least with bash, one might think that 'exec >($COMMAND)' would work
as intended, but it does not.

The common way to work around the lack of proper tools support is to
work with named pipes, alas, one of our most beloved platforms does not
really support named pipes.  Besides, we would need a pipe for every
script, as the whole point of this patch is to allow parallel execution.

Therefore, we handle the redirection in the following way: when '--tee'
was passed to the test script, the variable GIT_TEST_TEE_STARTED is set
(to avoid triggering that code path again) and the script is started
_again_, in a subshell, redirected to the command "tee".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-03 22:01:09 -08:00
Johannes Schindelin 4e1be63c3b Add valgrind support in test scripts
This patch adds the ability to use valgrind's memcheck tool to
diagnose memory problems in Git while running the test scripts.

It requires valgrind 3.4.0 or newer.

It works by creating symlinks to a valgrind script, which have the same
name as our Git binaries, and then putting that directory in front of
the test script's PATH as well as set GIT_EXEC_PATH to that directory.
Git scripts are symlinked from that directory directly.

That way, Git binaries called by Git scripts are valgrinded, too.

Valgrind can be used by specifying "GIT_TEST_OPTS=--valgrind" in the
make invocation. Any invocation of git that finds any errors under
valgrind will exit with failure code 126. Any valgrind output will go
to the usual stderr channel for tests (i.e., /dev/null, unless -v has
been specified).

If you need to pass options to valgrind -- you might want to run
another tool than memcheck, for example -- you can set the environment
variable GIT_VALGRIND_OPTIONS.

A few default suppressions are included, since libz seems to trigger
quite a few false positives. We'll assume that libz works and that we
can ignore any errors which are reported there.

Note: it is safe to run the valgrind tests in parallel, as the links in
t/valgrind/bin/ are created using proper locking.

Initial patch and all the hard work by Jeff King.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-03 22:00:58 -08:00
Johannes Schindelin 008849689e test-lib.sh: introduce test_commit() and test_merge() helpers
Often we just need to add a commit with a given (short) name, that will
be tagged with the same name.  Now, relatively complicated graphs can be
constructed easily and in a clear fashion:

	test_commit A &&
	test_commit B &&
	git checkout A &&
	test_commit C &&
	test_merge D B

will construct this graph:

	A - B
	  \   \
	    C - D

For simplicity, files whose name is the lower case version of the commit
message (to avoid a warning about ambiguous names) will be committed, with
the corresponding commit messages as contents.

If you need to provide a different file/different contents, you can use
the more explicit form

	test_commit $MESSAGE $FILENAME $CONTENTS

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-28 20:16:37 -08:00
Jakub Narebski fbd458a3f6 t/README: Add 'Skipping Tests' section below 'Running Tests'
Add description of GIT_SKIP_TESTS variable, taken almost verbatim
(adjusting for conventions in t/README) from the commit message in

   04ece59 (GIT_SKIP_TESTS: allow users to omit tests that are known to break)

Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-21 00:02:31 -07:00
Lea Wiemann 5e2c08c6f0 test-lib.sh: add --long-tests option
Add a --long-tests option to test-lib.sh, which enables tests to
selectively run more exhaustive (longer running, potentially
brute-force) tests.  Such exhaustive tests would only be useful if one
works on the specific module that is being tested -- for a general "cd
t/; make" to check whether everything is OK, such exhaustive tests
shouldn't be run by default since the longer it takes to run the
tests, the less often they are actually run.

Signed-off-by: Lea Wiemann <LeWiemann@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-19 14:21:42 -07:00
Brandon Casey 9231e3a953 t/Makefile: "trash" directory was renamed recently
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-18 21:39:58 -07:00
Junio C Hamano 41ac414ea2 Sane use of test_expect_failure
Originally, test_expect_failure was designed to be the opposite
of test_expect_success, but this was a bad decision.  Most tests
run a series of commands that leads to the single command that
needs to be tested, like this:

    test_expect_{success,failure} 'test title' '
	setup1 &&
        setup2 &&
        setup3 &&
        what is to be tested
    '

And expecting a failure exit from the whole sequence misses the
point of writing tests.  Your setup$N that are supposed to
succeed may have failed without even reaching what you are
trying to test.  The only valid use of test_expect_failure is to
check a trivial single command that is expected to fail, which
is a minority in tests of Porcelain-ish commands.

This large-ish patch rewrites all uses of test_expect_failure to
use test_expect_success and rewrites the condition of what is
tested, like this:

    test_expect_success 'test title' '
	setup1 &&
        setup2 &&
        setup3 &&
        ! this command should fail
    '

test_expect_failure is redefined to serve as a reminder that
that test *should* succeed but due to a known breakage in git it
currently does not pass.  So if git-foo command should create a
file 'bar' but you discovered a bug that it doesn't, you can
write a test like this:

    test_expect_failure 'git-foo should create bar' '
        rm -f bar &&
        git foo &&
        test -f bar
    '

This construct acts similar to test_expect_success, but instead
of reporting "ok/FAIL" like test_expect_success does, the
outcome is reported as "FIXED/still broken".

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-01 20:49:34 -08:00