One can run individual tests with python testfile.py Class.method. But
the output for tests only shows the method so looking at a test output
is not enough to tell how to run one particular test. Moreover, there
are many cases where a test file contains multiple classes, and we fail
to identify the difference between those classes.
Python's unittest has decorators to mark tests as skipped or expecting
to fail. Mozunit runner fails to print anything for them.
While here, somehow unify the output for TEST-UNEXPECTED-FAIL.
This leaves known failures silent about the failure. It would be better
to only show them with a flag, like -v, and to leave it to a followup.
Python's unittest separates errors and failures, where the former is
uncaught exceptions and the latter uncaught assertions. We were only
printing the former, making the mozunit runner output useless to debug
failures due to assertions in the code being tested.
I would usually edit the test to temporarily switch to unittest.main().
Enough is enough, handle failures properly.
At the same time, instead of printing all the errors one after the other
at the end, print them right after the TEST-UNEXPECTED-FAIL message.
Mike Hommey
Gregory Szorc
Yati Sagade
Ed Morley
Gervase Markham
Axel Hecht