Граф коммитов

26 Коммитов

Автор SHA1 Сообщение Дата
Dan Fandrich 296baf45a0 runtests: call initserverconfig() in the runner
This must be done so variables pick up the runner's unique $LOGDIR.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich 78d8bc4c63 runtests: use a per-runner random seed
Each runner needs a unique random seed to reduce the chance of port
number collisions. The new scheme uses a consistent per-runner source of
randomness which results in deterministic behaviour, as it did before.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich c95ca8dfeb runtests: use the correct fd after select
The code was using the wrong fd when determining which runner was ready
with a response.

Ref: #10818
Closes #11160
2023-05-20 22:15:15 -07:00
Dan Fandrich b43915b38f runtests: handle interrupted reads from IPC pipes
These can be interrupted by signals, especially SIGINT to shut down, and
must be restarted so the IPC call arrives correctly. If the read just
returns an error instead, the IPC calling state will go out of sync and
a proper shutdown won't happen.

Ref: #10818
2023-05-20 22:15:15 -07:00
Dan Fandrich 3d75029859 runtests: fix -c option when run with valgrind
The curl binary argument wasn't being quoted properly. This seems to
have broken at some point after quoting was added in commit 606b29fe.

Reported-by: Daniel Stenberg
Ref: #11073
Fixes #11074
Closes #11076
2023-05-05 09:52:38 -07:00
Dan Fandrich faebcee349 runtests: support creating more than one runner process
The controller currently only creates and uses one, but more are now
possible.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich 38465f9a55 runtests: spawn a new process for the test runner
When the -j option is given, a new process is spawned in which the test
programs are run and from which test servers are started. Only one
process can be started at once, but this is sufficient to test that the
infrastructure can isolate those functions in a new task. There should
be no visible difference between the two modes at the moment.

Ref: #10818
Closes #11064
2023-05-05 00:45:43 -07:00
Dan Fandrich d4a1b5b60c runtests: turn singletest() into a state machine
This allows it to run in a non-blocking manner.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich a98277fcc7 runtests: change runner interface to be asynchronous
Program arguments are marshalled and then written to the end of a pipe
which is later read from and the arguments unmarshalled before the
desired function is called normally.  The function return values are
then marshalled and written into another pipe when is later read from
and unmarshalled before being returned to the caller.

The implementation is currently blocking but can be made non-blocking
without any changes to the API.  This allows calling multiple runners
without blocking in the future.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich d4d22adf40 runtests: add a runner initialization function
This sets up the runner environment to start running tests.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich 82fa5ca836 runtests: remove directory from server filename variables
There will soon be multiple log directories so the paths will no longer
be static in runtests.pl. Also, get rid of $SERVER2IN which was not
used.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich 9fdad9dea1 runtests: use a function to obtain $LOGDIR for a test
This will no longer be static soon.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich 189f9e23b6 runtests: support buffering log messages in runner & servers
Log messages generated with logmsg can now be buffered and returned from
the runner as a return value.  This will be needed with parallel testing
to allow all messages for one test to be displayed together instead of
interspersed with messages of multiple tests. Buffering can be disabled
by setting a logging callback function with setlogfunc, which is
currently being done to preserve existing logging behaviour for now.

Some additional output is generated in verbose and debugprotocol modes,
which don't always use logmsg. These modes also impact some servers
which generate extra messages. No attempt is made to buffer everything
if these modes are enabled.

Ref: #10818
Closes #11016
2023-04-24 16:28:23 -07:00
Dan Fandrich b88ea272c1 runtests: create runner functions for clearlocks and stopservers
runtests.pl now uses runner for all server actions beyond the initial
variable configuration.

Ref: #10818
2023-04-24 16:04:24 -07:00
Dan Fandrich 640f4c2267 runtests: tightened servers package exports
The defaults are intended for runtests.pl, whereas runner.pm needs to
explicitly specify them.
2023-04-24 16:04:24 -07:00
Dan Fandrich 8da49c9e38 runtests: display logs on server failure in singletest()
This is closer to the place where logs are displayed on test failure.
Also, only display these logs if -p is given, which is the same flag
that controls display of test failure logs. Some server log files
need to be deleted later so that they stay around long enough to be
displayed on failure.

Ref: #10818
2023-04-24 16:03:44 -07:00
Dan Fandrich 90158f0bab runtests: turn a print into a logmsg
Also enable another couple of useful messages in verbose mode.

Ref: #10818
2023-04-24 12:13:17 -07:00
Dan Fandrich a549e046b1 runtests: refactor the main test loop into two
The test loop now has an initial loop that first runs through all
possible tests to build a set of those to attempt on this run based on
features and keywords and only then goes through that new list to run
them.  This actually makes it three loops through all tests cases, as
there is an existing loop that gathers possible test numbers from the
test files on disk.

This has two minor effects on the output: all the tests that will be
skipped are displayed at the start (instead of being interspersed with
other tests) and the -l option no longer shows a count of tests at the
end or a (misleading) statement that tests have run successfully. The
skipped tests are also omitted from the test results sent to AppVeyor
and Azure in CI builds.

Another effect is a reduction in the amount of work considered part of
the "Test definition reading and preparation time" reported with -r
making those figures slightly lower than before.

Ref: #10818
2023-04-22 13:07:35 -07:00
Dan Fandrich 020cf1c117 runtests: track only the current test timings in runner.pm
This avoids passing these data through through global variables, which
soon won't be possible.

Ref: #10818
2023-04-22 12:58:04 -07:00
Dan Fandrich 6210bc0111 runtests: simplify value returned regarding use of valgrind
As a side effect this will now also show in verbose mode that valgrind
is being skipped on tests that explicitly disable it, such as 600.

Ref: #10818
2023-04-22 12:58:03 -07:00
Dan Fandrich 04113319db runtests: fix problems on <killserver> failure
The verify time must be set in this case, like all cases. An error
message needs to be displayed as well.
2023-04-18 13:18:17 -07:00
Dan Fandrich 1f6a9f48aa runtests: fix perl warning when <tool> is wrong 2023-04-18 13:18:17 -07:00
Dan Fandrich 707f74c04a runtests: refactor into more packages
testutil.pm now contains a few miscellaneous functions that are used in
several places but have no better place to live.  subvariables moves to
servers.pm since most variables that it substitutes relate to servers,
so this is the most appropriate place. Rename a few functions for better
naming consistency.

Ref: #10818
Closes #10995
2023-04-18 13:18:17 -07:00
Dan Fandrich ba51b39732 runtests: call timestampskippedevents() in singletest
..rather than by the runner
2023-04-18 13:18:17 -07:00
Dan Fandrich 43b876d7df runtests: assume a newer Valgrind by default
The tests for an older Valgrind version should probably just be deleted,
given that they're testing for an 18-year-old version.
2023-04-18 13:18:17 -07:00
Dan Fandrich 390af1ed5e runtests: refactor test runner code into runner.pm
This is code that is directly responsible for running a single test.
This will eventually run in a separate process as part of the parallel
testing project.

Ref: #10818
2023-04-18 13:18:17 -07:00