We do this by setting the timestamp for the run (which is used for
graphing purposes) once we know we can actually do the run. We also
sleep for a second in order to avoid having 2 different runs show up
with the same timestamp.
This also has the happy effect of making it so sum(totals) in the sql
queries for the graphs don't end up showing weird spikes on a day when
multiple runs are made. Hooray!
Automagically parsing out the log information had screwed up help output
from programs when -h or --help was passed in (but not any other way)
because the ArgumentParser for getting log filename was intercepting
that. By having it not generate its own help, we fix that issue.
Had missed a reference to an old global variable that now needs to be
determined at runtime in run_spcshell (specifically, the xpcshell
variable). This fixes that, as well as adds some logging of the
xpcshell command line being run.
Normally we'd have been creating timestamped directories for any
netconfig/os pair that finished after the first netconfig/os pair to
finish for a particular srid. Now, the directories are named
srid_netconfig_os, so only re-runs of the whole 3-tuple will result in a
timestamped directory. This is what we want, really.
This avoids actually trying uploads to datazilla, which would not be
available during unit tests. Perhaps one day we'll mock it up, instead.
However, today is not that day.
Previously, we were serializing sub-objects (and sub-sub-objects) into
a string, before shoving them into the json object that gets passed
around the message queue. There's no particular need to do that, so
it's kind of dumb extra work we're doing in a couple places. This fixes
that by just passing around dicts serialized as json. Much cleaner, and
easier to read the contents while debugging, too!
*The inital call to self.reset() would barf because self.runconfig
didn't exist
*We were trying to put the logs in a directory under / instead of under
the stone ridge root
*Debug logs in unit test mode were going to the global worker log,
instead of the single-run worker log
*We would barf if the run-specific logs directory already existed
* Was using the wrong variable for operating system options, causing
option conflicts
* Also was trying to get an argument that exists in the config file,
instead!
We would crash trying to get the prefix for the files if our root
directory did not exist (I found this out by accidentally passing the
wrong path for a nightly build). This fixes that problem by deferring if
we don't find any files.
We should probably also just put the source path into the cloner, too,
since that's the only thing that uses it.
Nick Hurley