If we're tiling into FITS format, we have to emulate the funkiness in
the multi-WCS and multi-TAN processors: the data buffers for the
individual tiles need to be flipped vertically. So, copy-paste the code
to do that.
It's helpful to split the study tiling operation into two steps,
preparation and execution, because (1) you can only correctly apply WCS
information after preparation and (2) it's nice to check that the WCS is
all set up correctly before spending the time on the execution step. Here
we add some new APIs to Builder and use them in tile-study
We also improve tile-study to ensure that its image has negative
(JPEG-like) parity, which is required for WWT studies.
In an ideal world, we'd just automatically detect AVM and Do The Right
Thing, but not a lot of people are necessarily going to have pyavm
installed and we don't necessarily want to require it.
Part of this is making the decision to just unilaterally invert the
parity of the WCS that pyavm hands us based on the AVM data. From the
files that I've seen in the wild, this looks to be the correct thing to
do.
The reproject package is pretty memory-hungry, and for large images I
find that it can get killed for memory overuse pretty easily. Here we
mitigate that by reprojecting large images in horizontal bands, AKA
"chunks". This can reduce the memory consumption significantly, and for
small images the behavior is equivalent to what we had before.
Some FITS files have additional axes of size 1. We can work with them just fine, but we need
to strip out the non-celestial axes to make the rest of our machinery happy.
Haven't touched this code in a while, but WWT can now display tiled FITS
so we're about to use it a lot more! The new code has to be careful
about the parities of FITS vs. tilings but I believe that we're doing
that all correctly now. We now also emit WTML using the standard
framework and otherwise modernize this code.
No parallel implementation yet, just trying to tidy up the serial
approach first.
To render tiled FITS, things are a bit awkward because the individual
FITS files are rendered by WWT assuming a bottoms-up (positive parity)
data layout, but the overall tiling coordinate system is still top-down.
If you work through the geometry, the cascade process can be nearly
identical for both this case and the case where the tiles are also
top-down: all you need to change is to reorder the vertical stacking in
the cascade process. So, do that. In order for this to work, we need to
add some infrastructure to "know" the expected vertical data layout of
the pyramid tiles.
If you apply WCS info before setting up the imageset tiling parameters,
the info will be wrong. Add some checks to try and catch this. These
also revealed that tile-wwtl was doing the tiling twice!
In wwt_data_formats 0.10 we'll fix up our WCS processing to correctly
deal with different image parities. This exposes the fact that AstroPix
and Djangoplicity are returning data that, if interpreted directly into
WCS, are a bit wrong. Here we account for that. More work will need to
be done to tighten up the parity handling in toasty overall.
We had a mechanism to have certain images be marked as ignored, but
there wasn't a way for users to do any marking! This new command can
mark all rejected images to be ignored in the future.
Peter Williams
A. David Weigel