diff --git a/CHANGES.md b/CHANGES.md index 7cf4704..212e92d 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,4 +1,4 @@ -# 0.1.0dev0 (unreleased) +# 0.1.0.dev0 (unreleased) - No changes yet diff --git a/README.md b/README.md index 5a4dd34..1150ec1 100644 --- a/README.md +++ b/README.md @@ -6,12 +6,11 @@ [toasty] is a Python library that helps you create [TOAST] “tile pyramids” -from [FITS] astronomical image files. These multi-resolution all-sky maps can -be viewed in software such as the [AAS] [WorldWide Telescope]. +from astronomical image data. These multi-resolution all-sky maps can be +viewed in software such as the [AAS] [WorldWide Telescope]. [toasty]: https://toasty.readthedocs.io/ [TOAST]: https://doi.org/10.3847/1538-4365/aaf79e -[FITS]: https://en.wikipedia.org/wiki/FITS [AAS]: https://aas.org/ [WorldWide Telescope]: http://www.worldwidetelescope.org/ @@ -30,7 +29,37 @@ The easiest way to install [toasty] is through [pip]: pip install toasty ``` -[pip]: https://pypi.org/project/pip/ +[pip]: https://pip.pypa.io/ + +For more information, please see the full [toasty installation instructions]. + +[toasty installation instructions]: https://toasty.readthedocs.io/en/latest/installation.html + + +## Documentation, Examples, etc. + +For documentation and examples, go to: + +https://toasty.readthedocs.io/ + + +## Contributions + +Contributions to [toasty] are welcome! See +[the WorldWide Telescope contributors’ guide] for applicable information. We +use a standard workflow with issues and pull requests. All participants in +[toasty] and the WWT communities must abide by the [WWT Code of Conduct]. + +[the WorldWide Telescope contributors’ guide]: https://worldwidetelescope.github.io/contributing/ +[WWT Code of Conduct]: https://worldwidetelescope.github.io/code-of-conduct/ + + +## Release History + +Releases of [toasty] are logged in the file [CHANGES.md](./CHANGES.md), as +well as release listings maintained by +[GitHub](https://github.com/WorldWideTelescope/toasty/releases) and +[PyPI](https://pypi.org/project/toasty/#history). ## Dependencies @@ -53,137 +82,12 @@ pip install toasty [pytest]: https://docs.pytest.org/ -### Usage +## Legalities -```python -from toasty import toast - -toast(data_sampler, depth, base_dir, wtml_file=None, merge=True, base_level_only=False, - ra_range=None, dec_range=None, toast_tile=None, restart=False, top_layer=0) -``` - -where: - - * **data_sampler** function or string - - A function of (lon, lat) that samples a datasetat the input 2D coordinate arrays - - A string giving a base toast directory that contains the base level of toasted tiles, using this option, only the merge step takes place, the given directory must contain a "depth" directory for the given depth parameter - * **depth** int - - The depth of the tile pyramid to create (4^d tiles are created at a depth of d) - * **base_dir** str - - The path to create the files at - * **wtml_file** str (optional) - - The path to write a WTML file to. If not present, no file will be written - * **merge** bool or callable (default True) - How to treat lower resolution tiles. - - If True, tiles above the lowest level (highest resolution) will be computed by averaging and downsampling the 4 subtiles. - - If False, sampler will be called explicitly for all tiles - - If a callable object, this object will be passed the 4x oversampled image to downsample - * **base_level_only** bool (default False) - If True only the bottem level of tiles will be created. - In this case merge will be set to True, but no merging will happen, and only the highest resolution layer of images will be created. - * **ra_range** array (optional) - * **dec_range** array (optional) - To toast only a portion of the sky give min and max ras and decs ([minRA,maxRA],[minDec,maxDec]) in degrees. - If these keywords are used base_level_only will be automatically set to true, regardless of its given value. - * **toast_tile** array\[n,x,y\] (optional) - If this keyword is used the output will be all the subtiles of toast_tile at the given depth (base_level_only will be automatically set to true, regardless of its given value. - * **top_layer** int (optional) - If merging this indicates the uppermost layer to be created. +[toasty] is copyright Chris Beaumont, Clara Brasseur, and the AAS WorldWide +Telescope Team. It is licensed under the MIT License. +## Acknowledgments - -Toasty provides a few basic sampler functions: - - * **healpix_sampler** for sampling from healpix arrays - * **cartesian_sampler** for sampling from cartesian-projections - * **normalizer** for applying an intensity normalization after sampling - -### Examples - -To toast an all-sky, Cartesian projection, 8 byte image: - -```python -from toasty import toast, cartesian_sampler -from skimage.io import imread - -data = imread('allsky.png') -sampler = cartesian_sampler(data) -toastDirectory = 'toast' -depth = 8 # approximately 0.165"/pixel at highest resolution - -# Getting the full toast tile set for the entire sky -toast(sampler, depth, toastDirectory) - -# Toasting a specific region of the sky defined by ra/dec bounds -raRange = [208.8,212.2] -decRange = [52.5,56.8] -toast(sampler, depth, outtoastDirectoryput, ra_range=raRange, dec_range=decRange) - -# Toasting a specific region of the sky defined by a higher level toast tile -tile=[4,5,9] -toast(sampler, depth, toastDirectory, toast_tile=tile) - -# Creating only the bottom layer of toast tiles -toast(sampler, depth, toastDirectory,base_level_only=True) - -# Merging from a previously created toast layer up to a specified top layer -topLayer = 4 -toast(toastDirectory, depth, toastDirectory, top_layer=topLayer) -``` - -To apply a log-stretch to an all sky FITS image: - -```python -from toasty import toast, cartesian_sampler, normalizer -from astropy.io import fits - -data = fits.open('allsky.fits')[0].data -vmin, vmax = 100, 65535 -scaling = 'log' -contrast = 1 -sampler = normalizer(cartesian_sampler(data), vmin, vmax - scaling, bias, contrast) -output = 'toast' -depth = 8 -toast(sampler, depth, output) -``` - -To perform a custom transformation - -```python -from toasty import toast -from astropy.io import fits - -data = fits.open('allsky.fits')[0].data - -def sampler(x, y): - """ - x and y are arrays, giving the RA/Dec centers - (in radians) for each pixel to extract - """ - ... code to extract a tile from `data` here ... - -output = 'toast' -depth = 8 -toast(sampler, depth, output) -``` - -See ``toasty.tile`` for documentation on these functions. - -### Using with WorldWide Telescope -To quickly preview a toast directory named `test`, navigate to the directory -where `test` exists and run - -``` -python -m toasty.viewer test -``` - -This will start a web server, probably at [http://0.0.0.0:8000](http://0.0.0:8000) (check the output for the actual address). Open this URL in a browser to get a quick look at the data. - -For more information about using WorldWide Telescope with custom image data, -see [the official documentation](http://www.worldwidetelescope.org/Docs/worldwidetelescopedatafilesreference.html). The function `toasty.gen_wtml` can generate the wtml information for images generated with toasty. - -For an example of tiles generated with Toasty, see [The ADS All Sky Survey](http://adsass.org/wwt). The code used to generate these images is [here](https://github.com/ChrisBeaumont/adsass/blob/master/toast/toast.py). - -For an example of tiles generated using the Toasty functionality specific to this fork see [The STScI AstroView Panstarrs Survey](https://mast.stsci.edu/portal/Mashup/Clients/AstroView/AstroView.html?debug&avSurveyType=PANSTARRS). The code used to generate these images is [here](https://github.com/ceb8/toastPanstarrs). +Work on [toasty] is funded through the [AAS] [WorldWide Telescope] project. diff --git a/RELEASE_PROCESS.md b/RELEASE_PROCESS.md index eb407c4..308a5b0 100644 --- a/RELEASE_PROCESS.md +++ b/RELEASE_PROCESS.md @@ -9,15 +9,16 @@ These are notes for the `toasty` developers about how to create a new release. 5. For the final commit, update the version number in `setup.py` and `docs/conf.py`, and add a proper version and date to `CHANGES.md`. Commit with message `Release version ${version}`. -6. Create a pull request on GitHub for the new release. +6. Push to GitHub and create a pull request for the new release called + "Release PR for version $version". 7. Get it so that it passes CI, creating fixup commits as necessary. -8. When it's really really ready, `git clean -fxd && python setup.py sdist - bdist_wheel --universal && twine upload dist/*.tar.gz dist/*.whl`. If - `twine` finds problems, make any final changes and retry. +8. When it's really really ready, `git clean -fxd && python setup.py sdist && + twine upload dist/*.tar.gz`. If `twine` finds problems, make any final + changes and retry. 9. If needed, do a `git rebase -i` to make the version-bump commit the last one again. 10. `git tag v${version}` -11. Update the version number to `${cur_major}.${next_minor}.0dev0` and add a +11. Update the version number to `${cur_major}.${next_minor}.0.dev0` and add a new separator in `CHANGES.md` along the lines of `${version} (unreleased)`. Commit with a message of `Back to development.` 12. `git push` (with `-f` if history was rewritten) to the PR branch. This had diff --git a/docs/conf.py b/docs/conf.py index 2d17aec..8b047c7 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -4,8 +4,8 @@ project = 'toasty' author = 'Chris Beaumont and the AAS WorldWide Telescope Team' copyright = '2014-2019, ' + author -release = '0.1.0dev0' # the full version string; also update ../setup.py -version = '0.1.0dev0' # the "short" version +release = '0.1.0.dev0' # the full version string; also update ../setup.py +version = '0.1.0.dev0' # the "short" version extensions = [ 'sphinx.ext.autodoc', diff --git a/docs/examples.rst b/docs/examples.rst new file mode 100644 index 0000000..85c568c --- /dev/null +++ b/docs/examples.rst @@ -0,0 +1,137 @@ +============================= +Examples of How to Use Toasty +============================= + +Here we’ll summarize some of the ways that you can use toasty_. + +.. _toasty: https://toasty.readthedocs.io/ + + +Toasting a Cartesian, all-sky PNG image +======================================= + +This script processes an existing all-sky PNG image that uses a Cartesian +projection, using scikit-image_ to load the data:: + + from toasty import toast, cartesian_sampler + from skimage.io import imread + + data = imread('allsky.png') + sampler = cartesian_sampler(data) + output_directory = 'toast' + depth = 8 # approximately 0.165"/pixel at highest resolution + + # Make the full toast tile set for the entire sky: + toast(sampler, depth, output_directory) + +.. _scikit-image: https://scikit-image.org/ + + +Toasting subsets of the sky +=========================== + +You don’t have to build the full pyramid for the full sky all at once:: + + # Toast a specific region of the sky defined by RA/Dec bounds + toast(sampler, depth, output_directory, + ra_range = [208.8, 212.2], # degrees + dec_range = [52.5, 56.8], # degrees + ) + + # Toast a specific region of the sky defined by a higher-level TOAST tile + toast(sampler, depth, output_directory, + toast_tile = [4, 5, 9], + ) + + # Create only the bottom layer of toast tiles + toast(sampler, depth, output_directory, + base_level_only = True, + ) + + # Merge from a previously created toast layer up to a specified layer + toast(sampler, depth, output_directory, + top_layer = 4, + ) + + +Controlling how data are turned into RGB +======================================== + +Here we apply a log-stretch to an all sky FITS image:: + + from toasty import toast, cartesian_sampler, normalizer + from astropy.io import fits + + data = fits.open('allsky.fits')[0].data + + vmin, vmax = 100, 65535 + scaling = 'log' + contrast = 1 + sampler = normalizer( + cartesian_sampler(data), + vmin, vmax + scaling, bias, contrast + ) + + output_directory = 'toast' + depth = 8 + toast(sampler, depth, output_directory) + + +Non-Cartesian coordinate transformations +======================================== + +A custom “sampler” can be used to tell toasty_ what image values +correspond to what locations on the sky:: + + from toasty import toast + + def sampler(x, y): + """ + x and y are arrays, giving the RA/Dec centers + (in radians) for each pixel to extract + """ + ... code to produce pixel values goes here ... + + output_directory = 'toast' + depth = 8 + toast(sampler, depth, output_directory) + +See :meth:`toasty.tile` for documentation on sampler functions. + + +Previewing toasts with AAS WorldWide Telescope +============================================== + +To quickly preview a toast directory named ``mytoast``, run the command:: + + python -m toasty.viewer mytoast + +This will start a web server, probably at ``http://0.0.0.0:8000``. (Check the +program’s output for the actual address). Open this URL in a browser to get a +quick look at the data. + + +Example toasty outputs: ADS All-Sky Survey +========================================== + +For an example of tiles generated with Toasty, see `the ADS All Sky Survey +`_. The code used to generate these images is in `the +file toast.py`_ in the `adsass/wwt-frontend`_ repository on GitHub. This tile +pyramid was created by Chris Beaumont. + +.. _the file toast.py: https://github.com/adsass/wwt-frontend/blob/master/toast/toast.py +.. _adsass/wwt-frontend: https://github.com/adsass/wwt-frontend/ + + +Example toasty outputs: PanSTARRS in STScI AstroView +==================================================== + +Another example is a toasting of the “3π” survey of the PanSTARRS_ project, +viewable `here in STScI’s AstroView`_ interface. The code used to generate +these images is in the GitHub repository `ceb8/toastPanstarrs`_. This tile +pyramid was created by Clara Brasseur. + +.. _PanSTARRS: https://panstarrs.stsci.edu/ +.. _here in STScI’s AstroView: https://mast.stsci.edu/portal/Mashup/Clients/AstroView/AstroView.html?debug&avSurveyType=PANSTARRS +.. _ceb8/toastPanstarrs: https://github.com/ceb8/toastPanstarrs diff --git a/docs/index.rst b/docs/index.rst index 1de7e54..f4cdf3a 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,10 +2,15 @@ toasty: Creating TOAST tile pyramids in Python ============================================== -The ``toasty`` package helps you create `TOAST -`_ tile pyramids that can be used in -software such as the `AAS `_ `WorldWide Telescope -`_. +toasty_ is a Python library that helps you create TOAST_ “tile pyramids” +from astronomical image data. These multi-resolution all-sky maps can +be viewed in software such as the AAS_ `WorldWide Telescope`_. + +.. _toasty: https://toasty.readthedocs.io/ +.. _TOAST: https://arxiv.org/abs/1812.03926 +.. _AAS: https://aas.org/ +.. _WorldWide Telescope: http://www.worldwidetelescope.org/home + User guide ========== @@ -13,19 +18,22 @@ User guide .. toctree:: :maxdepth: 1 + overview + installation + examples api Getting help ============ -If you run into any issues when using ``toasty``, please open an issue `on its +If you run into any issues when using toasty_, please open an issue `on its GitHub repository `_. Acknowledgments =============== -Work on ``toasty`` is funded through the American Astronomical Society -(`AAS`_) `WorldWide Telescope`_ project. The first version of ``toasty`` was +Work on toasty_ is funded through the American Astronomical Society +(`AAS`_) `WorldWide Telescope`_ project. The first version of toasty_ was written by Chris Beaumont. diff --git a/docs/installation.rst b/docs/installation.rst new file mode 100644 index 0000000..79edb28 --- /dev/null +++ b/docs/installation.rst @@ -0,0 +1,44 @@ +================= +Installing toasty +================= + +Installing toasty with pip +========================== + +You can also install the latest release of toasty using pip_:: + + pip install toasty + +.. _pip: https://pip.pypa.io/en/stable/ + + +Dependencies +============ + +If you install toasty using pip_ as described above, any required dependencies +will get installed automatically. The `README in the Git repository`_ lists the +current dependencies if you would like to see an explict list. + +.. _README in the Git repository: https://github.com/WorldWideTelescope/toasty/#readme + + +Installing the developer version +================================ + +If you want to use the very latest developer version, you should clone `this +repository `_ and manually +install the package in “editable” mode:: + + git clone https://github.com/WorldWideTelescope/toasty.git + cd toasty + pip install -e . + +You will need to have Numpy_ and Cython_ installed on order to build the extension +module that toasty uses to speed up its core operations. + +.. _Numpy: https://cython.org/ +.. _Cython: https://numpy.org/ + +You can run the test suite with the command:: + + pytest toasty diff --git a/docs/overview.rst b/docs/overview.rst new file mode 100644 index 0000000..f594a69 --- /dev/null +++ b/docs/overview.rst @@ -0,0 +1,101 @@ +=========================== +Overview: What is Toasting? +=========================== + +So, toasty_ is for toasting. What does that even mean? + +.. _toasty: https://toasty.readthedocs.io/ + +This documentation will *not* go into detail because there are plans to +provide comprehensive information in a separate document that is not specific +to this particular library. + +Alas, that documentation has not been written yet. In the meantime, the best +detailed reference is `McGlynn et al. 2019`_, which explains and justifies the +Tesselated Octahedral Adaptive Spherical Transformation (TOAST) projection +that toasty_ is concerned with. See also the `WorldWide Telescope Projection +Reference`_, although that document is a bit out of date. (Which shouldn’t +matter, in principle, but its current Web expression may be a bit busted.) The +following text aims to give a quick overview of the concepts that these +documents work out in detail. + +.. _McGlynn et al. 2019: https://ui.adsabs.harvard.edu/abs/2019ApJS..240...22M/abstract +.. _WorldWide Telescope Projection Reference: https://worldwidetelescope.gitbook.io/projection-reference/ + + +The problem(s) +============== + +Say that you are writing a piece of software that aim to let users +interactively explore a map of the Earth, the sky, or another planet. For the +purposes of this discussion, the key thing is that these are all spherical +entities. The user may want to zoom out and view large swathes of these +spheres at once, or they may want to zoom way in and see extremely fine +detail. The user will probably be accessing the maps over a network, and the +maps may be far too large to transmit in over the network in their entirety. + +There are two problems to solve here. First, we need some way to store image +data covering an entire sphere. There is no unique best way to map the curved +surface of a sphere into a two-dimensional representation; in particular, many +common projections perform poorly at the poles. + +Second, once we have some image data that we can map onto a sphere in some +satisfactory way, we need a way to transmit pieces of it to the user +incrementally. If they zoom out, it won't be practical to send them a huge +chunk of the full-resolution map. + + +The solution(s) +=============== + +Our solution to the above problems is to construct “tile pyramids” using the +TOAST spherical projection. + +The TOAST projection does exactly what we need for the first problem: it maps +the entire sphere onto a 2D image that can easily be represented digitally. +While TOAST is not perfect in every way, it performs well at the poles and +maintains approximately uniform resolution at all locations on the sphere. If +you’re familiar with this topic, you may know that the HEALPix_ projection +also operates in this problem space. We won’t go into details here, but +suffice it to say that TOAST has some nice technical properties that make it +the preferred choice for software such as the AAS_ `WorldWide Telescope`_. In +particular, TOAST maps the sphere onto a square image, rather than the jagged +shape required by HEALPix. + +.. _HEALPix: https://healpix.jpl.nasa.gov/ +.. _AAS: https://aas.org/ +.. _WorldWide Telescope: http://www.worldwidetelescope.org/home + +A high-resolution full-sphere TOAST map may weigh in at a million pixels *on a +side*, or a trillion pixels total — far too large to manipulate in a +user-facing scenario. Creating a “tile pyramid” of the image makes it possible +to actually do useful things with a full-scale TOAST map. First, the +high-resolution map is broken into a set of square sub-images, “tiles,” each +256 pixels on a side. By the requirements of the TOAST standard, the number of +tiles on each side must be a power of 2. Then, lower-resolution maps are +created by downsampling the high-resolution map in 2×2 pixel blocks, yielding +derived images that are half as large as their parents along each axis. (Each +downsampled image therefore has 1/4 as many pixels as its parent.) By +construction, each of these lower-resolution maps can also be broken into +256×256 tiles. This process continues until a “level 0” map is created +consisting of a single tile that represents the full sphere. With a full +sphere having a solid angle of 4π steradians, each pixel in this level-0 map +covers about 0.0002 steradians or 0.6 square degrees. As a user zooms in or +out, or pans around the sphere, their computer is sent the tiles needed to +produce a visually acceptable view. Each individual tile file is small enough +that the data transfer remains tractable. + + +The role of toasty +================== + +The toasty_ module helps create these TOAST tile pyramids from astronomical +image data. There are essentially three problems that it tackles: + +1. Transforming image data from their native projection to the TOAST one. +2. Mapping scalar data values into RGB color images for user display. +3. Downsampling the high-resolution TOAST map all the way down to the level-0 + map. + +The process of creating TOAST tile pyramids from some input data is +colloquially referred to as “toasting.” diff --git a/setup.py b/setup.py index bff4d23..639c803 100644 --- a/setup.py +++ b/setup.py @@ -34,8 +34,8 @@ project homepage]. setup_args = dict( name = 'toasty', - version = '0.1.0dev0', # also update docs/conf.py - description = 'Generate TOAST image tile pyramids from FITS files', + version = '0.1.0.dev0', # also update docs/conf.py + description = 'Generate TOAST image tile pyramids from existing image data', long_description = get_long_desc(), long_description_content_type = 'text/markdown', url = 'https://toasty.readthedocs.io/',