planetary-computer-apis/README.md

Microsoft Planetary Computer APIs

Note: This repository serves as a reference implementation for deploying APIs on Azure. This code supports the production deployment of the Planetary Computer APIs. This repository is not meant to be reusable in other situations without significant modification, and the repository maintainers will not provide any support for non-development deployments of this code.

That said, feel free to crib any code that is useful!

STAC API + Tiler

This repository contains two components of the Planetary Computer APIs: the STAC API and the Tiler. These are implementations of the open source stac-fastapi and titiler projects. It uses titiler-pgstac to connect the tiler and database.

The pcstac project provides a STAC API which indexes Microsoft's publicly available geospatial data and an API for searching through this large collection. The pctiler provides visualization and data access capabilities for the data in the Planetary Computer.

Azure Functions (a.k.a. Funcs)

This repository also contains Azure Functions that provide additional endpoints for working with Planetary Computer data and metadata. This includes Function endpoints for generating images and animations based on STAC searches, using the tiler to render mosaiced data from Collections.

Collection configuration

See collection config for more on developing collection configurations.

Deployment

This repository hosts the code that is deployed in the Planetary Computer. It contains deployment code for hosting these services in Azure through running the published docker images and Helm charts in Azure Kubernetes Service (AKS), which we used to stand up a development version of the services. The production deployment code is not contained in this repository.

For documentation of how you can deploy your own test version of these services, refer to docs/01-deployment.md.

Development URLs

After building the project locally using the instructions below, you can access the development version of the services by pointing your browser to the following URLs:

STAC API (via nginx)	http://localhost:8080/stac
Tiler (via nginx)	http://localhost:8080/data
Funcs (vai nginx)	http://localhost:8080/f/image, etc..
STAC API (direct)	http://localhost:8081
Tiler (direct)	http://localhost:8082
Funcs (direct)	http://localhost:8083

To see the HTTP endpoints available for FastAPI servers, visit the OpenAPI documentation for each service:

STAC API	http://localhost:8080/stac/docs
Tiler API	http://localhost:8080/data/docs

The development data only includes a single collection naip, with a few items in it. You can verify the data is loaded correctly by visiting the following URL:

http://localhost:8080/stac/collections/naip

Building and Testing Locally

Requirements

The development environment is run almost entirely through docker containers. Developing locally requires docker-compose v1.27+.

Running the Planetary Computer API services in a local development environment

This project uses a variation on scripts to rule them all.

Environment setup and building images

Before setting up the local environment, you must set the AZURITE_ACCOUNT_KEY environment variable to the Azurite default account key, a string that can be found here.

For example, you can set the environment variable in your terminal with:

export AZURITE_ACCOUNT_KEY=<azurite_account_key>

To set up a local environment, use:

./scripts/setup

This will build containers, apply database migrations, and load the development data.

After migrations and development database loading are in place, you can rebuild the docker images with:

./scripts/update

pip dependencies in setup.py are collected and installed through requirements files. If you modify dependencies, run ./scripts/generate-requirements to regenerate requirements-*.txt used by Dockerfiles otherwise your dependency change will not be realized.

Running the services

There is a local proxy service that facilitates a local "managed identity" functionality, run as your local identity. Make sure to run

az login

To run the servers, use

./scripts/server

This will bring up the development database, STAC API, Tiler, Azure Functions, and other services. If at this point something errors out (e.g. nginx complaining about a config file), try deleting the containers/images and rerunning ./scripts/setup.

The STAC API can be found at http://localhost:8080/stac/ (goes through nginx) or http://localhost:8081 directly.

To hit the tiler, try going to http://localhost:8080/data/mosaic/info?collection=naip, although it will fail due to lack of an authorization header.

Testing and and formatting

To run tests, use one of the following (note, you don't need ./scripts/server running). If you get an immediate error related to library stubs, just run it again. The tiler tests may fail locally, TBD why.

./scripts/test
./scripts/test --stac
./scripts/test --tiler
./scripts/test --common

To format code, use

./scripts/format

Changing environments

By default, the stac, tiler, funcs, and supporting services will run against the development containers brought up by scripts/server. It can sometimes be convenient to test against other services, e.g. a test database deployed on Azure. To do that, you can create a new environment file for the services based on ./pc-stac.dev.env, ./pc-tiler.dev.env, and/or ./pc-funcs.dev.env. Any environment file named similarly will be .gitignore'd, so you can leave them in your local clone and avoid committing (e.g. ./pc-stac.testing.env). You then need to set the PC_STAC_ENV_FILE, PC_TILER_ENV_FILE, and PC_FUNCS_ENV_FILE to the environment files you want to use before running scripts/server. Note: Be careful not to run migrations with a non-dev database set - avoid scripts/setup, or ensure the migration connection is still using the local dev database even if using a remote test db.

Published images, charts, and functions

This project publishes images and helm charts, which are used in the deployment of the Planetary Computer.

Images

The following images are hosted in the Microsoft Container Registry:

• mcr.microsoft.com/planetary-computer-apis/stac
• mcr.microsoft.com/planetary-computer-apis/tiler

Only tagged builds will be published to MCR, untagged builds will only be published to the internal ACR pcccr.

Charts

See the Helm chart repository published to GitHub pages for the published charts.

Functions

See the Function package repository published to GitHub pages for the published Azure Functions.