* init
* update
* update flake8 config
* code format changes
* fix long lines
* fix imports
* function app changes
* code format changes
* test
* remove readme
* remove redundant packages and revert python version
* revert python version
* remove redundant type hints
* use settings class inherited from baseSettings
* change kql
* use placeholder for timer schedule
* change test parameter
* update assertions and logger
* remove import
* remove dash in table name
* update test id
* format
* add no-integration flag
* add dependencies
* test
* change trigger to run every hour
* use azure clients as context manager
* add context managers in test
* role assignment for function app
* change LAW name
* change role
* change provider's name
* change name of LAW
* better readability
* better readability
* format
* add logging
* UPDATE function settings
* suppress mypy warning
* update env variables for function app
* typo
---------
Co-authored-by: elay <yileihu@microsoft.com>
* Set root-path from uvicorn
Sets the root path in uvicorn to match that set via FastAPI. This is
required with updates to starlette, which were included previously.
Fixes https://github.com/microsoft/PlanetaryComputer/issues/360
* New black formatting
* Add map token endpoint to tiler service
The tiler service will generate a token for use against an azure maps
instance, using the identity of the tiler (when deployed) or the local
developer credentials (in local development).
A test has been added that requires a local identity, and this has been
skipped in CI, which does not have access to those kind of credentials.
This endpoint will be used by the Data Catalog app to avoid distributing
an azure maps key within that application.
* Remove unneeded role assignment
* Remove unused variables
* Upgrade pgstac to use db-based queryables
Upgrades to stac-fastapi and pgstac to make use of the pgstac.queryables
table being the source of API queryable endpoints.
* Changelog
Doesn't include features and item-search validation, but includes comments in
the validation script saying why.
Co-authored-by: Matt McFarland <mmcfarland@microsoft.com>
* Add with_backoff to funclib tile fetching
This adds an aync version of with_backoff and
uses it in tile fetching to retry on errors,
which can prevent gray blotches in images and animations
* Refactor stamps to apply ot Images
This change generalizes the stamps framework
to allow for stamps that don't require an
AnimationFrame in anticipation of using the
stamp logic with the image endpoint.
* Add optional geometry param to image request
* Enable pytest console testing for pcfuncs
* Fixes to RenderOptions
* Enable logo stamp on image exports
* Improvements to with_backoff_async
* Refactor to consolidate exporters
This commit further abstracts common logic
between image and animation, and refactors
the codebase to use the generalized
functionality.
* Bound type for class to avoid mypy error
Co-authored-by: Matt McFarland <mmcfarland@microsoft.com>
* Setup local and deployed funcs to use CORS
* nginx proxy facilitates CORS headers locally
* azurite container set to public access to local frontend can access
animation urls
* configure func apps deployed with CORS enabled
* Progress bar and branding options for animations
* Add animation hints for collection config
Optional better-than-default settings for animation frames, per
collection.
* Changelog
* Add logo stamp for animation endpoint
* Move stamps from animation to funclib
Stamps will get consumed by image endpoint in the future.
* Consolidate dev dependencies into requirements-dev.txt
* Add simple azure storage support to pccommon
* Set up azurite container for dev output
* Add pcfuncs codebase
This includes the "animation" and "image" functions,
as well as the "funclib" library that contains
reusable code for fetching rasters from the
PC data API.
* Add pcfuncs to dev servers and processes
* Add Azure Functions to deployment
* Move funcs information to main README
* Fix titiler link
* Avoid flake8 on setup.py
* Delete unused reprojection utility method/type
* Remove debug log information
* Don't restrict concurrency setting
* Add script for building queryable schemas
* Use stac-dev container to build queryables
* Add readme entry and update format script
* Add default title and improved queryable property keys
Co-authored-by: Matt McFarland <mmcfarland@microsoft.com>
* Add redis to docker-compose services
* Refactor get_request_ip into utils
* Rename TABLE_TTL -> TTL, will use it with caching
* Add redis caching and rate limiting to pccommon
* Add rate limit configuration
* Caching and rate limiting on STAC API
* Add redis to terraform
* Add redis config to helm charts
* Connect to redis for tests
* Add test for rate limit, but skip as it is nondeterministic
* Update CHANGELOG
* Implement backpressure
* Add backpressure to Helm chart
* Use decorators for rate_limit and back_pressure
* Add IP exception table that avoids rate limiting
* Get the IP from the last in the list, not first
If using an X-Forwarded-For to get the IP,
get the last one as that will be the IP coming from
the last proxy. The first IP can be anything set on
the header of the request. Since we take the
X-Azure-ClientIP header value first, this would
have not been used, but changing for best practice.
* And IP exception table to Helm chart
* Allow traffic from planetarycomputer-test
* Add ip-exception config to tiler docker-compose
Co-authored-by: Matt McFarland <mmcfarland@microsoft.com>
* Add additional dependencies to pccommon
* Fix mypy error in pctiler
Brining in the type stubs for cachetools caused
mypy to complain about unknown types for the key
function
* Refactor scripts to test pccommon
Also run flake8 on pccommon,
which wasn't happening
* Linting fixups
* Add tables classes
* Refactor collection config in pccommon
Also refactor CommonConfig to use pydantic settings.
Create a table setup for collection configuration
and container configuration.
Use cachetools to cache the configuration.
* Add Azurite setup
Encode collection configuration and container
configuration (which was hardcoded) as JSON.
This can be used to populate the initial
table structure in deployment as well, after
which this test data will diverge from production
settings.
* Update codebase to use refactored configuration
* Set azurite settings in docker-compose
Also account for environment prefix for DEBUG
that change with refactor to use BaseSettings in
CommonConfig
* Move to using only pytest for consistency
* Test get render config for naip
* Refactor config code layout
Enable configuration of TTL
* Run azurite setup in scripts/setup
Also fix setup_azurite
* Add mosaicInfo and queriables to collection config
* Remove usage of requirements.txt
This was being used inconsistently.
* Add script for local package install
* Fetch queryables from storage tables
* Use orjson in pccommon
* Use ORJSONResponse
* Remove unused endpoint prefixes
* Add mosiac/info endpoint
* Add method to fetch all rows
* Add CLI for loading and dumping config data
* Variable for k8s version; update dev
* Allow AKS to pull from ACR
* Storage Tables in terraform
* Add config table env vars to helm charts
* Update ingress apiVersion
* Make note in deploy README about updating tables
* Update CHANGELOG
* Linting/formatting
* Remove unused __init__ override
This was left over from a previous
implementation, should have been cleaned up.
* Remove unused vars in dev terraform
* Allow cli to dump configs by id
Co-authored-by: Matt McFarland <mmcfarland@microsoft.com>
* gnatsgo render config
* Override response limit for GET requests
Restores PC default limit of 250 items in response of GET searches and
collection item path queries.
* Update changelog
* Keep /items endpoint at default limit of 10
* Change gnatsgo collection ID to gnatsgo-rasters
* Add ability to scope test run
* Add test for search and /items page limit
* Update gnatsgo asset name
Co-authored-by: Rob Emanuele <rdemanuele@gmail.com>
* Add dedicated health check endpoint for pctiler
Also sets the liveness probe to use this new endpoint. Uses the same URL
path as pcstac, this will help isolate requests in the logs.
* Use constants for logging service name
* Fix local volume mount
The directory is copied in the Dockerfile, so it was loading on the
container, however this typo prevented reloading of pccommon when
changes were made without rebuilding the image.
* Consolidate tracing functionality
Rather than maintaining parity between two request tracing
implementations, create a common trace that can be used by both
projects.
* Prevent request tracing on health check endpoints
* Parse collection/item ids from search for logging
* Allow ACR overrides in dev deployments
Defaults to publish images for staging deploys
* Rename for Python convention
* Include request ip in trace
* Remove unused config
* Use request sensitive middleware for trace logs
Traditional middleware will corrupt usage of starlette request objects
by downstream route functions. Use a middleware class crafted for
accessing the request body without interfering with further processing.
* Lint: formatting
Auto-formatter
* Add helm chart publication
* Remove old cipublish path
* switch from "1" to "true"
GitHub sets CI to "true" rather than 1:
https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables
* wip -- attempt the run
* don't condition deploy step for now
* switch branch back to main
youc an use $default-branch in templates, but not
actual workflows:
https://github.blog/changelog/2020-07-22-github-actions-better-support-for-alternative-default-branch-names/
* switch branch back to main
youc an use $default-branch in templates, but not
actual workflows:
https://github.blog/changelog/2020-07-22-github-actions-better-support-for-alternative-default-branch-names/
* set azure env variables
* parse and echo from json
* don't set tenant id?
https://github.com/hashicorp/terraform-provider-azuread/issues/343#issuecomment-721455149
??
* oops i don't think these are getting passed through
* add override compose
this compose file un-sets the env variables that we don't have because
we don't have a .env file in ci, so I _think_ it should inherit them
from the environment
* add back mqe resource group
why was this deleted
am i being trolled
* Add path debugging
* check what tf keys we got
* remove api management from ingress 🤞🏻
* all container registries through github
* remove debug prints from jinja
* re-remove the mqe resource group
because it was deliberately missing, not accidentally
* skip tests for a sec
* debug echo cluster name and rg
* require az account env variables
* reuse tf env variable names
* don't point to `latest` tag
* Add workflow for publishing helm charts to GH Pages.
* Rename charts, separate published, reset versions
* Use -dev suffix for development release
* Publish dev charts only on main
* Rename mqe -> stac, dqe -> tiler in codebase
* Update README
Rename MQE to STAC API and DQE to Tiler; also editorally make things more terse
* Remove testdata, move loadtestdata to tests
* Remove out of date and generic docs for tiler
* Use cipublish to publish images
* Rename python packages to be prefixed with 'pc'
* Remove stac-vrt
* Remove unused model and method.
Also formatting
* Remove scripts/env
* Upgrade to stac-fastapi 2.2.0
* Delete unusued doc images
* Update deployment code with renames.
* Update cert-manager, other deployment fixes
* Test on cibuild, remove GA test branch trigger
* Fixup PR template
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: James Santucci <james.santucci@gmail.com>
elayrocks
Matt McFarland
Gustavo Hidalgo
Maitreyee Joshi
Pete Gadomski
Rob Emanuele
Nathan Zimmerman