Граф коммитов

74 Коммитов

Автор SHA1 Сообщение Дата
Matt McFarland d2b46c9ae8
Add support and tests for legacy mosaic tile routes (#234)
* Add support and tests for legacy mosaic tile routes

The tile route prefixes were reordered in recent versions of
titiler.pgstac. Both versions of the routes are supported in PC and
tests have been added to confirm.
2024-07-18 12:40:01 -04:00
elayrocks 23b5e6936b
Add subnet of pct sas kubernetes cluster to the storage account that holds ip ban table (#230)
Co-authored-by: elay <yileihu@microsoft.com>
2024-07-02 10:47:02 -07:00
Matt McFarland cfaf0a8992
Function identity and storage network access (#228)
* upgrade to linux app and use identity based storage connection for function app

* add back config

* remove settings for remote build, enable local build

* Set default deny network action on SA

* Add IP to tf state storage firewall

* Add subscription to shared access key setting

* Function debugging in new premium service plan

* Assign function subnet to output storage

* Cleanup

---------

Co-authored-by: elay <yileihu@microsoft.com>
Co-authored-by: Rob Emanuele <rdemaneuele@gmail.com>
2024-07-01 14:37:27 -04:00
elayrocks a08f05e5d1
Azure Function App which runs periodically to aggregate the bytes read per IP address over a window of time (#215)
* init

* update

* update flake8 config

* code format changes

* fix long lines

* fix imports

* function app changes

* code format changes

* test

* remove readme

* remove redundant packages and revert python version

* revert python version

* remove redundant type hints

* use settings class inherited from baseSettings

* change kql

* use placeholder for timer schedule

* change test parameter

* update assertions and logger

* remove import

* remove dash in table name

* update test id

* format

* add no-integration flag

* add dependencies

* test

* change trigger to run every hour

* use azure clients as context manager

* add  context managers in test

* role assignment for function app

* change LAW name

* change role

* change provider's name

* change name of LAW

* better readability

* better readability

* format

* add logging

* UPDATE function settings

* suppress mypy warning

* update env variables for function app

* typo

---------

Co-authored-by: elay <yileihu@microsoft.com>
2024-06-26 08:16:00 -07:00
Matt McFarland 69098c7014
Storage access fixes (#225)
* Use AzureAD auth for terraform backend

Move away from using shared key credentials for the backend auth in both
CI and local dev.

* Test deploy

* CLI + OIDC

* Debug

* Remove debug
2024-06-24 14:42:18 -04:00
Matt McFarland 9beb59b65b
Remove storage account keys from deployment (#224)
* Remove storage account keys from table access

* pcfuncs don't use keys

* Deploy and tests

* Contrib

* Remove temp

* Remove verbosity

* Move settings validation to startup check
2024-06-24 09:50:39 -04:00
Matt McFarland dc022d44b8
Auth against pccomponents for tagged image publish (#217)
* Auth against pccomponents for tagged image publish

* Reconcile node count with deployment
2024-06-11 16:06:04 -04:00
Matt McFarland 5628de76c2
Security upgrades for pcstac and pctiler (#214)
* Upgrade STAC libraries

* Updates for tiler
2024-06-11 13:52:48 -04:00
Matt McFarland 34ebd993c7
Use mcr for nginx-ingress (#213)
Also updates:
- 24 hour image cleaner cycle
- Don't specify k8s version; it's auto upgraded
2024-06-07 14:58:39 -04:00
elayrocks 39dc68add8 test 2024-05-02 12:17:38 -07:00
elayrocks d9f8247026 test 2024-05-02 11:33:16 -07:00
elayrocks cfd385dc01 test 2024-05-02 11:30:00 -07:00
elayrocks b6e42aebfd test 2024-05-02 10:09:37 -07:00
elayrocks 9f9fb83800 test 2024-05-02 10:03:55 -07:00
elayrocks f511643931 try Matt's fix 2024-05-02 09:47:47 -07:00
elayrocks 91fe97de14 test 2024-05-02 00:10:21 -07:00
elayrocks c995a47c28 test 2024-05-01 23:57:39 -07:00
elayrocks bab9c3d369 test 2024-05-01 23:53:23 -07:00
elayrocks 95bfabd873 test 2024-05-01 23:48:57 -07:00
elayrocks a94c00e8a0 test 2024-05-01 23:45:21 -07:00
elayrocks 8ed4dc589d test 2024-05-01 23:41:36 -07:00
elayrocks ee0f20d30d test 2024-05-01 23:29:48 -07:00
elayrocks cfa7afc76a test 2024-05-01 23:23:59 -07:00
elayrocks ea587c08ec test 2024-05-01 23:03:59 -07:00
elayrocks 214fdffce0 test 2024-05-01 23:00:31 -07:00
elayrocks 92ee274b3d test 2024-05-01 22:46:43 -07:00
elayrocks 3e7fc3e1df test 2024-05-01 21:23:32 -07:00
elayrocks ea62afc4a1 test 2024-05-01 19:13:09 -07:00
elayrocks b033600a45 remove secret 2024-05-01 18:59:49 -07:00
elayrocks 2493804f44 test 2024-05-01 18:51:48 -07:00
elayrocks a235a6d26f test 2024-05-01 18:42:48 -07:00
elayrocks 5c91f11425 test 2024-05-01 18:36:56 -07:00
elayrocks 2e424d8b2b test 2024-05-01 18:34:28 -07:00
elayrocks 2c38f89331 remove azlogin 2024-05-01 18:13:12 -07:00
elayrocks eb34587a3f test 2024-05-01 17:53:52 -07:00
elayrocks 0487135933 test 2024-05-01 17:46:13 -07:00
Gustavo Hidalgo 15e99db4b7 Rewrite default docker repository for CI deployment 2024-04-16 15:22:15 -04:00
Gustavo Hidalgo 6887ffc629
Add AKS cluster maintenance window. (#201) 2024-04-12 11:12:08 -04:00
Maitreyee Joshi 877ca1b6c5
use pccomponentstest in cipublish (#200) 2024-04-12 09:55:24 -04:00
Maitreyee Joshi b6213664fc
Change the CI helm chart to reference pcccr (#199)
* update image refs to private pcccr

* switch the unused ACR to pcccr
2024-04-11 15:09:44 -04:00
Matt McFarland 2c23410ce5
Package publish permissions and ICU lib for dotnet functools (#189)
Sets a the write permission for the chart/func package publish jobs, which need to write to the gh-pages branch of the repo. This permission was previously set at the repo level, and may not be adjustable. Also adds a libicu dependency required for func-tools on ubuntu 20.04. This package has an interactive step, so much be installed in noninteractive mode.
2024-03-18 13:16:46 -04:00
Matt McFarland ddd0f3133e
Minimal updates to deployment Dockerfile (#188)
Upgrade to a supported LTS release of Ubuntu and change the install
source of kubectl for compatibility.
2024-03-18 11:49:10 -04:00
Matt McFarland 1282fbb893
Add config/map/token endpoint to support Data Catalog (#187)
* Add map token endpoint to tiler service

The tiler service will generate a token for use against an azure maps
instance, using the identity of the tiler (when deployed) or the local
developer credentials (in local development).

A test has been added that requires a local identity, and this has been
skipped in CI, which does not have access to those kind of credentials.

This endpoint will be used by the Data Catalog app to avoid distributing
an azure maps key within that application.

* Remove unneeded role assignment

* Remove unused variables
2024-03-18 09:44:34 -04:00
Gustavo Hidalgo dcfd7c7666
Use Key Vault secrets instead of cert-manager secret in AKS cluster (#185)
* Update deployment script and add nginx-values.yaml

* Update Dockerfile, values.yaml, variables.tf, and lib script

* Update to satisfy pydantic and mypy

* More mypy induced changes

* More mypy changes
2024-02-05 09:03:07 -05:00
Tom Augspurger 35dd74cee1
Update to Azure Linux (#184) 2023-10-30 10:16:29 -05:00
Martin Cappi Dahl 6083ac7ae3
SA min TLS version (#172) 2023-04-25 11:55:49 -04:00
Rob Emanuele a4f79b41af
CI/CD engagement with the Key Vault Firewall (#161)
* kv firewall ip management

* env for kv firewall management

* Make azlogin executable

* Use CIDR range, ifconfig.me for firewall

* Terraform updates to not force tag change

---------

Co-authored-by: Martin Cappi Dahl <mcappi91@gmail.com>
2023-03-21 15:12:25 -04:00
Matt McFarland aada1bd72e
Upgrade AKS to supported version (#158)
* Upgrades aks to supported version

* Formatting
2023-03-03 20:15:14 -05:00
Matt McFarland 26a67e1266
Support collection-level vector tiles (#147)
* Fetch all configs when iterating over collections

Rather then fetch 1 render config at a time on the /collections
endpoint, fetch all at once and preserve the Dict for the request
duration.

* Allow POST CORS requests in dev env

* Vector tile support

* Add default msft:region attribute to collections

* Upgrade to postgres 14 and pgstac 0.6.13

Prod services operate on pg14

* Fix tests and setup

The API now uses table_service.get_entities and there is an Azurite bug
that prevents an empty string for "all records", so it was switched to a
specific PartitionKey filter string.

* Add logging for pbf requests

* Deployment

* Add logging and debug code

Analyze relative performance of different calls in the VT endpoint
chain.

* Fix Exceptions

* Changelog
2023-01-24 15:22:44 -05:00
Matt McFarland b4912be57c
Fix middleware runtime error and release prep fixes (#130)
* Temporarily use fork for starlette 0.21 release

The 0.21 release resolves a frequent error on our fastapi version.

See:
https://github.com/encode/starlette/pull/1710
https://github.com/encode/starlette/pull/1715

* Disable FTP as function app deploy option

Security controls

* Trace request attributes before invoking middleware

If an exception is raised in subsequent middlewares, added trace
attributes will still be logged to Azure. This allows us to find
requests that fail in the logs.

* Make config cache thread safe

cachetools cache is not thread safe and there were frequent exceptions
logged indicating that cache updates during async calls were failing
with key errors similar to those described in:

https://github.com/tkem/cachetools/issues/80

Add a lock per table instance synchronizes cache updates across threads
in.

* Lint

* Changelog
2022-10-25 10:44:07 -04:00