Граф коммитов

66 Коммитов

Автор SHA1 Сообщение Дата
Tamir Kamara 28693097cd
Upgrade Porter to v1 (#3014)
* Porter v1 core changes (#2977)
* Update templates for Porter v1 (#2985)
* Update docs for porter v1 (#2992)
2022-12-26 13:14:53 +00:00
Marcus Robinson d692717db7
Upgrade all docker base images to bullseye (#2946)
* Upgrade all docker base images to bullseye

* Up RP version

* remove jq version

* Remvoe buster comments

* Revert cli version and update changelog

* add pinned packages back.

* Update _version.py

* Update _version.py

Co-authored-by: Tamir Kamara <26870601+tamirkamara@users.noreply.github.com>
2022-12-08 18:13:53 +00:00
Tamir Kamara 0582086bfb
Update python dependencies (#2907)
* update python dependencies

* versions

* fix pyetst asyncio

* lint bash

* comment check in a test
2022-11-27 16:58:24 +00:00
Tamir Kamara 0d75382e05
Remove docker mixin (#2889)
* update changelog

* fix

* remove docker mixin

* update scripts

* changelog
2022-11-17 14:57:46 +00:00
Tamir Kamara 17492e10bb
Pin porter components versions (#2762) 2022-10-23 10:35:09 +00:00
Stuart Leeks 04ef8994c9
Add support for custom VM images in Guacamole user resources (#2634)
* Update RP deployment to take RP_BUNDLE values
* Update RP to load RP_BUNDLE_ vars and use in bundle params
* Update user-resource templates
2022-10-07 05:44:38 +01:00
David Moore dae21edce9
Re-working Status Update Queue Message Handling (#2700)
* sessions

* refactored and tests WIP

* tests refactored

* api v + changelog

* rp version

* core version!

* api version

* DeploymentStatusUpdate -> DeploymentStatusUpdater

* daemon thread
2022-10-06 09:12:20 +00:00
Tamir Kamara 1433311d8a
Upgrade Terraform Mixing to 1.0.0-rc.1 (#2677)
* terraform init in core/ops shouldn't use -upgrade

* update terraform in devcontainer

* always refresh

* tflint

* update changelog

* Remove doker env variables

* update changelog

* align bundles with new terraform mixin

* update changelog
2022-10-02 12:29:00 +00:00
Marcus Robinson af8dbaaf13
Fix AML when no users assigned a role, and upgrade Porter and TF mixin (#2639) 2022-09-22 22:00:07 +00:00
Tamir Kamara 4f13752f16
rp changes for porter show (#2542)
* rp changes for porter show

* update changelog
2022-09-05 11:31:08 +03:00
Stuart Leeks 60bc98c041
Unpin curl etc for resource processor (#2533)
* Unpin curl etc for resource processor
2022-08-31 11:04:15 +01:00
Tamir Kamara 513ddabb6c
Log analytics extension in VM/VMSS (#2520)
* LA in core VM/VMSS

* fix bug in RP logging

* update changelog
2022-08-28 11:16:38 +03:00
Tamir Kamara bd9396b078
Upgrade Terraform, AzureRM, Porter (#2501)
* update lock file script

* update lock files

* update timeout for gitea shared service e2e

* upgrade tf, tf-azurerm, porter

* native terraform vnet integration

* dump version
2022-08-24 19:12:54 +03:00
Tamir Kamara 9ea7664679
Update Resource Processor logging and disk size (#2486)
* fix resource processor logging severity

* enlarge RP disk to cache bundles longer

* bump core version
2022-08-21 10:19:35 +03:00
David Moore d3cd62b7bd
Collection of Pipeline Bugs (#2475)
* try except around getting azure status

* catch validation errors and prevent pipeline from hanging forever

* update primary resource on failure of pipeline

* strip string escape chars from outputs as they're passed back to cosmos

* versions
2022-08-17 17:39:06 +01:00
Tamir Kamara 498ec31079
release 0.4 version (#2356) 2022-07-27 14:02:58 +03:00
Tamir Kamara eba6764440
Resource processor checks its runner processes (#2329)
* cloud_RoleName for api

* cloud_RoleName for Resource Processor

* fix lint

* rp checks its runner processes
2022-07-25 19:34:34 +03:00
Tamir Kamara cf309e8626
Core apps use proper cloud_RoleName values (#2324)
* cloud_RoleName for api

* cloud_RoleName for Resource Processor

* fix lint

* bump version
2022-07-25 09:18:28 +03:00
Ross Smith bdb5e6e37c
2. Devcontainer on Az CLI 2.37 (#2172)
* Removed scripts folder

* Linting

* Linting #3

* Added in build step

* PR comment

* Revert db-migrate

* Admin Consent

* Workspace Admin Consent

* Removed unsued variables

* Split AAD scripts up

* Working on bash read prompt

* Add a final check

* Removing private file

* Moving to Az cli 2.37 for Graph

* Removed appName from credential reset

* update_resource_access

* Remove AAD_TENANT_ID

* Rp should use 2.37 az cli

* increase mgmt AzureRM version

* Bump version

* Pinning versions of apt-get packages

* pipe and copy

* WORKDIR

* Move requirements to /tmp folder

* --no-install-recommends

* --no-install-recommends
2022-07-13 15:00:16 +01:00
David Moore 800a5afaeb
Update all statuses (#2154)
* wip

* fixed up tests

* api/rp status fixed up

* e2e tests fixed up with new status, will fail appropriately

* updated states in UI / notifications

* api v

* rp v

* fixed outdated status check in op

* api v£

* fixed odd unit test

* api v

* added status comment
2022-06-30 16:22:08 +01:00
David Moore 9984e5ef22
Pipeline Property Substitution (#2052)
* initial substitution logic, without array manipulations

* gitea templates updated for testing

* outputs as complex types rather than just strings

* remove / replace wip

* array logic with tests

* refactored for clarity, moved substitutions into retry block

* moved gitea to using a pipeline

* updated docs

* api version bump

* rp bump

* gitea version

* Update docs/tre-templates/pipeline-templates/pipeline-schema.md

Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>

* Update api_app/tests_ma/test_service_bus/test_resource_request_sender.py

Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>

* Update resource_processor/vmss_porter/runner.py

Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>

* Update api_app/tests_ma/test_service_bus/test_resource_request_sender.py

Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>

* renamed gitea + nexus rule collections names to avoid clashes

* api version bump

Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>
2022-06-20 12:32:37 +01:00
Ross Smith 140428e05f
Use different identity to create applications (#1976)
* Use different identity to create applications

* Bump version

* Mandatory application_admin_client_id

* Missing porter parameters
2022-06-08 12:36:37 +01:00
Tamir Kamara 8af9adc0cf
update all versions to 0.3 (#1754) 2022-05-18 13:09:40 -07:00
Marcus Robinson 2457406572
Increase Azure CLI version (#1864) 2022-05-18 11:29:49 -07:00
tanya-borisova a4b64bea99
Resource Processor: Configure logging handler per process (#1784) 2022-05-11 18:36:12 +01:00
David Moore 6495ecb251
Set firewall props via the API (#1764)
* firewall vars init

* passing in vars via b64 encoding

* allow nested properties to be validated

* nested prop validation test

* api version bump

* tf fmt

* rp bump

* rm tf lock file

* lock file back?

* added rule to firewall patch in e2e

* auto-set optional priority

* Update _version.py

* Update resource_template.py

* Update _version.py
2022-05-09 09:15:48 +01:00
Tamir Kamara c2290e41e7
sync porter & azure cli versions (#1741)
* sync porter & azure cli versions

* update exports
2022-04-27 10:49:06 +03:00
David Moore bbb8b8dc19
Multi-Step Operations Logic (#1732)
* dev-vm added from linux-vm to avoid polluting real user_resources

* initial docs

* gaaaaaaaa api version

* Update pipeline-schema.md

Linting MD

* bash linting

* shellcheck

* +x

* linting

* fixed import

* ops doc updated

* creating steps, default main step

* initial patch logic

* submit post for user_resource succeeds

* happy path manually tested

* create / update / delete for all resource types

* fixed up unit tests

* e2e extended passing

* etag retrying and swagger specs update

* operation item creation unit test

* added unit tests for multi-step

* rp version bump

* api version bump again

* returning resource_to_send

* api bump

* pr feedback

* api v£

* versions£
2022-04-26 14:32:28 +01:00
Ross Smith 1571ee525a
3. AAD Managed by Workspace creation (Turned off at the moment) (#1701)
* Add API creds into KeyVault so that RP can read it

* Make this behaviour configurable

* workspace_creation_app_* parameters for RP

* App Service uses keyvault for the API credentials

* Terraform fmt

* Remove extra line

* AAD Managed by Workspace creation

* Terraform fmt

* Conditional creation of AAD

* Purge Protection off when debugging

* Porter flag to create Workspace AAD App

* Linting

* Linting (again!)

* Uninstall workspace works

* workspace-creation-creds.json

* Revert keyvault purge check

* No longer use aad.json

* Missing Api Id returns 500

* Make app_id mandatory

* Rename auth file

* Consolidated into base workspace

* AAD outputs

* Output AAD info from porter workspace creation

* Renamed the aad_auth.json file

* Moving authInformation into workspace properties

* Fix shellcheck

* ShellCheck

* Put authInformation back to previous state

* Linting

* Recover Soft Deleted Keys

* Explicit AppRoleIds

* api_id is defined in a different file

* PR comments

* RP version

* Increased api version
2022-04-25 16:35:05 +01:00
Ross Smith 76cd703395
Rename auth file (#1712)
* Rename auth file

* Increased version
2022-04-21 22:33:03 +01:00
Ross Smith 119abe16da
1. Add API creds into KeyVault so that RP can read it (#1702)
* Add API creds into KeyVault so that RP can read it

* Make this behaviour configurable

* workspace_creation_app_* parameters for RP

* App Service uses keyvault for the API credentials

* Terraform fmt

* Remove extra line

* Recover my key vault!
2022-04-19 23:00:21 +01:00
David Moore 0daf63ca7f
Session Termination (#1656)
* let a session end after message processing

* rp version bump
2022-04-06 14:10:03 +01:00
David Moore 7f9167c8ed
Service Bus Sessions (#1648)
* strip '-' from string in storage account name

* enabled sessions on send

* refactored receiver logic for sessions

* start of bulk/perf tests

* perf tests to test parallel executions

* fixed test after setting operationId as the correlation

* updated comment for .env
2022-04-04 19:01:47 +01:00
David Moore 167586f9fc
Parallel Resource Processing (#1595)
* strip '-' from string in storage account name

* multi processing for runner

* more consistent log messages

* moved to tf var

* bump rp version

* changed var to process_num
2022-03-25 16:47:33 +00:00
Tamir Kamara 020c36cb51
Fix Resource Processor logging (#1548)
* Fix Resource Processor logging

* increase freq of rp pruning job

* update more python versions

* update api version file

* var fix

* rp version update

* force e2e to run here

* debug test

* rp version

* cr comment

* reenable base workspace logging
2022-03-24 23:15:07 +02:00
Anat Balzam 15966aaf45
Add application health monitoring to VMSS resource processor (#1567)
* initial implementation

* add http server to rp

* add tf changes

Co-authored-by: Anat Balzam <anatbalzam@microsoft.com>
2022-03-23 11:54:15 +02:00
David Moore 879401c040
E2E Extended Tests Fix (#1554)
* strip '-' from string in storage account name

* fixed patch e2e test

* e2e updated to check for correct statuses

* rp version bump

Co-authored-by: Tamir Kamara <26870601+tamirkamara@users.noreply.github.com>
Co-authored-by: Ross Smith <ross-p-smith@users.noreply.github.com>
2022-03-21 12:21:17 +00:00
David Moore 7055250eba
Update Resources (#1532)
* strip '-' from string in storage account name

* updated template to take vm size var

* status update in runner, tested locally

* workspaces + services updates, tests fixed up

* e2e tests

* api version bump

* makefile for tests mimic debug process

* unwind local debug change

* unwind makefile change
2022-03-17 09:37:07 +00:00
James Griffin c69cd9d497
Jjgriff93/1462 bug resource processor type error (#1463)
* Fix arguments

* Bump version
2022-03-08 14:08:10 +00:00
James Griffin 717fe11676
Automate resource processor debugging (#1444)
* Separated out config and tidied

* Moved other methods out into modules

* Config fix

* Create sp in debug setup script and use tre.env to store creds

* Fixed docker sock perm

* tre-env -> private.env

* Bump RP version

* Updated docs

* Moved local debugging to main scripts folder

* Removed sudo

* Replaced tr with -r flag

* Removed fixed bug workaround from docs

* Explicit depends_on keyvault access policy to fix 403s

* Moved depends to module level
2022-03-08 00:54:16 +00:00
James Griffin 607abdea06
Implement custom actions (#1385)
* Added action route

* Handle custom actions in resource proc

* Added catch for unsupported operation state changes

* Added launch.json for resource proc debug

* Updated res proc docs to make debugging easier

* Added env sample

* Consistency of custom_actions naming

* Use invoke in runner

* custom_actions camelcasing

* Added template validation

* Moved start stop to correct place

* Fixed ubuntu default

* Bumped version

* Validate against template

* Added workspace service name get

* Added action routes to workspace and workspace services

* Bump versions

* Amended API permissions

* Added action field to operation

* PR amendments

* Amended outputs and fixed sb tests

* Fixed set()

* Added pythonpath and pip install

* Modified rp deploy to use new var names

* Removed unused blocks and shortened debug setup for RP

* reverted sb 503 -> 500

* SB auth warning

* Added use local creds flag
2022-03-01 17:10:13 +00:00
Elad Iwanir ca72939a77
Remove print (#1295) 2022-02-14 13:26:02 +02:00
Elad Iwanir fa489895cb
Consolidated TF modules (#1234)
* consolidated admin_jumpbox

* consolidating api-webapp

* consolidating firewall

* consolidate keyvault

* consolidate servicebus

* consolidate routetable

* consolidate statestore

* consolidate identity

* consolidate storage

* Upgrade tf to 1.1.5

Adding moved blocks

* Revert "consolidating firewall"

* var.location change to azurerm_resource_group.core.location
2022-02-10 15:44:22 +02:00
David Moore 375c8e3ba9
Operations History (#1153)
* initial refactoring and adding of operations

* moved comments in .env.sample to their own lines

* creating + updating operations, db migration script

* api routes + dependencies

* updated existing unit tests

* set ops permissions to match associated resource PATCH permissions

* returning operation in workspace create£

* added resourcePath to resource object

* resource path added to operations

* manual e-2-e tests. location header. deletes.

* fixed delete behaviour

* allow get by id to return non-active docs

* e2e tests simplified

* lint 4 eva

* keep calm and lint

* being trolled by a linter

* Update api_app/service_bus/deployment_status_update.py

Co-authored-by: tanya-borisova <tborisova@microsoft.com>

* added comment for Not Deployed status

* lint off

Co-authored-by: tanya-borisova <tborisova@microsoft.com>
2022-02-07 13:12:56 +00:00
James Griffin ce6db6d12e
Fix for broken tf install (#1172) 2022-01-25 22:45:18 +00:00
Marcus Robinson 4c18c1a8da
Pin az cli version (#1071) 2021-11-09 11:08:36 +00:00
Marcus Robinson 120c1fcefe
Add request tracing to API app (#1039)
* Fix requirements.txt differences and add http middleware for request logging

* Optimise docker file, up version, move where key accessed.
2021-10-14 08:30:03 +01:00
Denis Cepun ac39b5304a
Add version files to images (#951)
Add version files to images
2021-09-29 21:49:29 +03:00
Marcus Robinson 9bc39d4863
Add management locks and disable porter logs (#952) 2021-09-29 12:25:19 +01:00
Marcus Robinson 60c9b62df1
Switch to table storage (#927) 2021-09-24 09:11:15 +01:00