Orkestra is a cloud-native release orchestration and lifecycle management (LCM) platform for the fine-grained orchestration of inter-dependent helm charts and their dependencies
Перейти к файлу
Nitish Malhotra f1c574a36b
Merge dev branch into main (#36)
* feat: Initialise application controller with chart pull & push functionality (#16)

* feat: init application reconciler

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* use controller-gen 0.2.9

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* fix:ci

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* chore: introduct groupName

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* feat: add groupID

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* feat: add fetch function

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* feat: add fetch function

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* add test for AB#13

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* initiate push logic

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* chore: complete push function

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* chore: add configuration

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* fix: golangcilint and tests

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* chore: change branch name pattern to dev (#23)

* retrigger pipeline

Signed-off-by: Kush Trivedi <kushthedude@gmail.com>

* Nitishm/feature/ab#15/appgroup reconciler scaffold (#25)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Scaffolding for application group reconciler

Created the reconciler method for ApplicationGroup controller

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

Added reconciler lookup functionality

Base functionality of application reconciler.
Still needs the generation functions for Argo Workflow object.

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

Workflow interface and appgroup reconciler features

Added a bunch of changes to this commit:
- Created a Workflow engine interface to support any type of workflow
engine
- Added logic to appgroup_reconciler. Mostly WIP but sets the skeleton
for the upcoming commits.

Fixes ab#21

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

feat: AppGroup reconciler update

TODO:
- Generate the HelmRelease object
- Set the executor template

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

feat: ApplicationGroup first pass dev complete

- Needs more unit tests
- Not yet fit for usage

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

Added unit tests (incomplete WIP)

WIP UTs for ApplicationGroup functions:

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Fix go.mod errors (#26)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* chore: change branch name pattern to dev (#23)

* Fix go.mod dependencies

go.mod was failing to resolve dependencies

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

Co-authored-by: Kush Trivedi <44091822+kushthedude@users.noreply.github.com>

* Application and ApplicationReconciler and supported packages into dev branch (#28)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* chore: change branch name pattern to dev (#23)

* Fix go.mod dependencies

go.mod was failing to resolve dependencies

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Add additional tests for Argo.go functions

Added more test and test cases

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Updated status field for application type

Made Chartstatus inline so that we reference it under obj.status instead
of obj.status.status which causes stutter

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Created registry Client and related methods

- Modifications to rest of the code tree to support registry client

Still a WIP

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* New configurer and additional controller members

Added new members to Application and ApplicationGroup controller objects
Refactored the controller Config struct

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Restructuring w/ major changes

Bunch of changes to the controller and reconciler
Updated configurer and config.yaml format
Changes to application and applicationgroup CRD Status fields

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

Co-authored-by: Kush Trivedi <44091822+kushthedude@users.noreply.github.com>

* Nitishm/feat/chart/push (#31)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* chore: change branch name pattern to dev (#23)

* Fix go.mod dependencies

go.mod was failing to resolve dependencies

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Add additional tests for Argo.go functions

Added more test and test cases

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Updated status field for application type

Made Chartstatus inline so that we reference it under obj.status instead
of obj.status.status which causes stutter

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Created registry Client and related methods

- Modifications to rest of the code tree to support registry client

Still a WIP

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* New configurer and additional controller members

Added new members to Application and ApplicationGroup controller objects
Refactored the controller Config struct

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Restructuring w/ major changes

Bunch of changes to the controller and reconciler
Updated configurer and config.yaml format
Changes to application and applicationgroup CRD Status fields

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Update orkestra-core illustration (#30)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Update diagram and switch harbor for chartmuseum

Update the orkestra-core diagram replacing harbor for chartmuseum

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Implemented the Push functionlity

Verfied manually that charts get uploaded to the staging directory

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* bug: resolve merge errors

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

Co-authored-by: Kush Trivedi <44091822+kushthedude@users.noreply.github.com>

* Fixed lint errors and test failures (#33)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* chore: change branch name pattern to dev (#23)

* Fix go.mod dependencies

go.mod was failing to resolve dependencies

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Add additional tests for Argo.go functions

Added more test and test cases

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Updated status field for application type

Made Chartstatus inline so that we reference it under obj.status instead
of obj.status.status which causes stutter

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Created registry Client and related methods

- Modifications to rest of the code tree to support registry client

Still a WIP

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* New configurer and additional controller members

Added new members to Application and ApplicationGroup controller objects
Refactored the controller Config struct

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Restructuring w/ major changes

Bunch of changes to the controller and reconciler
Updated configurer and config.yaml format
Changes to application and applicationgroup CRD Status fields

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Update orkestra-core illustration (#30)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Update diagram and switch harbor for chartmuseum

Update the orkestra-core diagram replacing harbor for chartmuseum

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Implemented the Push functionlity

Verfied manually that charts get uploaded to the staging directory

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* bug: resolve merge errors

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Modifications and enhanced argo logic

- Multiple changes in argo object to support generation of valid workflow
- Added overlays field to Applications CR as a workaround for https://github.com/kubernetes/kubernetes/issues/98683
- Helm chart for orkestra init. All dependency charts helm-operator, chartmuseum, argo added to chart dependencies.
- Update main.go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Fix lint and argo_test errors

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

Co-authored-by: Kush Trivedi <44091822+kushthedude@users.noreply.github.com>

* chore: Update docker targets in Make and comment cleanup (#34)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* chore: change branch name pattern to dev (#23)

* Fix go.mod dependencies

go.mod was failing to resolve dependencies

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Add additional tests for Argo.go functions

Added more test and test cases

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Updated status field for application type

Made Chartstatus inline so that we reference it under obj.status instead
of obj.status.status which causes stutter

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Created registry Client and related methods

- Modifications to rest of the code tree to support registry client

Still a WIP

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* New configurer and additional controller members

Added new members to Application and ApplicationGroup controller objects
Refactored the controller Config struct

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Restructuring w/ major changes

Bunch of changes to the controller and reconciler
Updated configurer and config.yaml format
Changes to application and applicationgroup CRD Status fields

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Update orkestra-core illustration (#30)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Update diagram and switch harbor for chartmuseum

Update the orkestra-core diagram replacing harbor for chartmuseum

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Implemented the Push functionlity

Verfied manually that charts get uploaded to the staging directory

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* bug: resolve merge errors

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Modifications and enhanced argo logic

- Multiple changes in argo object to support generation of valid workflow
- Added overlays field to Applications CR as a workaround for https://github.com/kubernetes/kubernetes/issues/98683
- Helm chart for orkestra init. All dependency charts helm-operator, chartmuseum, argo added to chart dependencies.
- Update main.go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Fix lint and argo_test errors

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Docker target changes and TODO comment cleanup

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

Co-authored-by: Kush Trivedi <44091822+kushthedude@users.noreply.github.com>

* [chore] Helm chart for orkestra and relevant changes (#35)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* chore: change branch name pattern to dev (#23)

* Fix go.mod dependencies

go.mod was failing to resolve dependencies

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Add additional tests for Argo.go functions

Added more test and test cases

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Updated status field for application type

Made Chartstatus inline so that we reference it under obj.status instead
of obj.status.status which causes stutter

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Created registry Client and related methods

- Modifications to rest of the code tree to support registry client

Still a WIP

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* New configurer and additional controller members

Added new members to Application and ApplicationGroup controller objects
Refactored the controller Config struct

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Restructuring w/ major changes

Bunch of changes to the controller and reconciler
Updated configurer and config.yaml format
Changes to application and applicationgroup CRD Status fields

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Update orkestra-core illustration (#30)

* Add PUML sequence diagrams

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Setup azure-pipeline for Go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Update diagram and switch harbor for chartmuseum

Update the orkestra-core diagram replacing harbor for chartmuseum

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Implemented the Push functionlity

Verfied manually that charts get uploaded to the staging directory

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* bug: resolve merge errors

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* feat: Modifications and enhanced argo logic

- Multiple changes in argo object to support generation of valid workflow
- Added overlays field to Applications CR as a workaround for https://github.com/kubernetes/kubernetes/issues/98683
- Helm chart for orkestra init. All dependency charts helm-operator, chartmuseum, argo added to chart dependencies.
- Update main.go

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Fix lint and argo_test errors

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Docker target changes and TODO comment cleanup

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Helm chart for orkestra and related changes

- Created the helm chart for orkestra
- Made relevant changes based on feedback from running it on the cluster

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Create example dir and edits to README.md

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Update example and set ownersreference for helmrelease

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Fix argo tests failures from previous set of changes

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

* Fix lint error in argo test

Signed-off-by: Nitish Malhotra <nitish.malhotra@gmail.com>

Co-authored-by: Kush Trivedi <44091822+kushthedude@users.noreply.github.com>

Co-authored-by: Kush Trivedi <44091822+kushthedude@users.noreply.github.com>
2021-02-02 18:49:42 -08:00
api/v1alpha1 Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
assets Update orkestra-core illustration (#30) 2021-01-28 18:47:01 -08:00
chart/orkestra Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
config Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
controllers Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
docs Nitishm/docs/sequence diagram update (#17) 2021-01-11 09:54:52 -08:00
examples/simple Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
hack chore: add licence header in project (#13) 2021-01-05 10:55:51 -05:00
pkg Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
scripts feat: add make target for kubebuilder setup (#19) 2021-01-20 15:21:23 +05:30
testwith Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
.gitignore Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
.golangci.yml Kubebuilder scaffolding for Orkestra Project (#7) 2021-01-05 09:46:53 -05:00
CONTRIBUTING.md feat: add pre-requisities documentation for the orkestra project (#2) 2020-12-16 14:06:58 -05:00
Dockerfile Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
LICENSE.TXT feat: add pre-requisities documentation for the orkestra project (#2) 2020-12-16 14:06:58 -05:00
Makefile Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
PROJECT Kubebuilder scaffolding for Orkestra Project (#7) 2021-01-05 09:46:53 -05:00
README.md Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
SECURITY.md feat: add pre-requisities documentation for the orkestra project (#2) 2020-12-16 14:06:58 -05:00
azure-pipelines.yml chore: change branch name pattern to dev (#23) 2021-01-25 14:37:12 +05:30
config.yaml Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
go.mod Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
go.sum Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00
main.go Merge dev branch into main (#36) 2021-02-02 18:49:42 -08:00

README.md

Orkestra

Orkestra is a cloud-native release orchestration platform that allows you to manage the lifecycle and orchestrate the release of groups of Kubernetes Helm applications through Kubernetes Custom Resource Objects. Orkestra works by generating dependency driven DAG workflows to orchestrate the deployment and upgrade of multiple applications within a Kubernetes cluster. Additionally, Orkestra can also orchestrate the deployment of multiple microservice (helm dependencies - sub-charts) within a parent Helm chart.

Overview

What is it?

Orkestra renders a DAG based workflow for deploying applications to a Kubernetes cluster by leveraging popular and mature open-source frameworks like Argo (Workflows), Flux Helm Operator and Chart-museum

What problems does it solve?

Complex applications oftentimes require intelligent release orchestration and lifecycle management which is not provided by Helm itself.

Take, for example, Continuous Deployment of mission-critical applications - like 5G core Network Functions or NFs

  • Network Functions are applications that rely on a rich ecosystem of infrastructure and PaaS (platform-as-a-service) components to be present on the cluster before they can be deployed. This establishes a hard dependency between the applications and the infra/paas applications.

How it works

To solve the complex application orchestration problem Orkestra builds a Directed Acyclic Graph using the application, and it's dependencies and submits it to Argo Workflow. The Workflow nodes use workflow-executor nodes to deploy a HelmRelease object into the cluster. This HelmRelease object signals Flux's HelmOperator to perform a "Helm Action" on the referenced chart.

  1. Submit Application and ApplicationGroup CRs
  2. For each Application in ApplicationGroup download Helm chart from “primary” Helm Registry
  3. (*optional) For each dependency in the Application chart, if dependency chart is embedded in charts/ directory, push to ”staging” Helm Registry (Chart-museum).
  4. Generate and submit Argo Workflow DAG
  5. (Executor nodes only) Submit and probe deployment state of HelmRelease CR.
  6. Fetch and deploy Helm charts referred to by each HelmRelease CR to the Kubernetes cluster. (*optional) Embedded subcharts are fetched from the “staging” registry instead of the “primary/remote” registry.

Sequence Diagram

See sequence diagrams

Features

  • Built for Kubernetes - custom controller built using the kubebuilder project
  • Easy to use - familiar declarative spec using Kubernetes Custom Resource Definitions
  • Dependency management - DAG-based workflows for groups of application charts and their sub-charts using Argo Workflow
  • Works with any Continous Deployment system - bring your own CD to deploy Orkestra Custom Resources. Works with any Kubernetes compatible Continuous Deployment framework like FluxCD and ArgoCD.
  • Built for GitOps - describe your desired set of applications (and dependencies) declaratively and manage them from a version-controlled git repository.

Contributing

For instructions about setting up your environment to develop and extend the operator, please see contributing.md

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Reporting security issues and security bugs

For instructions on reporting security issues and bugs, please see security.md