Граф коммитов

29 Коммитов

Автор SHA1 Сообщение Дата
Eliise S bdeb1284a2 Add running load tests in pipeline 2020-03-04 20:08:42 +00:00
Eliise S d578e6cbc0 Add grafana charts for load testing and mockAPI
Co-authored-by: Lawrence Gripper <info@grippers.co.uk>
2020-03-04 20:08:42 +00:00
Eliise 984e6ed5de
Add ability to configure the number of max concurrent reconciles (#154)
* Add MAX_CONCURRENT_RUN_RECONCILES to yaml
2020-02-17 17:32:08 +11:00
Dave Storey dd3aa96f99
Adding grafana dashboard for azure-databricks-operator (#157)
* dashboard configmap added

* moved the Grafana docs to own section

* updated the docs for installing grafana dashboard
2020-02-05 22:51:43 +11:00
Eliise 09e4bc29e2
Use correct yaml (#159) 2020-02-05 21:04:29 +11:00
azadeh khojandi c145ca41ce
Issue/142 (#143)
* Add cluster name support to dbricks run
2020-01-09 11:26:30 +11:00
Dave Storey 49d44fa6e5 Extended databricks operator to report metrics into Prometheus (#104)
* Add initial extra metrics to controllers
* Update ServiceMonitor to allow prometheus to connect
2019-12-17 17:52:34 +11:00
azadeh khojandi bc604f475c
update build and clean up (#134)
* update build and clean up
2019-12-12 20:40:12 +11:00
azadeh khojandi c8c53ee353
upgrade kubebuilder (#133)
* upgrade kubebuilder
2019-12-12 10:52:48 +11:00
Jason ec20116f04 Updated SecretScope type. 2019-12-02 20:54:05 +11:00
JasonTheDeveloper 44eda83fdd
Merge branch 'master' into fix/acl-99 2019-12-02 11:21:20 +11:00
azadeh khojandi 8a81450e22
Adding support for using existing_cluster_name (#126)
* Adding support for using existing_cluster_name as well as existing_cluster_id #86

* add new tests and clean up tests
2019-12-02 09:56:09 +11:00
Jason 85e7b9bedc Added status updates to config to prevent wasted cycles. 2019-11-27 11:12:39 +11:00
Jason c0a5ccfd71 Updated secret scope type. 2019-11-27 11:04:03 +11:00
Stuart Leeks 5615c1f3e3 Add golangci-lint and fixup flagged issues (#116)
* Configure linter
* install golangci-lint in pipeline
2019-11-26 12:43:14 +11:00
azadeh khojandi d3dbafff0a
update sdk (#107) 2019-11-14 17:04:25 +11:00
azadeh khojandi c934ed028f
update databricks-sdk-golang version (#97) 2019-10-24 11:44:54 +11:00
azadeh khojandi e2043910cf
change group API version from beta1 to alpha1 (#78) 2019-10-14 09:45:28 +11:00
azadeh khojandi b20fd52b16
update Api Group version - breaking change (#74) 2019-10-10 13:21:51 +11:00
azadeh khojandi b9a0b57634
Add missing manager_auth_proxy_patch (#72) 2019-10-10 08:53:51 +11:00
Azadeh Khojandi 424192f09d update devcontainer
increase resource limit and change rbac for events
2019-10-03 13:10:27 +10:00
azadeh khojandi 8d57edb36a
adding back manager_image_patch (#58) 2019-09-03 17:19:40 +10:00
Xinyun Zhou 0f722a710f feature: workspaceitem (#57)
* init commit for workspace item

* add support for workspace item
2019-08-09 09:37:23 +10:00
Xinyun Zhou 0d2b10e078 allow the cluster to be refreshed after submission and fix cluster id type (#56) 2019-08-08 13:32:03 +10:00
Xinyun Zhou c273324cf5 Add support for DBFS Blocks and Clusters (#55)
* init commit on cluster code generation

* generate dbfsblock types

* edit sample crd

* change dcluster status type to support float values, generate crd

* fix existing cluster setting is a pointer to bool

* implement logic for dbfs blocks

* add dbfsblock update support

* implemented cluster, improved test coverage

* make byte secret to be base64 encoded

* fix naming convention and add FromDataBricksClusterInfo description
2019-07-29 16:56:23 +10:00
Xinyun Zhou 04c2c3ecbf Using native DataBricks API model (Job, Run, Secret) (#53)
* Initial secret scopes kubebuilder create

* add djob, generate manifests

* start to implement job

* implement basic job structure

* Hydrating secret scope and implement submit secret scope

* half way on run

* Handle submitted SecretScopes with update implementation

* upgrade sdk

* fix job name reference

* Add missing implementation to handle submitted secret scope in reconcile

* handle run submit and run now

* Secret scope implementation that works

* better error handling and logging

* fix minor inconsistency in djob and run

* remove notebookjob type

* add test coverage

* adding tests, better naming consistency

* add html cover report

* Support referencing K8s secrets for DB secret scope secrets

* add azure devops test coverage

* install tooling for testing

* add test files

* make test pass for djob, adding kill and wait for gracefully shutting down manager

* add run job test

* adding testing for secretscope and support for byte value secrets

* add event recorder to secretscope. fix sample

* update recorder in test setup

* fix name of test recorder

* increasing timeout for secret scope test

* further increase to scope test timeout

* another increase to test timeout

* Update README.md

* export db client in tests to do cleanup before/after tests.

* documentation update

* adding deploy.md

* updated readme.md resources.md and contributing.md

* adding roadmap

* Updating contributing.md

* Updating contributing.md

* extending job waiting time because databricks may not reconcile in time

* fix typo, fix context to use Background, remove noteboojob reference in PROJECT

* Change Principal
2019-07-23 11:26:20 +10:00
Jacob Zhou a04eef531b fix type tests do not honor test-existing 2019-07-04 16:24:22 +10:00
azadeh khojandi 0ff6685484
V2/cicd (#42)
* update readme and add missing env vars
2019-06-26 12:25:05 +10:00
Rian Finnegan b3d09ba997 Pure Go (#34)
* Rewrite the operator to use go directly to communicate with Databricks (#19)

* remove swagger

* migrate to go mod

* better submit logic

* move operator out

* clean up rbac

* organise codebase

* use local kustomize and extend tests

* fix delete error when job does not exist

* adding tests for notebookspec

* adding secret testing, add logging

* adding sleep to allow more stable tests

* add extra column and state

* added shortnames (nbj)

* rename to azure pipelines

delete path filter in pipeline yaml

* adding codegen annotation for rbac, shotname and columns (#32)

* always requeue the reconciler (#33)

* add databricks host and token to test env

* fix delete data inconsistency and improved status struct

* updated timeout to 60 seconds

* disable install kustomize step

* adding binary in lfs

* checkout lfs in build

* update readme

* bump databricks sdk to remove manual deepcopy code

* cherry pick readme change

* Feat/readme (#38)

* update readme

* upgrade kubebuilder version 

* re-apply a few changes

* adding tests for types

* test not really working but it's in there

* fix context and describe field

* all tests pass

* adding readme and pipeline

* adding docs image

* make pipeline work with gomodule

* try with installation of kubebuilder and kustomize

* forgot to set permission

* adding deep equal test to only update status when changed

* fix old group name to now be databricks
2019-06-26 11:21:09 +10:00