* docs(helm): schematic diagram in mermaid format
* Clarifies that dependencies can be external or internal to cluster
* Explicitly show namespace containing secrets
* feat(server): add switchable admin authz override
* fix(server): make sure tests work with the new admin override
* feat(server authz): make sure to add all requested roles to server admins in admin override mode
* chore(frontend): use bitnami/openresty as base image for frontend Dockerfile
openresty/openresty was not being patched as frequently as we would like, resulting in numerous
vulnerabilities without resolution. bitnami/openresty is being patched more frequently.
Some additional changes were necessary when porting our frontend between these distributions:
- html files are in /app
- nginx.conf is in /opt/bitnami/openresty/nginx/conf/nginx.conf
- envsubst is not available by default in bitnami/openresty and needs to be copied in
- Nginx.conf - we wrap the server block in http block and overwrite root nginx.conf
- using the existing bitnami/openresty nginx.conf as a server block alone causes issues with bitnami/openresty, as bitnami/openresty provides a root nginx.conf which conflicts with directives in Speckle's server block
- we copy the directives from openresty/openresty (which are known to work with Speckle's server block), and apply them alongside Speckle's server block. This creates a new root nginx.conf which we can overwrite the default on the image.
- nginx should use a port available to non sudo/root user, we have selected 8080 instead of previous 80
- need to explicitly output nginx logs to stderr / stdout
Created a readonly root file system on Kubernetes. This requires the following changes:
- emptyDir volumes are mounted in kubernetes to allow bitnami/openresty to write to specific locations
- explicitly include and copy mime.types file to nginx configuration directory
Due to the change to non-privileged port number (8080), the following subsequent changes were required:
- Update 1-click deployment script to match frontend at port 8080
- Updates docker-compose-speckle.yaml file
Co-authored-by: Gergő Jedlicska <gergo@jedlicska.com>
* style(server): fix formatting
* fix(preview-service): fix chromium deps in Dockerfile
* feat(helm chart): expose file uploads disable flag in the helm chart
* fix(helm chart): value name fix
* fix(helm): its values
* style(server): fix formatting
* fix(preview-service): fix chromium deps in Dockerfile
* feat(helm chart): expose file uploads disable flag in the helm chart
* fix(helm chart): value name fix
* Improves error logging
- use pino error logger correctly by passing in error as first argument
* monitor deployment: Filter logging at INFO level and above
* Use structured logging to create parameters for monitoring results
* Add structured logging to obj fileimport service
* Fileimport service, fix and improve logging
- use child logger with additional context where possible
- select appropriate logging level
- fix duplicated context in log statement
* REST endpoints, add context to structured logging and remove same context from message
* Webhook service provides context to bound logger to properly use structured logging
- Pass bound logger containing context to `makeNetworkRequest`
- do not log url, as it may contain a secret (like Discord's webhook urls), instead log the webhook Id
- log error message when network call fails
* upload: make better use of structured logging when recording data
* pino-pretty when in dev or test mode
- pino-pretty configured to send to stderr
* LOG_PRETTY env var
* Silence structured logging during testing
- can not rely on determining the port number by reading from stdout/stderr
- instead we determine which port is free, then create our server on that port
- we then poll that port until the server is ready before commencing tests
* Allow puppeteer to install chromium
* Do not need to install chromium separately
* Moves speckle-server, webhook-service, fileimport-service, monitoring-deployment, and test-deployment images to Distroless.
Partially addresses https://github.com/specklesystems/speckle-server/issues/883
* preview-service uses similar image for building and production stages
* explicitly include chromium-common dependency to prevent error in preview service
* Bump chromium packages due to package versions not being found
* Handle machine-id in distroless
- distroless has no shell, so node-machine-id will result in an error
- this commit introduces error handling and defaults to a uuid v4 in the case of an error
* Update binary location for readiness and liveness checks to match the binary location in Distroless
* Allow node binary path to be set as environment variable in fileimport service
* chore(node): upgrades to node 18
Node 16 was out of support (but not security upgrades), so bumping to next stable version.
https://github.com/specklesystems/speckle-server/issues/1187
* Update server liveness and readiness probes for node 18
* Bump web-ifc to 0.0.36
* Apply `--no-experimental-fetch` flag to fileimport-service to prevent issues in web-ifc (via emscripten) with node 18
* Revert "Revert structured logging 2 (#1240)"
This reverts commit 78ecaeffcb.
* Logging should not be bundled into core shared directory
* making sure observability stuff isnt bundled into frontend
Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
* Revert "'@' shortcut must come after it is configured in bootstrap (#1239)"
This reverts commit 967329473f.
* Revert "Structured logging (attempt 2) (#1234)"
This reverts commit 444d2ca7dd.
* Revert "Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)"
This reverts commit 63e6581162.
* Use pino-http instead of express pino logger
* Use correct reference to knex and do not instantiate HttpLogger prematurely
* Adds missing dependency for pino to webhook-service
* Do not instantiate middleware when passed to express
* Refactor to move logging into shared
* Copy shared packages into dockerfiles
* Build shared workspace in docker build-stage for fileimport & webhook
* each log line is a json object
* structured logging allows logs to be ingested by machines and the logs to be indexed and queried addresses #1105
* structured logging allows arbitrary properties to be appended to each log line, and ingestion of logs to remain robust
* Structured logging provided by `pino` library
* Add `express-pino-logger` dependency
* Remove `debug`, `morgan`, and `morgan-debug` and replace with structured logging
* `console.log` & `console.error` replaced with structured logging in backend
* Remove `DEBUG` environment variable and replace with `LOG_LEVEL`
- Note that there is a test which reads from a logged line on `stdout`. This is not robust, it would be better to use the childProcess.pid to look up the port number.
* Log errors at points we explicitly send error to Sentry
* Amend indentation of a couple of log messages to align indentation with others
* fix(helm chart): networkPolicy supports distinct namespaces for prometheus pod & servicemonitor
Network policy did not allow ingress from prometheus if it was deployed in a different namespace
from the servicemonitor. This PR allows the ingress to be configured to match the operator's
requirements.
addresses https://github.com/specklesystems/gitOps/issues/68
* provides additional validation and error output when getting secrets
* Fix for kubernetes network policies using s3 details from ConfigMap
* Remove blocking of 10.0.0.0/8 range as this also prevents access to cloud provider private IPs
* Update values.schema.json
* feat(helm): s3 configuration can be loaded from configmap
- Variables for s3's configuration can now be read in from a configmap in the cluster. This allows
deployment tooling, such as Terraform or CloudFormation, to dynamically create an s3 bucket and
create a configmap with the necessary values. This decouples the cluster deployment from the helm
release.
* Update values.schema.json for helm chart
- also include changes from a previous commit that had not been included previously
* feat(helm chart): secrets can be referenced from different kubernetes Secret resources
Currently secrets have to be referenced from a single kubernetes Secret resource (default name
'server-vars'). This PR allows each secret to be loaded from a separate kubernetes Secret. If
values for individual secrets are not provided, it defaults to the previous single kubernetes
resource. This single kubernetes secret should now be considered deprecated in favour of individual
references.
* Fix error in Redis key
* Fix DNS egress for Redis in CiliumNetworkPolicy
- only give access to optional secrets if the component is enabled
* Values should be empty by default to allow for backwards compatibility
* fix(helm chart): allow egress in server Network Policies to Apollo
The Cilium and Kubernetes network policies currently do not allow egress from the server to Apollo
for graphql monitoring.
Kubernetes Network Policies don't allow domain names. We have an open support ticket with Apollo
Studio to request which CIDR to limit egress to. Until then, we will need to open egress to
everywhere if a Kubernetes Network Policy is used.
* fix(helm chart): remove unused values from helm chart
Previous commit introduced two additional values that are not being used for s3. This commit
removes them.
* Looks up domain or IP from secret for redis and postgres
- undertakes a kubectl get on the secret. The user or service account that deploys helm must have permissions to view the secret.
- fix: matchName for domain instead of matchPattern
- fix: typo in protocol
* Only allow monitoring ingress if monitoring is enabled
* Port can be determine from the provided secret
- updates values.yaml to only require port for postgres and redis for inCluster endpoints
* feat(helm chart): deployes Cilium Network Policies when configured
Cilium Network Policies provide more features over regular Kubernetes Network Policies, but Cilium
is not available everywhere. When selected by an operator, Cilium Network Policies will be deployed
instead of Kubernetes Network Policies.
Fixes https://github.com/specklesystems/speckle-server/issues/913
* Cilium Network Policy for fileimport service.
* tested only for external host.
* Still to test internal pod and external IP.
* Cilium network policy for file import service restricts DNS
* allows egress to service instead of endpoint
* file import service uses service url of speckle-server
* helper functions for server and dns
* DRY the prometheus selector
* CiliumNetworkPolicy for frontend
* CiliumNetworkPolicy for monitoring service
* CiliumNetworkPolicy for preview service
* CiliumNetworkPolicy for test
* CiliumNetworkPolicy for webhook_service
* CiliumNetworkPolicy for Server
* Test should egress to domain, not internally
* Test should be in tests directory to match Helm convention for tests
* Test should explicitly deny ingress from everywhere
* Server needs to egress to canonical domain (i.e. itself)
- DNS and egress for canonical domain added to Server
- As Test also egresses via canonical domain to access Server, we do not require the intra-cluster ingress to the server from the test pod
- Explicitly deny all egress from frontend
* WIP update to schema.json
* Breaking Change: inCluster network policies supported for cilium
* Breaking change: kubernetes network policy podSelector and namespaceSelector are now at a different level
* Updates schema.json
* add notes to remove egress once bug is fixed
ServiceAccounts for each service do not mount service account token (which allows access to the
kubernetes API), and limit the secrets each user of the service account has access to.
Fixes https://github.com/specklesystems/speckle-server/issues/859
* feat(helm chart): node affinities, tolerations etc. are configurable
Kubernetes operators should be able to configure Speckle to be deployed on certain nodes based on
rules they provide. This commit allows affinity, nodeSelector, tolerations, and
topologySpreadConstrains to be provided by the operator.
fixes https://github.com/specklesystems/speckle-server/issues/861
Fileimport service retreives blobs via the server storage API, and not directly from s3. Fileimport
service no longer requires information or credentials about s3.
* Allow save object to S3 in different region
* feat(helm & docker-compose): adds S3_REGION to helm chart & docker-compose
Explicitly adding the environment variable to deployment configuration files provides system operators with documentation of its existence.
Set to empty by default, which will result in the default value being used.
Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
* feat(helm chart): network policies are provided for all services
Network policies are used to deny arbitrary egress and ingress to a pod, providing more security
hardening.
Fix https://github.com/specklesystems/speckle-server/issues/860
* NetworkPolicies for remaining services
* Network policies are configurable but enabled by default
* fix to naming
* Use named port
* Helper function for defining redis egress
* Network policy is more tightly defined to port for service if fqdn
* if an IP is provided for redis, postgres, or blob storage, egress is limited to that IP
* Note about limitations
* Simplifies networkpolicy logic by requiring variables to be provided in values.yaml
* default disable networkpolicy, otherwise end users will have to provide all the additional values and that could become confusing
* supports dependencies being deployed within the same cluster
* Disable network policies by default
* Ensure the host name does not contain a port
* Exclude (likely) kubernetes IP ranges from allowed egress
* Add explicit ingress to the server from fileimport and test
* disable test networkpolicy if test is disabled
* Allow egress to sentry
* remove access to s3 from preview service
* remove access to redis from fileimport service
* Allow prometheus ingress to metrics endpoints
* tightens ingress by restricting to the prometheus pod in a single namespace
* Limit ingress on the server to the nginx ingress controller and prometheus
* Limit ingress to frontend to just the nginx ingress controller
* Fileimport does not require s3
* feat(helm chart): prometheus monitoring namespace and release name should be configurable
Currently Speckle assumes prometheus is deployed in the 'speckle' namespace and is deployed as a
release named 'kube-prometheus-stack'. This commit introduces non-breaking changes that allow
custom values for these to be provided, defaulting to the current assumed values if they are not
provided.
fixes https://github.com/specklesystems/speckle-server/issues/863
* Fix serviceMonitor so that it can find services in a different namespace
* Namespace selector is not required if the default namespace is being used
* feat(helm chart): add SecurityContext to pods and containers
Speckle pods should run with minimal privileges and capabilities to function.
Fix https://github.com/specklesystems/speckle-server/issues/857
* Update securityContext for all pods
* frontend runs as nonroot and readonly root filesystem
- set fsgroup for all pods with volumes
* Frontend requires write directory at /etc/nginx/conf.d
* Allow openresty log directory to be writable
* feat(helm local test): add test container into the make script
Co-authored-by: Gergő Jedlicska <gergo@jedlicska.com>
* refactor(helm chart): dRY for some labels
* Metadata for Chart.yaml
* refactor(helm chart): dRY using common selector labels
Able to remove `app` and `project` labels from each template and incorporate into definitions
Iain Sproat
Gergő Jedlicska
spgoad
Peter Grainger