Azure
/
helm-elasticstack
зеркало из https://github.com/Azure/helm-elasticstack.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Upgrade to elasticstack 6.4.3 (#9) 

* merge back (#1)

* Replace gas with gosec (#5)

* Fix typo in Readme (#3)

* bumping dockerfiles, charts and READMEs to 6.4.3
This commit is contained in:
Bernd Kiefer 2018-11-27 11:42:07 +01:00 коммит произвёл GitHub
Родитель cc78c0f425
Коммит 5bbb17080f
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
8 изменённых файлов: 130 добавлений и 101 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

98
README.md
Просмотреть файл

@ -6,38 +6,64 @@
These [Helm](https://github.com/kubernetes/helm) charts bootstrap a production ready [Elastic Stack](https://www.elastic.co/products) service on a Kubernetes cluster managed by [Azure Container Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes) and other Azure services. These [Helm](https://github.com/kubernetes/helm) charts bootstrap a production ready [Elastic Stack](https://www.elastic.co/products) service on a Kubernetes cluster managed by [Azure Container Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes) and other Azure services.
The following features are included: The following features are included:
* Deployment for [Elasticsearch](https://www.elastic.co/products/elasticsearch), [Kibana](https://www.elastic.co/products/kibana) and [Logstash](https://www.elastic.co/products/logstash) services * Deployment for [Elasticsearch](https://www.elastic.co/products/elasticsearch), [Kibana](https://www.elastic.co/products/kibana) and [Logstash](https://www.elastic.co/products/logstash) services
* Deployment script which retrieves the secrets and certificates from [Azure Key Vault](https://azure.microsoft.com/en-us/services/key-vault/) and injects them into the Helm charts * Deployment script which retrieves the secrets and certificates from [Azure Key Vault](https://azure.microsoft.com/en-us/services/key-vault/) and injects them into the Helm charts
* TLS termination and load balancing for Kibana using [NGINX Ingress Controller](https://github.com/kubernetes/ingress-nginx) * TLS termination and load balancing for Kibana using [NGINX Ingress Controller](https://github.com/kubernetes/ingress-nginx)
* [Azure Active Directory](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-authentication-scenarios) authentication for Kibana * [Azure Active Directory](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-authentication-scenarios) authentication for Kibana
* Integration with [Azure Redis Cache](https://azure.microsoft.com/en-us/services/cache/) which acts as middleware for log events between the Log Appenders and Logstash * Integration with [Azure Redis Cache](https://azure.microsoft.com/en-us/services/cache/) acting as middleware for log events between the Log Appenders and Logstash
* TLS connection between Logstash and Redis Cache handled by [stunnel](https://www.stunnel.org/) * TLS connection between Logstash and Redis Cache handled by [stunnel](https://www.stunnel.org/)
* Support for [Multiple Data Pipelines](https://www.elastic.co/blog/logstash-multiple-pipelines) in Logstash which allows multiple Redis Caches as input (e.g one Redis cluster per environment) * Support for [Multiple Data Pipelines](https://www.elastic.co/blog/logstash-multiple-pipelines) in Logstash allowing multiple Redis Caches as input (e.g one Redis cluster per environment)
* Installation of a [Curator](https://github.com/elastic/curator) cron job that cleans up daily all indexes which are older than 30 days * Installation of a [Curator](https://github.com/elastic/curator) cron job that cleans up daily all indexes which are older than 30 days
* Installation of [Elasticsearch Index Templates](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/indices-templates.html) as a pre-deployment step * Installation of [Elasticsearch Index Templates](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/indices-templates.html) as a pre-deployment step
* Installation of [Elasticsearch Watches](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/watcher-api.html) as a post deployment step. The watches can be used for alerts and notifications over Microsoft Teams/Slack webhook or email * Installation of [Elasticsearch Watches](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/watcher-api.html) as a post deployment step. The watches can be used for alerts and notifications over Microsoft Teams/Slack webhook or email
* Installation of [Elasticsearch x-pack license](https://license.elastic.co/download) as a post deployment step * Installation of [Elasticsearch x-pack license](https://license.elastic.co/download) as a post deployment step
# Architecture <!-- TOC -->
- [Introduction](#introduction)
- [Architecture](#architecture)
- [Azure Resources](#azure-resources)
- [Azure Key Vault](#azure-key-vault)
- [Public Static IP and DNS Domain](#public-static-ip-and-dns-domain)
- [Redis Cache](#redis-cache)
- [Application for Azure Active Directory](#application-for-azure-active-directory)
- [Microsoft Teams/Slack incoming Webhook](#microsoft-teamsslack-incoming-webhook)
- [Customize Logstash Configuration](#customize-logstash-configuration)
- [Multiple Data Pipelines](#multiple-data-pipelines)
- [Indexes Clean Up](#indexes-clean-up)
- [Index Templates](#index-templates)
- [Index Watches](#index-watches)
- [Elasticsearch License](#elasticsearch-license)
- [Installation](#installation)
- [NGINX Ingress Controller](#nginx-ingress-controller)
- [Elasticsearch Cluster](#elasticsearch-cluster)
- [Kibana and Logstash](#kibana-and-logstash)
- [Rolling Update](#rolling-update)
- [Contributing](#contributing)
<!-- /TOC -->
## Architecture
![architecture](images/architecture.png?row=true) ![architecture](images/architecture.png?row=true)
# Azure Resources ## Azure Resources
A few Azure resources need to be provisioned before proceeding with the Helm charts installation. A few Azure resources need to be provisioned before proceeding with the Helm charts installation.
## Azure Key Vault ### Azure Key Vault
All secrets and certificates used by the charts are stored in an Azure Key Vault. The deployment script is able to fetch them and to inject them further into the charts. All secrets and certificates used by the charts are stored in an Azure Key Vault. The deployment script is able to fetch them and to inject them further into the charts.
You can create a new Key Vault with default permissions: You can create a new Key Vault with default permissions:
```console ```console
az keyvault create --name <KEYVAULT_NAME> --resource-group <RESOURCE_GROUP> az keyvault create --name <KEYVAULT_NAME> --resource-group <RESOURCE_GROUP>
``` ```
It is recommended that you use two different principals to operate the Key Vault: It is recommended that you use two different principals to operate the Key Vault:
* A _Security Operator_ who has read/write access to secrets, keys and certificates. This principal should be only used for setting up the Key Vault or rotate the secrets. * A _Security Operator_ who has read/write access to secrets, keys and certificates. This principal should be only used for setting up the Key Vault or rotate the secrets.
* A _Deployment Operator_ who is only able to read secrets. This principal should be used to perform the deployment. * A _Deployment Operator_ who is only able to read secrets. This principal should be used to perform the deployment.
@ -49,7 +75,7 @@ az keyvault set-policy --upn <SECURITY_OPERATOR_USER_PRINCIPAL> --name <KEYVAULT
az keyvault set-policy --upn <DEPLOYMENT_OPERATOR_USER_PRINCIPAL> --name <KEYVAULT_NAME> --resource-group <RESOURCE_GROP> --secret-permissions get list az keyvault set-policy --upn <DEPLOYMENT_OPERATOR_USER_PRINCIPAL> --name <KEYVAULT_NAME> --resource-group <RESOURCE_GROP> --secret-permissions get list
``` ```
## Public Static IP and DNS Domain ### Public Static IP and DNS Domain
You can allocate a public static IP in Azure. This IP will be used to expose Kibana to the world. You can allocate a public static IP in Azure. This IP will be used to expose Kibana to the world.
@ -78,7 +104,7 @@ The private key password must be also stored in a different secret, such that it
az keyvault secret set --name kibana-certificate-key-password --vault-name <KEYVAULT_NAME> --value <PASSWORD> az keyvault secret set --name kibana-certificate-key-password --vault-name <KEYVAULT_NAME> --value <PASSWORD>
``` ```
## Redis Cache ### Redis Cache
The Azure Redis Cache is used as a middleware between the Log Appenders and Logstash service. This is quite scalable and it also decouples the Log Appenders from Elastic Stack service. You can use any Log Appender which is able to write log events into Redis. The Azure Redis Cache is used as a middleware between the Log Appenders and Logstash service. This is quite scalable and it also decouples the Log Appenders from Elastic Stack service. You can use any Log Appender which is able to write log events into Redis.
@ -93,7 +119,7 @@ You have to store one of the Redis Keys in Key Vault.
az keyvault secret set --vault-name <KEYVAULT_NAME> --name logstash-dev-redis-key --value=<REDIS_KEY> az keyvault secret set --vault-name <KEYVAULT_NAME> --name logstash-dev-redis-key --value=<REDIS_KEY>
``` ```
## Application for Azure Active Directory ### Application for Azure Active Directory
An Azure Active Directory application of type _Web app/API_ is required in order to use the AAD as an identity provider for Kibana. The authentication is provided by [oauth2_proxy](https://github.com/bitly/oauth2_proxy) reverse proxy which is deployed in the same POD as Kibana. An Azure Active Directory application of type _Web app/API_ is required in order to use the AAD as an identity provider for Kibana. The authentication is provided by [oauth2_proxy](https://github.com/bitly/oauth2_proxy) reverse proxy which is deployed in the same POD as Kibana.
@ -119,8 +145,7 @@ az keyvault secret set --name kibana-oauth-cookie-secret --vault-name <KEYVAULT
You should also update the access list with the emails of the users from your organization which are allowed to access Kibana. The white list is in [oauth2-proxy-config-secret.yaml](charts/kibana-logstash/templates/secrets/oauth2-proxy-config-secret.yaml) file. You should also update the access list with the emails of the users from your organization which are allowed to access Kibana. The white list is in [oauth2-proxy-config-secret.yaml](charts/kibana-logstash/templates/secrets/oauth2-proxy-config-secret.yaml) file.
## Microsoft Teams/Slack incoming Webhook ### Microsoft Teams/Slack incoming Webhook
The [Elasticsearch Watcher](https://www.elastic.co/guide/en/elasticsearch/reference/master/watcher-api.html) can post notifications into a webhook. For example, you can use a Microsoft Teams webhook, which can be created following these [instructions](https://docs.microsoft.com/en-us/microsoftteams/platform/concepts/connectors). The [Elasticsearch Watcher](https://www.elastic.co/guide/en/elasticsearch/reference/master/watcher-api.html) can post notifications into a webhook. For example, you can use a Microsoft Teams webhook, which can be created following these [instructions](https://docs.microsoft.com/en-us/microsoftteams/platform/concepts/connectors).
@ -132,9 +157,9 @@ az keyvault secret set --vault-name <KEYVAULT_NAME> -n elasticsearch-watcher-web
If you want instead to use a [Slack Incoming Webhook](https://api.slack.com/incoming-webhooks), you can adjust the configuration in the [post-install-watches-secret.yaml](charts/kibana-logstash/templates/post-install-watches-secret.yaml) file. If you want instead to use a [Slack Incoming Webhook](https://api.slack.com/incoming-webhooks), you can adjust the configuration in the [post-install-watches-secret.yaml](charts/kibana-logstash/templates/post-install-watches-secret.yaml) file.
# Customize Logstash Configuration ## Customize Logstash Configuration
## Multiple Data Pipelines ### Multiple Data Pipelines
Multiple data pipelines can be defined in the [values.yaml](charts/kibana-logstash/environments/acs/values.yaml) file by creating multiple `stunnel` connections as follows: Multiple data pipelines can be defined in the [values.yaml](charts/kibana-logstash/environments/acs/values.yaml) file by creating multiple `stunnel` connections as follows:
@ -159,48 +184,48 @@ stunnel:
port: 6378 port: 6378
``` ```
## Indexes Clean Up ### Indexes Clean Up
The old indexes are cleaned up by the [Curator](https://github.com/elastic/curator) tool which is executed daily by a cron job. Its configuration is available in [curator-actions.yaml](charts/kibana-logstash/templates/config/curator-actions.yaml) file. You should adjust it according with your needs. The old indexes are cleaned up by the [Curator](https://github.com/elastic/curator) tool which is executed daily by a cron job. Its configuration is available in [curator-actions.yaml](charts/kibana-logstash/templates/config/curator-actions.yaml) file. You should adjust it according to your needs.
## Index Templates ### Index Templates
The [Elasticsearch Index Templates](https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-templates.html) are installed automatically by a pre-install job. They are defined in the [pre-install-templates-config.yaml](charts/kibana-logstash/templates/pre-install-templates-config.yaml) file. The [Elasticsearch Index Templates](https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-templates.html) are installed automatically by a pre-install job. They are defined in the [pre-install-templates-config.yaml](charts/kibana-logstash/templates/pre-install-templates-config.yaml) file.
## Index Watches ### Index Watches
The [Elasticsearch Watches](https://www.elastic.co/guide/en/elasticsearch/reference/master/watcher-api.html) are also installed automatically by a post-install job. They can be used to trigger any alert or notification based on search queries. The watches configuration is available in [post-install-watches-secret.yaml](charts/kibana-logstash/templates/post-install-watches-secret.yaml) file. You should update this configuration according with you needs.
## Elasticsearch License The [Elasticsearch Watches](https://www.elastic.co/guide/en/elasticsearch/reference/master/watcher-api.html) are also installed automatically by a post-install job. They can be used to trigger any alert or notification based on search queries. The watches configuration is available in [post-install-watches-secret.yaml](charts/kibana-logstash/templates/post-install-watches-secret.yaml) file. You should update this configuration according to your needs.
### Elasticsearch License
In case you have an [Elasticsearch x-pack license](https://license.elastic.co/download), you can install it when [elasticsearch chart](charts/elasticsearch/README.md) is deployed. In case you have an [Elasticsearch x-pack license](https://license.elastic.co/download), you can install it when [elasticsearch chart](charts/elasticsearch/README.md) is deployed.
# Installation ## Installation
## NGINX Ingress Controller ### NGINX Ingress Controller
The `nginx-ingress` will act as a frontend load balancer and it will provide TLS termination for Kibana public endpoint. You can get the latest version from [kubernetes/charts/stable/nginx-ingress](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress). Before starting the installation, you have to update a few Helm values from `values.yaml` file. The `nginx-ingress` will act as a frontend load balancer and it will provide TLS termination for the Kibana public endpoint. Get the latest version from [kubernetes/charts/stable/nginx-ingress](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress). Before starting the installation, updating e a few Helm values from `values.yaml` file is necessary.
You should enable the Kubernetes RBAC by setting: Enable the Kubernetes RBAC by setting:
```console ```console
rbac.create=true rbac.create=true
``` ```
And set your static public IP allocated in Azure, as a load balancer frontend IP: And set the static public IP allocated in Azure, as a load balancer frontend IP:
```console ```console
controller.service.loadBalancerIP: "<YOUR PUBLIC IP>" controller.service.loadBalancerIP: "<YOUR PUBLIC IP>"
``` ```
You can install now the helm package with the following commands: Install now the helm package with the following commands:
```console ```console
cd charts/stable/nginx-ingress cd charts/stable/nginx-ingress
helm install -f values.yaml -n nginx-ingress . helm install -f values.yaml -n nginx-ingress .
``` ```
After the installation is done, you should verify that your public IP is properly assigned to the controller. After the installation is done, verify that the public IP is properly assigned to the controller.
```console ```console
$> kubectl get svc nginx-ingress-nginx-ingress-controller $> kubectl get svc nginx-ingress-nginx-ingress-controller
@ -209,9 +234,9 @@ NAME TYPE CLUSTER-IP EXTERNAL-I
nginx-ingress-nginx-ingress-controller LoadBalancer 10.0.26.141 <YOUR-PUBLIC-IP> 80:32321/TCP,443:31990/TCP 10m nginx-ingress-nginx-ingress-controller LoadBalancer 10.0.26.141 <YOUR-PUBLIC-IP> 80:32321/TCP,443:31990/TCP 10m
``` ```
## Elaticsearch Cluster ### Elasticsearch Cluster
Kibana requires an Elasticsearch cluster which can be installed using the [elasticsearch chart](charts/elasticsearch/README.md). You can create a deployment using the `deploy.sh` script available in the chart. Check the [README](charts/elasticsearch/README.md) file for more details: Kibana requires an Elasticsearch cluster which can be installed using the [elasticsearch chart](charts/elasticsearch/README.md). Create a deployment using the `deploy.sh` script available in the chart. Check the [README](charts/elasticsearch/README.md) file for more details:
```console ```console
./deploy.sh -e acs -n elk ./deploy.sh -e acs -n elk
@ -219,17 +244,17 @@ Kibana requires an Elasticsearch cluster which can be installed using the [elast
The command will install an Elasticsearch cluster in the `elk` namespace using the `acs` environment variables. The command will install an Elasticsearch cluster in the `elk` namespace using the `acs` environment variables.
## Kibana and Logstash ### Kibana and Logstash
You can install now the [kibana-logstash](charts/kibana-logstash) chart using the `deploy.sh` script available in the chart. Check the [README](charts/kibana-logstash/README.md) file for more details. You can install now the [kibana-logstash](charts/kibana-logstash) chart using the `deploy.sh` script available in the chart. Check the [README](charts/kibana-logstash/README.md) file for more details.
```console ```console
./deploy.sh -n elk -d <DOMAIN> -v <KEYVAULT_NAME> ./deploy.sh -n elk -d <DOMAIN> -v <KEYVAULT_NAME>
``` ```
> Note that you have to replace the `DOMAIN` with your Kibana DNS domain and the `KEYVAULT_NAME` with your Azure Key Vault name.
This command will install Kibana and Logstash in the `elk` namespace using the `acs` environment variables. If everything works well, you should see the following output: > Note to replace the `DOMAIN` with the Kibana DNS domain and the `KEYVAULT_NAME` with the Azure Key Vault name.
This command installs Kibana and Logstash in the `elk` namespace using the `acs` environment variables. If everything works well, the following output should be shown:
```console ```console
Checking az command Checking az command
@ -245,7 +270,7 @@ Installing mse-elk helm chart
Done Done
``` ```
And your deployment should look like this: And the deployment should look like this:
```console ```console
$> kubectl get pods --namespace elk $> kubectl get pods --namespace elk
@ -273,16 +298,15 @@ nginx-ingress-nginx-ingress-controller-7f7488c7c7-wkx42 1/1 Running
nginx-ingress-nginx-ingress-default-backend-7c8bbc9879-cvl79 1/1 Running 0 1h nginx-ingress-nginx-ingress-default-backend-7c8bbc9879-cvl79 1/1 Running 0 1h
``` ```
# Rolling Update ### Rolling Update
You can upgrade the charts after the initial installation whenever you have a change, by simply executing again the deployment scripts with the same arguments. Helm will create a new release for you. You can upgrade the charts after the initial installation whenever you have a change, by simply executing again the deployment scripts with the same arguments. Helm will create a new release for you.
## Contributing
# Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit https://cla.microsoft.com. the rights to use your contribution. For details, visit [https://cla.microsoft.com](https://cla.microsoft.com).
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide When you submit a pull request, a CLA-bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions

7
charts/elasticsearch/README.md
Просмотреть файл

@ -1,3 +1,5 @@
# Elasticsearch helm chart
## Introduction ## Introduction
This chart bootstraps an [Elasticsearch cluster](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html) on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. This chart bootstraps an [Elasticsearch cluster](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html) on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager.
@ -5,7 +7,8 @@ This chart bootstraps an [Elasticsearch cluster](https://www.elastic.co/guide/en
It is based on [clockworksoul/helm-elasticsearch](https://github.com/clockworksoul/helm-elasticsearch) chart. It is based on [clockworksoul/helm-elasticsearch](https://github.com/clockworksoul/helm-elasticsearch) chart.
## Prerequisites ## Prerequisites
- Kubernetes 1.8+ e.g. deployed with [Azure Container Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes)
- Kubernetes 1.8+ e.g. deployed with [Azure Container Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes)
## Configuration ## Configuration
@ -14,7 +17,7 @@ The following table lists some of the configurable parameters of the `elasticsea
| Parameter | Description | Default | | Parameter | Description | Default |
| ---------------------------------------------- | ---------------------------------------- | --------------------------------------------------------- | | ---------------------------------------------- | ---------------------------------------- | --------------------------------------------------------- |
| `image.repository` | elasticsearch image with Azure repository | `mseoss/elasticsearch` | | `image.repository` | elasticsearch image with Azure repository | `mseoss/elasticsearch` |
| `image.tag` | specify image tag | `6.2.4` | | `image.tag` | specify image tag | `6.4.3` |
| `image.pullSecrets` | specify image pull secrets | `nil` (does not add image pull secrets to deployed pods) | | `image.pullSecrets` | specify image pull secrets | `nil` (does not add image pull secrets to deployed pods) |
| `image.pullPolicy` | image pull policy | `Always` | | `image.pullPolicy` | image pull policy | `Always` |
| `license.install` | install x-pack license | `false` | | `license.install` | install x-pack license | `false` |

2
charts/elasticsearch/environments/acs/values.yaml
Просмотреть файл

@ -1,6 +1,6 @@
image: image:
repository: mseoss/elasticsearch repository: mseoss/elasticsearch
tag: 6.2.4 tag: 6.4.3
pullPolicy: Always pullPolicy: Always
license: license:

87
charts/kibana-logstash/README.md
Просмотреть файл

@ -1,55 +1,58 @@
# Kibana-Logstash helm chart
## Introduction ## Introduction
This chart bootstraps [Kibana](https://www.elastic.co/guide/en/kibana/current/index.html) with [Logstash](https://www.elastic.co/guide/en/logstash/current/index.html) on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. This chart bootstraps [Kibana](https://www.elastic.co/guide/en/kibana/current/index.html) with [Logstash](https://www.elastic.co/guide/en/logstash/current/index.html) on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager.
## Prerequisites ## Prerequisites
- Kubernetes 1.8+ (e.g. deployed with [Azure Container Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes))
- Kubernetes 1.8+ (e.g. deployed with [Azure Container Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes))
## Configuration ## Configuration
The following table lists some of the configurable parameters of the `kibana-logstash` chart and their default values: The following table lists some of the configurable parameters of the `kibana-logstash` chart and their default values:
| Parameter | Description | Default | | Parameter | Description | Default |
| ---------------------------------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------- | |---------------------------------------|---------------------------------------------------------------------|------------------------------------------------------------------------|
| `image.pullPolicy` | General image pull policy | `Always` | | `image.pullPolicy` | General image pull policy | `Always` |
| `image.pullSecrets` | General image pull secrets | `nil` (does not add image pull secrets to deployed pods) | | `image.pullSecrets` | General image pull secrets | `nil` (does not add image pull secrets to deployed pods) |
| `kibana.image.repository` | Kibana image | `docker.elastic.co/kibana/kibana` | | `kibana.image.repository` | Kibana image | `docker.elastic.co/kibana/kibana` |
| `kibana.image.tag` | Kibana image tag | `6.2.4` | | `kibana.image.tag` | Kibana image tag | `6.4.3` |
| `kibana.replicas` | Number of Kibana instances started | `3` | | `kibana.replicas` | Number of Kibana instances started | `3` |
| `kibana.ingress.host` | Kibana DNS domain | `nil` (must be provided during installation) | | `kibana.ingress.host` | Kibana DNS domain | `nil` (must be provided during installation) |
| `kibana.ingress.public.cert` | Kibana public TLS certificate | `nil` (must be provided during installation) | | `kibana.ingress.public.cert` | Kibana public TLS certificate | `nil` (must be provided during installation) |
| `kibana.ingress.private.key` | Kibana private TLS key | `nil` (must be provided during installation) | | `kibana.ingress.private.key` | Kibana private TLS key | `nil` (must be provided during installation) |
| `logstash.image.repository` | Logstash image | `mseoss/logstash` | | `logstash.image.repository` | Logstash image | `mseoss/logstash` |
| `logstah.image.tag` | Logstash image tag | `6.2.4` | | `logstash.image.tag` | Logstash image tag | `6.4.3` |
| `logstah.replicas` | Number of Logtash instances started | `3` | | `logstash.replicas` | Number of Logtash instances started | `3` |
| `logstah.queue.storageclass` | Storage class used for Logstash queue PV | `default` | | `logstash.queue.storageclass` | Storage class used for Logstash queue PV | `default` |
| `logstah.queue.distk_capacity` | Disk capacity of Logstash queue PV | `50Gi` | | `logstash.queue.distk_capacity` | Disk capacity of Logstash queue PV | `50Gi` |
| `stunnel.image.repository` | Stunnel image | `mseoss/stunnel` | | `stunnel.image.repository` | Stunnel image | `mseoss/stunnel` |
| `stunnel.image.tag` | Stunnel image tag | `5.44` | | `stunnel.image.tag` | Stunnel image tag | `5.44` |
| `stunnel.connections.dev.redis.host` | Address of Redis where the logs for `dev` environment are cached | `dev-logschache.redis.cache.windows.net` | | `stunnel.connections.dev.redis.host` | Address of Redis where the logs for `dev` environment are cached | `dev-logschache.redis.cache.windows.net` |
| `stunnel.connections.dev.redis.port` | Port of Redis where logs for `dev` environment are cached | `6380` | | `stunnel.connections.dev.redis.port` | Port of Redis where logs for `dev` environment are cached | `6380` |
| `stunnel.connections.dev.redis.key` | Key of Redis where logs for `dev` environment are cached | `nil` (must be provided during installation) | | `stunnel.connections.dev.redis.key` | Key of Redis where logs for `dev` environment are cached | `nil` (must be provided during installation) |
| `stunnel.connections.dev.logcal.host` | Local host where Redis connection for `dev` environment is tunneled | `127.0.0.1` | | `stunnel.connections.dev.logcal.host` | Local host where Redis connection for `dev` environment is tunneled | `127.0.0.1` |
| `stunnel.connections.dev.logcal.port` | Local port where Redis connection for `dev` environment is tunneled | `6379` | | `stunnel.connections.dev.logcal.port` | Local port where Redis connection for `dev` environment is tunneled | `6379` |
| `oauth.image.repository` | oauth2_proxy image | `mseoss/oauth2_proxy` | | `oauth.image.repository` | oauth2_proxy image | `mseoss/oauth2_proxy` |
| `oauth.image.tag` | oauth2_proxy image tag | `v2.2` | | `oauth.image.tag` | oauth2_proxy image tag | `v2.2` |
| `oauth.client.id` | Azure AD application ID | `nil` (must be provided during installation) | | `oauth.client.id` | Azure AD application ID | `nil` (must be provided during installation) |
| `oauth.client.secret` | Azure AD application secret | `nil` (must be provided during installation) | | `oauth.client.secret` | Azure AD application secret | `nil` (must be provided during installation) |
| `oauth.cookie.secret` | Secrete used to sign the Kibana SSO cookie | `nil` (must be provided during installation) | | `oauth.cookie.secret` | Secrete used to sign the Kibana SSO cookie | `nil` (must be provided during installation) |
| `oauth.cookie.expire` | Kibana SSO cookie expiration time | `168h0m` | | `oauth.cookie.expire` | Kibana SSO cookie expiration time | `168h0m` |
| `oauth.cookie.refresh` | Kibana SSO cookie refresh time | `60m` | | `oauth.cookie.refresh` | Kibana SSO cookie refresh time | `60m` |
| `curator.image.repository` | Curator image | `docker.io/bobrik/curator` | | `curator.image.repository` | Curator image | `docker.io/bobrik/curator` |
| `curator.image.tag` | Curator image tag | `latest` | | `curator.image.tag` | Curator image tag | `latest` |
| `curator.install` | Indicates if curator cron job is created | `true` | | `curator.install` | Indicates if curator cron job is created | `true` |
| `curator.image.index_prefix` | Prefix of the index over which curator runs | `dev` (should be the same like stunnel.connection.[env] | | `curator.image.index_prefix` | Prefix of the index over which curator runs | `dev` (should be the same like stunnel.connection.[env] |
| `templates.image.repository` | Elastic template tool image | `mseoss/elastictemplate` | | `templates.image.repository` | Elastic template tool image | `mseoss/elastictemplate` |
| `templates.image.tag` | Elastic template image tag | `latest` | | `templates.image.tag` | Elastic template image tag | `latest` |
| `templates.image.install` | Indicates if elastic template pre-install job is executed | `true` | | `templates.image.install` | Indicates if elastic template pre-install job is executed | `true` |
| `watcher.image.repository` | Elastic watcher tool image | `mseoss/elasticwatcher` | | `watcher.image.repository` | Elastic watcher tool image | `mseoss/elasticwatcher` |
| `watcher.image.tag` | Elastic watcher image tag | `latest` | | `watcher.image.tag` | Elastic watcher image tag | `latest` |
| `watcher.image.install` | Indicates if elastic watcher post-install job is executed | `true` | | `watcher.image.install` | Indicates if elastic watcher post-install job is executed | `true` |
| `watcher.webhooks.teams` | Microsoft teams webhook (watcher will post here the alerts) | `nil` (must be provided during installation) | | `watcher.webhooks.teams` | Microsoft teams webhook (watcher will post here the alerts) | `nil` (must be provided during installation) |
| `watcher.indices` | Index prefixes where watches will be executed | ``"dev-logstash-*\"`(env prefix the same like stunnel.connection.[env] | | `watcher.indices` | Index prefixes where watches will be executed | ``"dev-logstash-*\"`(env prefix the same like stunnel.connection.[env] |
> Note that you can define multiple Redis connections. The helm chart will create a Logstash data pipeline for each of connection. > Note that you can define multiple Redis connections. The helm chart will create a Logstash data pipeline for each of connection.

4
charts/kibana-logstash/environments/acs/values.yaml
Просмотреть файл

@ -4,7 +4,7 @@ image:
kibana: kibana:
image: image:
repository: docker.elastic.co/kibana/kibana repository: docker.elastic.co/kibana/kibana
tag : 6.2.4 tag : 6.4.3
replicas: 3 replicas: 3
env_var: env_var:
ELASTICSEARCH_URL: "http://elasticsearch:9200" ELASTICSEARCH_URL: "http://elasticsearch:9200"
@ -28,7 +28,7 @@ kibana:
logstash: logstash:
image: image:
repository: mseoss/logstash repository: mseoss/logstash
tag : 6.2.4 tag : 6.4.3
replicas: 3 replicas: 3
queue: queue:
storageclass: default storageclass: default

3
dockerfiles/elasticsearch/Dockerfile
Просмотреть файл

@ -1,6 +1,5 @@
ARG VERSION=latest ARG VERSION=latest
FROM docker.elastic.co/elasticsearch/elasticsearch:${VERSION} FROM docker.elastic.co/elasticsearch/elasticsearch:${VERSION}
# Install Plugins # Install Plugins
RUN elasticsearch-plugin install repository-azure --verbose RUN elasticsearch-plugin install repository-azure --verbose --batch

15
dockerfiles/elasticsearch/README.md
Просмотреть файл

@ -3,24 +3,23 @@
Docker image based on official [Elasticsearch image](https://www.elastic.co/guide/en/logstash/current/docker.html) with some extra plugins installed. Docker image based on official [Elasticsearch image](https://www.elastic.co/guide/en/logstash/current/docker.html) with some extra plugins installed.
Plugins: Plugins:
* repository-azure
# Build * `repository-azure` - [Azure Repository Plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-azure.html#repository-azure)
## Build
The `VERSION` environment variable defines the version of the elasticsearch base image. The `VERSION` environment variable defines the version of the elasticsearch base image.
``` ```bash
export VERSION=6.2.4 export VERSION=6.4.3
docker build -t mseoss/elasticsearch:${VERSION} --build-arg VERSION=${VERSION} . docker build -t mseoss/elasticsearch:${VERSION} --build-arg VERSION=${VERSION} .
docker push mseoss/elasticsearch:${VERSION} docker push mseoss/elasticsearch:${VERSION}
``` ```
# Run ## Run
Elasticsearch image can be started for testing with the following command: Elasticsearch image can be started for testing with the following command:
``` ```bash
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" mseoss/elasticsearch:${VERSION} docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" mseoss/elasticsearch:${VERSION}
``` ```

11
dockerfiles/logstash/README.md
Просмотреть файл

@ -3,25 +3,26 @@
Docker image based on official [Logstash image](https://www.elastic.co/guide/en/logstash/current/docker.html) with some extra plugins installed. Docker image based on official [Logstash image](https://www.elastic.co/guide/en/logstash/current/docker.html) with some extra plugins installed.
Plugins: Plugins:
* logstash-input-redis * logstash-input-redis
* logstash-output-elasticsearch * logstash-output-elasticsearch
* logstash-output-csv * logstash-output-csv
# Build ## Build
The `VERSION` environment variable defines the version of the logstash base image. The `VERSION` environment variable defines the version of the logstash base image.
``` ```bash
export VERSION=6.2.4 export VERSION=6.4.3
docker build -t mseoss/logstash:${VERSION} --build-arg VERSION=${VERSION} . docker build -t mseoss/logstash:${VERSION} --build-arg VERSION=${VERSION} .
docker push mseoss/logstash:${VERSION} docker push mseoss/logstash:${VERSION}
``` ```
# Run ## Run
Logstash image can be started for testing with the following command: Logstash image can be started for testing with the following command:
``` ```bash
docker run --rm -it -v ~/pipeline/:/usr/share/logstash/pipeline/ mseoss/logstash:${VERSION} docker run --rm -it -v ~/pipeline/:/usr/share/logstash/pipeline/ mseoss/logstash:${VERSION}
``` ```