зеркало из https://github.com/microsoft/pai.git
generate pai_service_name label from app label (#1056)
This commit is contained in:
Родитель
b7619deb4b
Коммит
5f6c3b30f9
|
@ -65,6 +65,9 @@ data:
|
|||
- source_labels: [__meta_kubernetes_pod_name]
|
||||
action: replace
|
||||
target_label: k8s_pod_name
|
||||
- source_labels: [__meta_kubernetes_pod_label_app]
|
||||
action: replace
|
||||
target_label: pai_service_name
|
||||
{% if clusterinfo['prometheusinfo']['alerting'] %}
|
||||
alerting:
|
||||
alertmanagers:
|
||||
|
|
|
@ -148,4 +148,66 @@ If service decide to leverage node-exporter to expose metrics, service should mo
|
|||
located in `/datastorage/prometheus` and write file with extension `.prom`.
|
||||
|
||||
For expose using port, service should annotate pod with `prometheus.io/scrape`, `prometheus.io/path`
|
||||
and `prometheus.io/port`.
|
||||
and `prometheus.io/port`. For service with label key `app`, prometheus will generate `pai_service_name`
|
||||
label for generated metrics.
|
||||
|
||||
Prometheus will list all pods and get those pods with `prometheus.io/scrape` is true, and then,
|
||||
prometheus will issue request to `http://${pod_host_ip}:${prometheus.io/port}${prometheus.io/path}` to
|
||||
get metrics.
|
||||
|
||||
For example, following deployment:
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: foo
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app: foo
|
||||
template:
|
||||
metadata:
|
||||
name: foo
|
||||
labels:
|
||||
app: foo
|
||||
annotations:
|
||||
prometheus.io/scrape: "true"
|
||||
prometheus.io/path: "/"
|
||||
prometheus.io/port: "80"
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx
|
||||
ports:
|
||||
- containerPort: 80
|
||||
volumeMounts:
|
||||
- name: workdir
|
||||
mountPath: /usr/share/nginx/html
|
||||
initContainers:
|
||||
- name: install
|
||||
image: busybox
|
||||
command: ["sh", "-c", "echo pai_service_count 1 > /work-dir/index.html"]
|
||||
volumeMounts:
|
||||
- name: workdir
|
||||
mountPath: "/work-dir"
|
||||
volumes:
|
||||
- name: workdir
|
||||
emptyDir: {}
|
||||
hostNetwork: true
|
||||
```
|
||||
|
||||
will generate metrics like:
|
||||
|
||||
`pai_service_count{instance="10.151.40.4:80",job="pai_serivce_exporter",k8s_pod_name="foo-69869cc4d6",pai_service_name="foo"} 1`
|
||||
|
||||
metric labels `k8s_pod_name` and `pai_service_name` are generated from pod name and pod label value which has key `app`.
|
||||
|
||||
In above example, we use nginx to accept requests from prometheus and expose metrics we write in
|
||||
initContainers. It is best for service owner to use
|
||||
[prometheus client](https://prometheus.io/docs/instrumenting/clientlibs/) to expose metrics.
|
||||
|
||||
Note that, currently pai do not assume
|
||||
[kubernetes network model](https://kubernetes.io/docs/concepts/cluster-administration/networking/)
|
||||
enabled, so service has to use hostNetwork, this implies service owner should carefully choose port
|
||||
number to avoid port conflict, it is best to configure port through pai service configuration.
|
||||
|
|
Загрузка…
Ссылка в новой задаче