зеркало из https://github.com/microsoft/MonitoFi.git
Rebrand to MonitoFi
This commit is contained in:
Родитель
e1272be362
Коммит
4be8855319
|
@ -16,7 +16,7 @@ on:
|
|||
- main
|
||||
|
||||
env:
|
||||
IMAGE_NAME: mifi
|
||||
IMAGE_NAME: monitofi
|
||||
|
||||
jobs:
|
||||
# See also https://docs.docker.com/docker-hub/builds/automated-testing/
|
||||
|
@ -98,7 +98,7 @@ jobs:
|
|||
with:
|
||||
push: true
|
||||
tags: |
|
||||
dtushar/mifi:latest
|
||||
dtushar/monitofi:latest
|
||||
-
|
||||
name: Image digest
|
||||
run: echo ${{ steps.docker_build.outputs.digest }}
|
||||
|
|
|
@ -17,4 +17,4 @@ jobs:
|
|||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
repository: dtushar/mifi
|
||||
repository: dtushar/monitofi
|
||||
|
|
14
Dockerfile
14
Dockerfile
|
@ -2,16 +2,16 @@
|
|||
# // Licensed under the MIT license.
|
||||
FROM python:3.7-slim
|
||||
|
||||
RUN mkdir /opt/nifimonitor
|
||||
RUN mkdir /opt/monitofi
|
||||
|
||||
ENV VIRTUAL_ENV=/opt/nifimonitor/nifimonitor-env
|
||||
ENV VIRTUAL_ENV=/opt/monitofi/monitofi-env
|
||||
RUN python -m venv $VIRTUAL_ENV
|
||||
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
|
||||
|
||||
COPY ./requirements.txt /opt/nifimonitor/requirements.txt
|
||||
RUN pip install -r /opt/nifimonitor/requirements.txt
|
||||
COPY ./requirements.txt /opt/monitofi/requirements.txt
|
||||
RUN pip install -r /opt/monitofi/requirements.txt
|
||||
|
||||
WORKDIR /opt/nifimonitor
|
||||
COPY ./nifi_monitor.py /opt/nifimonitor/nifi_monitor.py
|
||||
WORKDIR /opt/monitofi
|
||||
COPY ./monitofi.py /opt/monitofi/monitofi.py
|
||||
|
||||
CMD ["python", "/opt/nifimonitor/nifi_monitor.py"]
|
||||
CMD ["python", "/opt/monitofi/monitofi.py"]
|
16
README.md
16
README.md
|
@ -27,7 +27,7 @@
|
|||
Using a Simple Instrumentation Key Received when creating an Application Insights Resource in Azure, all the MonitoFi monitoring data can be pushed to Azure. This data can be viewed using Grafana by importing included AIDashboard. Queries for data in Application Insights are written using Kusto Query Language and are easy to modify.
|
||||
|
||||
Visit https://github.com/tushardhadiwal/docker-influxdb-grafana for Grafana And InfludDB Support.\
|
||||
Visit https://github.com/microsoft/MiFi for MonitoFi Source Code.
|
||||
Visit https://github.com/microsoft/MonitoFi for MonitoFi Source Code.
|
||||
|
||||
## Architecture
|
||||
|
||||
|
@ -35,9 +35,9 @@
|
|||
|
||||
#### Quick Start
|
||||
|
||||
Run following commands on machine which has access to NiFi cluster:
|
||||
Run following commands on machine which has access to Apache NiFi cluster:
|
||||
```sh
|
||||
git clone git@github.com:microsoft/MiFi.git
|
||||
git clone git@github.com:microsoft/MonitoFi.git
|
||||
```
|
||||
Fully Automated Script will take care of running InfluxDB & Grafana container , configuring datasources in grafana, importing pre included dashboards in grafana, Adding notification channels if needed, Running MonitoFi container against your cluster. Please set variables at the top of the script to desired state.
|
||||
```sh
|
||||
|
@ -63,8 +63,8 @@ docker run -d \
|
|||
-p 3003:3003 \
|
||||
-p 3004:8083 \
|
||||
-p 8086:8086 \
|
||||
-v /home/$(whoami)/nifimonitor/influx:/var/lib/influxdb \
|
||||
-v /home/$(whoami)/nifimonitor/grafana:/var/lib/grafana \
|
||||
-v /home/$(whoami)/monitofi/influx:/var/lib/influxdb \
|
||||
-v /home/$(whoami)/monitofi/grafana:/var/lib/grafana \
|
||||
dtushar/docker-influxdb-grafana:latest
|
||||
```
|
||||
You can now visit http://localhost:3003 to access Grafana with following credentials.
|
||||
|
@ -84,20 +84,20 @@ Run this command to run MonitoFi against your NiFi Cluster :
|
|||
|
||||
```sh
|
||||
docker run \
|
||||
--name=mifi1 \
|
||||
--name=monitofi1 \
|
||||
--network=host -d \
|
||||
-e INFLUXDB_SERVER="influxdb-grafana" \
|
||||
-e ENDPOINT_LIST="controller/cluster,flow/cluster/summary,flow/process-groups/root,flow/status,counters,system-diagnostics" \
|
||||
-e SLEEP_INTERVAL=300 \
|
||||
-e API_URL='http://localhost:8080/nifi-api/' \
|
||||
--restart unless-stopped \
|
||||
dtushar/mifi:1.0
|
||||
dtushar/monitofi:1.0
|
||||
```
|
||||
|
||||
If your NiFi Cluster is SECURE and supports login via certificate then please add the following to the above command, this will mount the certificate into MonitoFi Container:
|
||||
```sh
|
||||
-e SECURE=True \
|
||||
-v $(pwd)/keystore.pkcs12:/opt/nifimonitor/cert.pkcs12 \
|
||||
-v $(pwd)/keystore.pkcs12:/opt/monitofi/cert.pkcs12 \
|
||||
-e CERT_PASS="PasswordForCertificate" \
|
||||
```
|
||||
|
||||
|
|
6
clean.sh
6
clean.sh
|
@ -1,8 +1,8 @@
|
|||
#!/bin/bash
|
||||
# // Copyright (c) Microsoft Corporation.
|
||||
# // Licensed under the MIT license.
|
||||
sudo rm -rf /home/$(whoami)/nifimonitor
|
||||
sudo rm -rf /home/$(whoami)/monitofi
|
||||
docker stop influxdb-grafana
|
||||
docker rm influxdb-grafana
|
||||
docker stop mifi1
|
||||
docker rm mifi1
|
||||
docker stop monitofi1
|
||||
docker rm monitofi1
|
18
deploy.sh
18
deploy.sh
|
@ -42,12 +42,12 @@ fi
|
|||
|
||||
#Create Folders for Grafana & Influxdb
|
||||
echo "Creating necessary volumes for Grafana and InfluxDB..."
|
||||
mkdir -p /home/$(whoami)/nifimonitor/{influx,grafana}
|
||||
mkdir -p /home/$(whoami)/monitofi/{influx,grafana}
|
||||
|
||||
#Run Grafana & InfluxDB Container
|
||||
#Note in below command since host netowrking is used, -p flags are ignored. They are kept there to show user which ports are running applications. User can access them with localhost:PORT on host machine directly.
|
||||
echo "Launching InfluxDB & Grafana Container"
|
||||
docker run -d --name influxdb-grafana --network=host -p 3003:3003 -p 3004:8083 -p 8086:8086 -v /home/$(whoami)/nifimonitor/influx:/var/lib/influxdb -v /home/$(whoami)/nifimonitor/grafana:/var/lib/grafana dtushar/docker-influxdb-grafana:latest
|
||||
docker run -d --name influxdb-grafana --network=host -p 3003:3003 -p 3004:8083 -p 8086:8086 -v /home/$(whoami)/monitofi/influx:/var/lib/influxdb -v /home/$(whoami)/monitofi/grafana:/var/lib/grafana dtushar/docker-influxdb-grafana:latest
|
||||
|
||||
#This Is necessary to provide enough time to Grafana to spin up before making API Calls.
|
||||
sleep 30
|
||||
|
@ -75,19 +75,19 @@ fi
|
|||
|
||||
|
||||
if [ "$USE_AZURE" = true ] ; then
|
||||
#Run NiFi Monitor container
|
||||
echo "Running MiFi 1.0 Container with Azure AppInsights & InfluxDB Enabled..."
|
||||
#Run MonitoFi container
|
||||
echo "Running MonitoFi 1.0 Container with Azure AppInsights & InfluxDB Enabled..."
|
||||
echo "Please Make sure to add Instrumentation Key for your Application Insights Resource in the IKEY Variable"
|
||||
docker run --name=mifi1 --network=host -d -e INFLUXDB_SERVER="localhost" -e ENDPOINT_LIST="controller/cluster,flow/cluster/summary,flow/process-groups/root,flow/status,counters,system-diagnostics,system-diagnostics?nodewise=true" -e SLEEP_INTERVAL=300 -e API_URL=$NiFi_API_URL -e IKEY=$AppInsights_InstrumentationKey --restart unless-stopped dtushar/mifi:1.0
|
||||
docker run --name=monitofi1 --network=host -d -e INFLUXDB_SERVER="localhost" -e ENDPOINT_LIST="controller/cluster,flow/cluster/summary,flow/process-groups/root,flow/status,counters,system-diagnostics,system-diagnostics?nodewise=true" -e SLEEP_INTERVAL=300 -e API_URL=$NiFi_API_URL -e IKEY=$AppInsights_InstrumentationKey --restart unless-stopped dtushar/monitofi:1.0
|
||||
elif [ "$USE_AZURE" = false ] ; then
|
||||
#Run NiFi Monitor container
|
||||
echo "Running MiFi 1.0 Container..."
|
||||
docker run --name=mifi1 --network=host -d -e INFLUXDB_SERVER="localhost" -e ENDPOINT_LIST="controller/cluster,flow/cluster/summary,flow/process-groups/root,flow/status,counters,system-diagnostics,system-diagnostics?nodewise=true" -e SLEEP_INTERVAL=300 -e API_URL=$NiFi_API_URL --restart unless-stopped dtushar/mifi:1.0
|
||||
#Run MonitoFi container
|
||||
echo "Running MonitoFi 1.0 Container..."
|
||||
docker run --name=monitofi1 --network=host -d -e INFLUXDB_SERVER="localhost" -e ENDPOINT_LIST="controller/cluster,flow/cluster/summary,flow/process-groups/root,flow/status,counters,system-diagnostics,system-diagnostics?nodewise=true" -e SLEEP_INTERVAL=300 -e API_URL=$NiFi_API_URL --restart unless-stopped dtushar/monitofi:1.0
|
||||
fi
|
||||
|
||||
echo "Importing NiFi Monitor Dashboard to Grafana"
|
||||
#Please Adjust the queries according to number of nodes in your NiFi cluster after deployment.
|
||||
#Adding Dashboard For NiFI Monitor
|
||||
#Adding Dashboard For MonitoFi
|
||||
curl -X "POST" "http://localhost:3003/api/dashboards/db" -H "Content-Type: application/json" --user $GRAFANA_USERNAME:$GRAFANA_PASSWORD --data-binary @Grafana/Dashboards/NiFiMonitorDashboard.json | jq
|
||||
|
||||
if [ "$USE_AZURE" = true ] ; then
|
||||
|
|
2
env.list
2
env.list
|
@ -4,7 +4,7 @@ SLEEP_INTERVAL=300
|
|||
MODE ="unlimited" # In limited mode, only NUMBEROFITERATIONS API calls are made before exiting.
|
||||
NUMBER_OF_ITERATIONS = 2
|
||||
|
||||
SECURE=True #If Set to True, Please add following volume mount with certificate -v $(pwd)/keystore.pkcs12:/opt/nifimonitor/cert.pkcs12
|
||||
SECURE=True #If Set to True, Please add following volume mount with certificate -v $(pwd)/keystore.pkcs12:/opt/monitofi/cert.pkcs12 to docker run command
|
||||
CERT_PASS="PasswordForCertificate"
|
||||
|
||||
IKEY="REPLACE_ME" #AppInsights Instrumentation Key"
|
||||
|
|
|
@ -22,7 +22,7 @@ IKEY = os.getenv('IKEY', "REPLACE_ME")
|
|||
SLEEP_INTERVAL = int(os.getenv('SLEEP_INTERVAL', 300))
|
||||
|
||||
SECURE = os.getenv('SECURE', False) # Is NiFi Cluster Secure
|
||||
CERT_FILE = os.getenv('CERT_FILE', '/opt/nifimonitor/cert.pkcs12')
|
||||
CERT_FILE = os.getenv('CERT_FILE', '/opt/monitofi/cert.pkcs12')
|
||||
CERT_PASS = os.getenv('CERT_PASS', 'REPLACE_ME')
|
||||
|
||||
INFLUXDB_SERVER = os.getenv('INFLUXDB_SERVER', "127.0.0.1") # IP or hostname to InfluxDB server
|
||||
|
@ -39,7 +39,7 @@ conditions = {
|
|||
}
|
||||
|
||||
# Sysout Logging Setup
|
||||
logger = logging.getLogger("nifi-monitor")
|
||||
logger = logging.getLogger("monitofi")
|
||||
logger.setLevel(logging.INFO)
|
||||
syshandler = logging.StreamHandler(sys.stdout)
|
||||
syshandler.setLevel(logging.INFO)
|
Загрузка…
Ссылка в новой задаче