зеркало из https://github.com/Azure/iotedgedev.git
Sep each class into own file, add --monitor-events, beef up readme
This commit is contained in:
Jon Gallant
Родитель
66e4cc286c
Коммит
fa5c0ef192
|
|
@ -76,4 +76,6 @@ venv
|
|||
py36
|
||||
.pypirc
|
||||
test_project
|
||||
README
|
||||
README
|
||||
|
||||
node_modules
|
||||
305
README.md
305
README.md
|
|
@ -1,50 +1,106 @@
|
|||
# Azure IoT Edge Dev Tool
|
||||
|
||||
The Azure IoT Edge Dev Tool **greatly simplifies** your Azure IoT Edge development process. It has everything you need to get started and helps with your day-to-day IoT Edge development.
|
||||
The **Azure IoT Edge Dev Tool** greatly simplifies [Azure IoT Edge](https://azure.microsoft.com/en-us/services/iot-edge/) development down to simple CLI commands driven by Environment Variables. This tool will:
|
||||
|
||||
You will be able to do all of the following with simple one line commands.
|
||||
- Get you started with IoT Edge development with the [IoT Edge Dev Container](#dev-machine-setup) and IoT Edge Project Scaffolding that contains a sample module and all the required configuration files.
|
||||
- Speed up your inner-loop dev (dev, debug, test) by reducing multi-step build & deploy processes into one-line CLI commands and well as drive your outer-loop CI/CD pipeline. You can use all the same commands in both stages of your development life-cycle.
|
||||
|
||||
|
||||
1. Install the Azure IoT Edge Dev Tool: `pip install azure-iot-edge-dev-tool`
|
||||
1. Create a new Edge project: `iotedgedev project --create`
|
||||
1. Build, Push and Deploy modules: `iotedgedev modules --build --deploy`
|
||||
1. Setup and Start the Edge Runtime: `iotedgedev runtime --setup --start`
|
||||
1. View and Save Docker log files: `iotedgedev docker --logs`
|
||||
1. Use a Custom Container Registry: `iotedgedev docker --setup-registry`
|
||||
The **Azure IoT Edge Dev Tool** enables you to do all of the following with simple one-line CLI commands.
|
||||
|
||||
1. **Install**: Install the Azure IoT Edge Dev Tool:
|
||||
|
||||
`pip install azure-iot-edge-dev-tool`
|
||||
|
||||
1. **Project**: Create a new IoT Edge project that includes a sample module and all the the required configuration files.
|
||||
|
||||
`iotedgedev project --create edgeproject1`
|
||||
|
||||
1. **Build & Deploy**: Build, Push and Deploy modules:
|
||||
|
||||
`iotedgedev modules --build --deploy`
|
||||
|
||||
> This will `build`, `publish`, `docker build, tag and push` and `deploy modules` to your IoT Edge device.
|
||||
1. **Setup & Start**: Setup and Start the IoT Edge Runtime:
|
||||
|
||||
`iotedgedev runtime --setup --start`
|
||||
|
||||
1. **View Messages**: View Messages Sent from IoT Edge to IoT Hub:
|
||||
|
||||
`iotedgedev iothub --monitor-events`
|
||||
|
||||
1. **View Logs**: View and Save Docker log files:
|
||||
|
||||
`iotedgedev docker --logs`
|
||||
|
||||
1. **Setup Custom Registry**: Use a Custom Container Registry:
|
||||
|
||||
`iotedgedev docker --setup-registry`
|
||||
|
||||
The project was created by Microsofties that work with IoT Edge customers, who have found it very helpful. We hope to get a variation of this officially supported by the Azure IoT team in the near future. Your contributions and feedback to this project will help us build an amazing developer experience, so please do not hesitate to participate.
|
||||
|
||||
Please see [Azure IoT Edge Dev Resources](https://github.com/jonbgallant/azure-iot-edge-dev) for links to official docs and other IoT Edge dev information.
|
||||
|
||||
## Videos
|
||||
|
||||
### [Azure IoT Edge Dev Tool in 2 Minutes](https://www.youtube.com/watch?v=NsnxMshMhmA)
|
||||
[](https://www.youtube.com/watch?v=NsnxMshMhmA)
|
||||
|
||||
https://www.youtube.com/watch?v=NsnxMshMhmA
|
||||
|
||||
[](https://www.youtube.com/watch?v=k5ZtTmHgs_8)
|
||||
|
||||
https://www.youtube.com/watch?v=k5ZtTmHgs_8
|
||||
|
||||
### [Azure IoT Edge Dev Tool: Introduction](https://www.youtube.com/watch?v=lcDFX8PXqUQ)
|
||||
[](https://www.youtube.com/watch?v=lcDFX8PXqUQ)
|
||||
|
||||
https://www.youtube.com/watch?v=lcDFX8PXqUQ
|
||||
### [Azure IoT Edge Dev Tool with Windows Subsystem for Linux (WSL)](https://www.youtube.com/watch?v=k5ZtTmHgs_8)
|
||||
[](https://www.youtube.com/watch?v=k5ZtTmHgs_8)
|
||||
|
||||
|
||||
## Setup
|
||||
### Azure Setup
|
||||
1. [Create Azure IoT Hub](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-csharp-csharp-getstarted#create-an-iot-hub)
|
||||
1. [Create Azure Container Registry](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-get-started-portal)
|
||||
- Make sure you enable Admin Access when you create the Azure Container Registry
|
||||
1. Create Edge Device using the Azure Portal
|
||||
- In your IoT Hub, click "IoT Edge", then click "Add IoT Edge Device"
|
||||
|
||||
#### Manual Setup
|
||||
1. [**Create Azure IoT Hub**](https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-csharp-csharp-getstarted#create-an-iot-hub)
|
||||
1. **Create Edge Device** using the Azure Portal
|
||||
- In your IoT Hub, click "IoT Edge", then click "Add IoT Edge Device"
|
||||
1. [**Create Azure Container Registry**](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-get-started-portal)
|
||||
> Only needed when you want to push your modules to a central registry. You can run a local registry by setting the .env/CONTAINER_REGISTRY_SERVER setting to `localhost:5000`.
|
||||
|
||||
- Make sure you enable Admin Access when you create the Azure Container Registry
|
||||
|
||||
#### Automated Setup
|
||||
|
||||
You can also deploy the IoT Hub and Container Registry with this **Deploy to Azure** template:
|
||||
|
||||
[](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fjonbgallant%2Fazure-iot-edge-dev-tool%2Fmaster%2Fassets%2Fdeploy%2FARMDeployment%2Fazuredeploy.json)
|
||||
|
||||
### Dev Machine Setup
|
||||
|
||||
Here's what you need to do to get `iotedgedev` running on your dev machine. If you are using a separate Edge device, like a Raspberry Pi, you do not need to run all of these steps on your Edge device, you can just use `iotedgectl` directly . See the [Edge Device Setup](#edge-device-setup) section below for more information on setting up your Edge device.
|
||||
Here's what you need to do to get Azure IoT Edge Dev Tool (aka `iotedgedev`) running on your dev machine. If you are using a separate Edge device, like a Raspberry Pi, you do not need to run all of these steps on your IoT Edge device, you can just use `iotedgectl` directly on the device. See the [IoT Edge Device Setup](#iot-edge-device-setup) section below for more information on setting up your Edge device.
|
||||
|
||||
> Note: See the ["Test Coverage"](#test-coverage) section below to see what this module has been tested with.
|
||||
> Note: See the ["Test Coverage"](#test-coverage) section below to see what the Edge Dev Tool has been tested with.
|
||||
|
||||
#### IoT Edge Dev Tool Container
|
||||
You can use the IoT Edge Dev Tool container to avoid having to install all the dependencies on your local dev machine.
|
||||
|
||||
> (Only runs on Linux Containers at this time. Windows Container support coming soon.)
|
||||
|
||||
1. Install Docker
|
||||
|
||||
1. Create Local Folder on Host to contain your projects.
|
||||
|
||||
`c:/temp/edgeprojects`
|
||||
|
||||
1. Start Container:
|
||||
|
||||
`docker run -it -v /var/run/docker.sock:/var/run/docker.sock -v c:/temp/edgeprojects:/home jongallant/iotedgedev`
|
||||
|
||||
1. Change to the `/home` directory:
|
||||
|
||||
`cd /home`
|
||||
|
||||
1. Skip all the install steps below and jump to the [Usage](#usage) section below.
|
||||
|
||||
#### Local Dev Machine Setup
|
||||
|
||||
> You don't need to do this if you are running the container as described in the previous step.
|
||||
|
||||
1. Install **[Docker](https://docs.docker.com/engine/installation/)**
|
||||
- Windows
|
||||
|
|
@ -69,18 +125,16 @@ Here's what you need to do to get `iotedgedev` running on your dev machine. If y
|
|||
sudo sh -c "echo Defaults env_keep += \"DOCKER_HOST\" >> /etc/sudoers.d/docker"
|
||||
```
|
||||
- Linux
|
||||
1. **Do not** install via `sudo apt install docker.io`. Use the proper steps for [CE here](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-docker-ce), or use the [convenience script](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-using-the-convenience-script).
|
||||
|
||||
|
||||
|
||||
1. Install **Python 2.7 or Python 3**
|
||||
- We've seen some issues with docker.io. If Edge doesn't run for you, then try installing Docker CE directly instead of via docker.io. Use the [CE install steps](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-docker-ce), or use the [convenience script](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-using-the-convenience-script).
|
||||
|
||||
1. Install **Python 2.7+ or Python 3.6+**
|
||||
- Windows - [Install from Python's website](https://www.python.org/downloads/)
|
||||
- Linux - `sudo apt install python-pip` or `sudo apt install python3-pip`
|
||||
|
||||
1. Install **[.NET Core SDK](https://www.microsoft.com/net/core#windowscmd)**
|
||||
- The .NET Core SDK does not run on ARM, so you do not need to install this on Raspberry Pi.
|
||||
|
||||
1. Install **Dependencies**
|
||||
1. Install **System Dependencies**
|
||||
|
||||
> You can also run under a Python Virtual Environment. See the [Python Virtual Environment Setup](#python-virtual-environment-setup) instructions below for details on how to set that up.
|
||||
|
||||
|
|
@ -96,108 +150,76 @@ Here's what you need to do to get `iotedgedev` running on your dev machine. If y
|
|||
sudo pip install --upgrade setuptools pip
|
||||
sudo apt install python2.7-dev libffi-dev libssl-dev -y
|
||||
```
|
||||
1. Install **IoTHub-Explorer**
|
||||
|
||||
Currently, the only way to view IoT Hub Messages via CLI is with the iothub-explorer npm package. You need to install that with the following command.
|
||||
|
||||
```
|
||||
npm i -g iothub-explorer
|
||||
```
|
||||
|
||||
You can also use [Device Explorer](https://github.com/Azure/azure-iot-sdk-csharp/releases) as well, but it is Windows only. The Azure IoT team is working on porting the monitor-events functionality to the official az iot CLI - but for now we'll use the npm package.
|
||||
|
||||
1. Install **`azure-iot-edge-dev-tool`**
|
||||
|
||||
> You do not need to run this on the Raspberry Pi Edge device. See the [Edge Device Setup](#edge-device-setup) section below for more information on setting up your Edge device.
|
||||
> You do not need to run this on the Raspberry Pi Edge device. See the [IoT Edge Device Setup](#iot-edge-device-setup) section below for more information on setting up your Edge device.
|
||||
|
||||
1. Via Pip - Use this if you want to use the tool as is.
|
||||
```
|
||||
pip install azure-iot-edge-dev-tool
|
||||
```
|
||||
|
||||
1. Via GitHub - Use this if you want to make changes to this tool.
|
||||
|
||||
1. Clone or Fork this Repository
|
||||
|
||||
> Replace `project-name` with the name of your project.
|
||||
|
||||
`git clone https://github.com/jonbgallant/azure-iot-edge-dev-tool.git project-name`
|
||||
|
||||
1. Install Dependencies
|
||||
|
||||
> Use 'sudo' for Mac/Linux/RaspberryPi.
|
||||
|
||||
`pip install -U -r requirements.txt`
|
||||
|
||||
1. Install iotedgedev Module
|
||||
|
||||
The following command will make `iotedgedev` available in your terminal.
|
||||
|
||||
From the root of repo:
|
||||
|
||||
`pip install -e .`
|
||||
```
|
||||
pip install azure-iot-edge-dev-tool
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Here's how you create a project, build and deploy our modules, and then setup and start the Edge runtime.
|
||||
Here's how you create a project, build and deploy modules, and then setup and start the Edge runtime.
|
||||
|
||||
### Step 1: Create Azure IoT Edge Project
|
||||
|
||||
The following command will setup the folder structure required for this module
|
||||
|
||||
> Replace `edge-project` with the name of your project. Use `.` to create in the current folder.
|
||||
> Replace `edgeproject1` with the name of your project. Use `.` to create in the current folder.
|
||||
|
||||
```
|
||||
iotedgedev project --create edge-project
|
||||
iotedgedev project --create edgeproject1
|
||||
```
|
||||
|
||||
#### Folder Structure
|
||||
|
||||
When you create a new project, it will have the following contents:
|
||||
|
||||
1. **config folder** - Contains sample config files for both modules and runtime.
|
||||
|
||||
1. **build folder** - Contains the files outputted by the .NET Core SDK.
|
||||
|
||||
1. **modules folders** - Contains all of the modules for your Edge project.
|
||||
1. **modules folders** - Contains all of the modules for your IoT Edge project.
|
||||
- The iotedgedev module assumes that you'll structure your Dockerfiles exactly like the filter-module sample. Have a Docker folder in the root of the project, then subfolders within that to support multiple Docker files.
|
||||
|
||||
> It is important that you follow this structure or the module will not work. Please make suggestions or fork this project if you would like a different behavior.
|
||||
|
||||
1. **.env** - Contains all the required Environment Variables for your IoT Edge Project.
|
||||
|
||||
1. **.gitignore** - A .gitignore file for your IoT Edge projects.
|
||||
|
||||
1. **Configuration File Templates** - There are two files in the root of the project:
|
||||
|
||||
- `deployment.template.json` - Contains the config that is deployed to your IoT Edge device. It contains references to your modules, module routes and desired property information.
|
||||
- `runtime.template.json` - Contains the config used by your IoT Edge runtime. It contains your device connection string, your container registry settings and is used when you call the `runtime --setup` command.
|
||||
|
||||
1. **.config folder** - All expanded config files are copied to this folder and these files are used at runtime.
|
||||
|
||||
1. **build folder** - Contains the files outputted by the .NET Core SDK.
|
||||
|
||||
1. **logs folder** - Contains all the Docker log files for the Runtime and your modules.
|
||||
|
||||
1. **.env** - Contains all the required Environment Variables for Edge Project. Rename to .env and add your settings.
|
||||
|
||||
1. **.gitignore** - A suggested .gitignore file for your Edge projects.
|
||||
|
||||
### Step 2: Update Environment Variables
|
||||
|
||||
The settings used for this module are stored in a .env file in the root of your project. System or User Environment Variables take precedence over values in .env file.
|
||||
|
||||
1. Open `.env` and set variables
|
||||
|
||||
1. Runtime Home Directory
|
||||
You only need to set the `IOTHUB_CONNECTION_STRING` AND `DEVICE_CONNNECTION_STRING` settings, which you can get from the Azure Portal.
|
||||
|
||||
- For Linux/Raspberry Pi, change: `RUNTIME_HOME_DIR="/etc/azure-iot-edge"`
|
||||
```
|
||||
IOTHUB_CONNECTION_STRING=""
|
||||
DEVICE_CONNECTION_STRING=""
|
||||
```
|
||||
|
||||
1. Active Modules
|
||||
|
||||
- You can tell `iotedgedev` which modules you want to build by including them in the `ACTIVE_MODULES` setting. Comma separated.
|
||||
|
||||
1. Active Docker Directories
|
||||
|
||||
- You can tell `iotedgedev` which Docker files to build and push by including the Dockerfile's parent folder in the `ACTIVE_DOCKER_DIRS` setting.
|
||||
|
||||
1. At a minimum, you need to set the following values, which you you can get from your Azure account:
|
||||
```
|
||||
IOTHUB_NAME="iot hub name"
|
||||
IOTHUB_KEY="iot hub key"
|
||||
DEVICE_CONNECTION_STRING="edge device connection string"
|
||||
EDGE_DEVICE_ID="edge device id"
|
||||
RUNTIME_HOST_NAME="the computer name that your edge will run on"
|
||||
CONTAINER_REGISTRY_SERVER=""
|
||||
CONTAINER_REGISTRY_USERNAME=""
|
||||
CONTAINER_REGISTRY_PASSWORD=""
|
||||
```
|
||||
1. Update Config
|
||||
|
||||
If you are running on Raspberry Pi you need to use the arm32v7 Dockerfile. Open `config/modules.json`, find the `filter-module` line and replace `linux-x64` with `arm32v7`.
|
||||
|
||||
Replace this:
|
||||
`"image": "${CONTAINER_REGISTRY_SERVER}/filter-module:linux-x64-${CONTAINER_TAG}",`
|
||||
|
||||
With this:
|
||||
`"image": "${CONTAINER_REGISTRY_SERVER}/filter-module:arm32v7-${CONTAINER_TAG}",`
|
||||
> You can use the `DOTENV_FILE` Environment Variable to point to a different .env file, such as .env.integration or .env.test. This is helpful in CI/CD pipeline scenarios where you'll want to target different environments and devices to ensure all scenarios are tested.
|
||||
|
||||
### Step 3: Build and Deploy Modules
|
||||
|
||||
|
|
@ -207,11 +229,11 @@ The settings used for this module are stored in a .env file in the root of your
|
|||
iotedgedev modules --build --deploy
|
||||
```
|
||||
|
||||
The- `--build` command will build each module in the `modules` folder and push it to your container registry. The- `--deploy` command will apply the `build/modules.json` configuration file to your Edge device.
|
||||
The- `--build` command will build each module in the `modules` folder and push it to your container registry. The- `--deploy` command will apply the generated `.config/deployment.json` configuration file to your IoT Edge device.
|
||||
|
||||
You can configure what modules will be built and deployed using the `ACTIVE_MODULES` env var in the `.env` file.
|
||||
You can configure what modules will be built and deployed by using the `ACTIVE_MODULES` env var in the `.env` file. You can configure which Dockerfiles get built and deployed by using the `ACTIVE_DOCKER_DIRS` env var.
|
||||
|
||||
### Step 4: Setup and Start the Edge Runtime
|
||||
### Step 4: Setup and Start the IoT Edge Runtime
|
||||
|
||||
> Use 'sudo' for Linux/RaspberryPi
|
||||
|
||||
|
|
@ -219,26 +241,25 @@ You can configure what modules will be built and deployed using the `ACTIVE_MODU
|
|||
iotedgedev runtime --setup --start
|
||||
```
|
||||
|
||||
The- `--setup` command will apply the `/build/runtime.json` file to your Edge device. The- `--start` command will start the Edge runtime.
|
||||
The- `--setup` command will apply the `/.config/runtime.json` file to your IoT Edge device. The- `--start` command will start the IoT Edge runtime.
|
||||
|
||||
### Step 5: Monitor Messages
|
||||
|
||||
#### Device Explorer
|
||||
```
|
||||
iotedgedev iothub --monitor-events
|
||||
```
|
||||
|
||||
You can use the [Device Explorer](https://github.com/Azure/azure-iot-sdk-csharp/releases/download/2017-12-2/SetupDeviceExplorer.msi) to monitor the messages that are sent to your IoT Hub.
|
||||
|
||||
#### iothub-explorer
|
||||
|
||||
You can also use the [iothub-explorer](https://github.com/Azure/iothub-explorer) npm package:
|
||||
This will print messages sent from the device specified in DEVICE_CONNECTION_STRING. To use this command, you first need to install the [iothub-explorer](https://github.com/Azure/iothub-explorer) npm package with the following command:
|
||||
|
||||
```bash
|
||||
npm i -g iothub-explorer
|
||||
iothub-explorer --login "iothubowner connection string" monitor-events "your device id"
|
||||
```
|
||||
|
||||
> NOTE: iothub-explorer is included in the [IoT Edge Dev Tool Container](#iot-edge-dev-tool-container)
|
||||
|
||||
### Step 6: Create a new Module
|
||||
|
||||
After you have everything running from the Edge Tool project template, the next step is to develop all the custom modules you need for your scenario. Here's how you do that:
|
||||
After you have everything running from the IoT Edge Tool project template, the next step is to develop all the custom modules you need for your scenario. Here's how you do that:
|
||||
|
||||
1. Install the .NET Core Module Template
|
||||
|
||||
|
|
@ -258,8 +279,6 @@ After you have everything running from the Edge Tool project template, the next
|
|||
|
||||
Now, when you run `iotedgedev modules --build` you will see that `mymodule` is also built and pushed to your container registry.
|
||||
|
||||
> Without `iotedgedev` you would have had to do many more steps to get to this point.
|
||||
|
||||
1. Add Message Property
|
||||
|
||||
We are going to add a new custom property to the message as it is flowing through the system so we can see that our module is working.
|
||||
|
|
@ -274,7 +293,7 @@ After you have everything running from the Edge Tool project template, the next
|
|||
|
||||
1. Add Module to Config
|
||||
|
||||
1. Open `/config/modules.json`.
|
||||
1. Open `deployment.template.json`.
|
||||
1. Copy and paste the filter-module section and change filter-module to mymodule.
|
||||
|
||||
```javascript
|
||||
|
|
@ -295,7 +314,7 @@ After you have everything running from the Edge Tool project template, the next
|
|||
|
||||
By default, your new module acts as a simple pass through. It receives messages and passes them to the output. We will add `mymodule` in between our existing `sensor` and `filter` modules.
|
||||
|
||||
1. Open `/config/modules.json`
|
||||
1. Open `deployment.template.json`
|
||||
1. Replace `$edgeHub.properties.desired.routes` with the following:
|
||||
|
||||
```javascript
|
||||
|
|
@ -310,7 +329,7 @@ After you have everything running from the Edge Tool project template, the next
|
|||
|
||||
Now that we have the module created, code added, and config updated, we are going to rebuild, deploy our module.
|
||||
|
||||
> You will notice that the Edge Runtime automatically detects a new deployment, retrieves the new module, applies the new route and keeps sending messages.
|
||||
> You will notice that the IoT Edge Runtime automatically detects a new deployment, retrieves the new module, applies the new route and keeps sending messages.
|
||||
|
||||
`iotedgedev modules --build --deploy`
|
||||
|
||||
|
|
@ -318,11 +337,8 @@ After you have everything running from the Edge Tool project template, the next
|
|||
|
||||
Now when we view the messages flowing through the system, we'll see an additional 'abc' property:
|
||||
|
||||
Here's how with the [iothub-explorer](https://github.com/Azure/iothub-explorer) npm package:
|
||||
|
||||
```bash
|
||||
npm i -g iothub-explorer
|
||||
iothub-explorer --login "iothubowner connection string" monitor-events "your device id"
|
||||
iotedgedev iothub --monitor-events
|
||||
```
|
||||
|
||||
```javascript
|
||||
|
|
@ -343,7 +359,7 @@ After you have everything running from the Edge Tool project template, the next
|
|||
}
|
||||
```
|
||||
|
||||
That's all there is to it. You can now get started implementing your Edge scenario!
|
||||
That's all there is to it. You can now get started implementing your IoT Edge scenario!
|
||||
|
||||
## Commands
|
||||
The `iotedgedev` module has the following commands:
|
||||
|
|
@ -353,6 +369,11 @@ The `iotedgedev` module has the following commands:
|
|||
`iotedgedev project --help`
|
||||
- `--create TEXT` Creates a new Azure IoT Edge project. Use `--create .` to create in current folder. Use `--create TEXT` to create in a subfolder.
|
||||
|
||||
**iothub**
|
||||
|
||||
`iotedgedev iothub --help`
|
||||
- `--monitor-events` Displays events that are sent from IoT Hub device to IoT Hub.
|
||||
|
||||
**runtime**
|
||||
|
||||
`iotedgedev runtime --help`
|
||||
|
|
@ -366,7 +387,7 @@ The `iotedgedev` module has the following commands:
|
|||
|
||||
`iotedgedev modules --help`
|
||||
- `--build` Builds and pushes modules specified in ACTIVE_MODULES Environment Variable to specified container registry.
|
||||
- `--deploy` Deploys modules to Edge device using modules.json in build/config directory.
|
||||
- `--deploy` Deploys modules to Edge device using deployment.json in the /.config directory.
|
||||
|
||||
**docker**
|
||||
|
||||
|
|
@ -387,7 +408,7 @@ The `iotedgedev` module has the following commands:
|
|||
|
||||
### Setup Container Registry
|
||||
|
||||
You can also use `iotedgedev` to host the Edge runtime from your own Azure Container Registry or a Local Container Registry. Set the `.env` values for your Container Registry and run the following command. It will pull all the Edge containers from Dockerhub, tag them and upload them to the container registry you have specified in `.env`.
|
||||
You can also use `iotedgedev` to host the IoT Edge runtime from your own Azure Container Registry or a Local Container Registry. Set the `.env` values for your Container Registry and run the following command. It will pull all the IoT Edge containers from Dockerhub, tag them and upload them to the container registry you have specified in `.env`.
|
||||
|
||||
> Use 'sudo' for Linux/RaspberryPi
|
||||
|
||||
|
|
@ -399,7 +420,7 @@ iotedgedev docker --setup-registry
|
|||
### View Docker Logs
|
||||
|
||||
#### Show Logs
|
||||
The iotedgedev module also include a "Show Logs" command that will open a new command prompt for each module it finds in your Edge config. Just run the following command:
|
||||
The iotedgedev module also include a "Show Logs" command that will open a new command prompt for each module it finds in your IoT Edge config. Just run the following command:
|
||||
|
||||
> Note: I haven't figured out how to launch new SSH windows in a reliable way. It's in the backlog. For now, you must be on the desktop of the machine to run this command.
|
||||
|
||||
|
|
@ -435,22 +456,22 @@ Instead of using a cloud based container registry, you can use a local Docker re
|
|||
|
||||
`iotedgedev` will look for `localhost` in your setting and take care of the rest for you.
|
||||
|
||||
## Edge Device Setup
|
||||
## IoT Edge Device Setup
|
||||
|
||||
The `iotedgedev` module is intended to help with Edge development and doesn't necessarily need to be taken on as a dependency in production or integration environments, where you'll likely want to use the `iotedgectl` module directly. You can use `iotedgedev` to generate your runtime.json file on your dev machine, copy that to your Edge device and then use the following command to setup and start your Edge Runtime.
|
||||
The `iotedgedev` module is intended to help with IoT Edge development and doesn't necessarily need to be taken on as a dependency in production or integration environments, where you'll likely want to use the `iotedgectl` module directly. You can use `iotedgedev` to generate your runtime.json file on your dev machine, copy that to your IoT Edge device and then use the following command to setup and start your IoT Edge Runtime.
|
||||
|
||||
```
|
||||
iotedgectl setup --config-file runtime.json
|
||||
iotedgectl start
|
||||
```
|
||||
|
||||
Having said that, there's nothing stopping you from deploying `iotedgedev` to your Edge device. It may be helpful if you want to run the `iotedgedev docker --clean` command to clean up Docker containers and images. Or if you want to run `iotedgedev docker --show-logs` to see all the log files on the device or `iotedgedev docker --save-logs` to output to the LOGS_PATH directory.
|
||||
Having said that, there's nothing stopping you from deploying `iotedgedev` to your IoT Edge device. It may be helpful if you want to run the `iotedgedev docker --clean` command to clean up Docker containers and images. Or if you want to run `iotedgedev docker --show-logs` to see all the log files on the device or `iotedgedev docker --save-logs` to output to the LOGS_PATH directory.
|
||||
|
||||
> Please note that the .NET Core SDK does not support ARM, so you will not be able to run `modules --build` or `modules --deploy` directly on a Raspberry Pi.
|
||||
|
||||
### Raspberry Pi
|
||||
|
||||
Whether you use `iotedgedev` or directly use `iotedgecgtl` on the Raspberry Pi, you will still need to run the following commands before you run the Edge Runtime.
|
||||
Whether you use `iotedgedev` or directly use `iotedgecgtl` on the Raspberry Pi, you will still need to run the following commands before you run the IoT Edge Runtime.
|
||||
|
||||
```
|
||||
sudo pip install --upgrade setuptools pip
|
||||
|
|
@ -458,6 +479,15 @@ Whether you use `iotedgedev` or directly use `iotedgecgtl` on the Raspberry Pi,
|
|||
sudo pip install -U azure-iot-edge-dev-tool
|
||||
```
|
||||
|
||||
##### Config Changes
|
||||
If you are running on Raspberry Pi you need to use the arm32v7 Dockerfile. Open `deployment.template.json`, find the `filter-module` line and replace `linux-x64` with `arm32v7`.
|
||||
|
||||
Replace this:
|
||||
`"image": "${CONTAINER_REGISTRY_SERVER}/filter-module:linux-x64-${CONTAINER_TAG}",`
|
||||
|
||||
With this:
|
||||
`"image": "${CONTAINER_REGISTRY_SERVER}/filter-module:arm32v7-${CONTAINER_TAG}",`
|
||||
|
||||
## Python Virtual Environment Setup
|
||||
|
||||
You can run `iotedgedev` inside a Python Vritual Environment.
|
||||
|
|
@ -497,7 +527,7 @@ This module has been tested with the following:
|
|||
- Windows Subsystem for Linux, Ubuntu 16.04
|
||||
- Ubuntu 16.04
|
||||
- Mac Sierra 10.12.6
|
||||
- Raspberry Pi with Raspbian Stretch (**.NET Core Runtime Only**, .NET Core SDK not supported on ARM.) - You cannot use Raspberry Pi as a Edge dev machine, but it can host the Edge runtime.
|
||||
- Raspberry Pi with Raspbian Stretch (**.NET Core Runtime Only**, .NET Core SDK not supported on ARM.) - You cannot use Raspberry Pi as a IoT Edge dev machine, but it can host the IoT Edge runtime.
|
||||
- Python 2.7.13 and Python 3.6.3
|
||||
- Docker Version 17.09.1-ce-win42 (14687), Channel: stable, 3176a6a
|
||||
|
||||
|
|
@ -521,7 +551,7 @@ This module has been tested with the following:
|
|||
|
||||
1. Latest Docker Image Not Pulled from Registry
|
||||
|
||||
By design, Docker only pulls images that have been changed. When you are developing you tend to push new images on a frequent basis, which Docker will not pull unless it has a unique tag. You could assign every push a new tag, or you can simply run the following command on the Edge device.
|
||||
By design, Docker only pulls images that have been changed. When you are developing you tend to push new images on a frequent basis, which Docker will not pull unless it has a unique tag. You could assign every push a new tag, or you can simply run the following command on the IoT Edge device.
|
||||
|
||||
```
|
||||
iotedgedev runtime --restart
|
||||
|
|
@ -562,6 +592,11 @@ Please use the [GitHub issues page](https://github.com/jonbgallant/azure-iot-edg
|
|||
Please fork, branch and pull-request any changes you'd like to make.
|
||||
|
||||
#### Contributor Dev Machine Setup
|
||||
|
||||
1. Clone or Fork this Repository
|
||||
|
||||
`git clone https://github.com/jonbgallant/azure-iot-edge-dev-tool.git`
|
||||
|
||||
1. Install **[Microsoft Visual C++ Build Tools](http://landinghub.visualstudio.com/visual-cpp-build-tools)**
|
||||
|
||||
1. Install **OpenSSL 1.1.0g**
|
||||
|
|
@ -569,7 +604,7 @@ Please fork, branch and pull-request any changes you'd like to make.
|
|||
1. [Download from OpenSSL's website](https://www.openssl.org/source/openssl-1.1.0g.tar.gz)
|
||||
1. Extract the downloaded .tar.gz to C:\OpenSSL\
|
||||
|
||||
1. Make sure both **Python 2.7 and Python 3** are installed
|
||||
1. Make sure both **Python 2.7 and Python 3.6** are installed
|
||||
|
||||
1. Install dependencies
|
||||
|
||||
|
|
@ -579,11 +614,19 @@ Please fork, branch and pull-request any changes you'd like to make.
|
|||
pip install -r requirements_dev.txt
|
||||
```
|
||||
|
||||
#### Run the tests
|
||||
1. Editable Mode
|
||||
|
||||
Run the following command from the root of the project to see changes to iotedgedev commands as you change code.
|
||||
|
||||
```
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
#### Run Tests
|
||||
|
||||
Run the following command to run tests.
|
||||
|
||||
tox
|
||||
`tox`
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
Двоичные данные
assets/edgedevtool2mins.png
Двоичные данные
assets/edgedevtool2mins.png
Двоичный файл не отображается.
|
До Ширина: | Высота: | Размер: 29 KiB После Ширина: | Высота: | Размер: 14 KiB |
Двоичные данные
assets/edgedevtoolintro.png
Двоичные данные
assets/edgedevtoolintro.png
Двоичный файл не отображается.
|
До Ширина: | Высота: | Размер: 29 KiB После Ширина: | Высота: | Размер: 14 KiB |
Двоичные данные
assets/edgedevtoolwsl.png
Двоичные данные
assets/edgedevtoolwsl.png
Двоичный файл не отображается.
|
До Ширина: | Высота: | Размер: 33 KiB После Ширина: | Высота: | Размер: 15 KiB |
|
|
@ -5,13 +5,14 @@ from __future__ import absolute_import
|
|||
|
||||
import click
|
||||
import sys
|
||||
from .iotedgedev import Docker
|
||||
from .iotedgedev import Modules
|
||||
from .iotedgedev import Runtime
|
||||
from .iotedgedev import Project
|
||||
from .iotedgedev import Utility
|
||||
from .iotedgedev import EnvVars
|
||||
from .iotedgedev import Output
|
||||
from .dockercls import Docker
|
||||
from .modules import Modules
|
||||
from .runtime import Runtime
|
||||
from .project import Project
|
||||
from .utility import Utility
|
||||
from .envvars import EnvVars
|
||||
from .output import Output
|
||||
from .iothub import IoTHub
|
||||
|
||||
output = Output()
|
||||
envvars = EnvVars(output)
|
||||
|
|
@ -50,6 +51,18 @@ def project(create):
|
|||
proj = Project(output)
|
||||
proj.create(create)
|
||||
|
||||
@click.command(context_settings=CONTEXT_SETTINGS)
|
||||
@click.option(
|
||||
'--monitor-events',
|
||||
default=False,
|
||||
required=False,
|
||||
is_flag=True,
|
||||
help="Displays events that are sent from IoT Hub device to IoT Hub.")
|
||||
def iothub(monitor_events):
|
||||
if monitor_events:
|
||||
utility = Utility(envvars, output)
|
||||
ih = IoTHub(envvars, output, utility)
|
||||
ih.monitor_events()
|
||||
|
||||
@click.command(context_settings=CONTEXT_SETTINGS)
|
||||
@click.option(
|
||||
|
|
@ -63,7 +76,7 @@ def project(create):
|
|||
default=False,
|
||||
required=False,
|
||||
is_flag=True,
|
||||
help="Deploys modules to Edge device using modules.json in build/config directory.")
|
||||
help="Deploys modules to Edge device using deployment.json in the /.config directory.")
|
||||
def modules(build, deploy):
|
||||
utility = Utility(envvars, output)
|
||||
dock = Docker(envvars, utility, output)
|
||||
|
|
@ -215,6 +228,7 @@ main.add_command(runtime)
|
|||
main.add_command(modules)
|
||||
main.add_command(docker)
|
||||
main.add_command(project)
|
||||
main.add_command(iothub)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
|
|
|||
|
|
@ -3,13 +3,17 @@ class ConnectionString:
|
|||
self.value = value
|
||||
self.data = dict()
|
||||
|
||||
parts = value.split(';')
|
||||
for part in parts:
|
||||
subpart = part.split('=', 1)
|
||||
self.data[subpart[0].lower()] = subpart[1].strip()
|
||||
if self.value:
|
||||
parts = value.split(';')
|
||||
if len(parts) > 0:
|
||||
for part in parts:
|
||||
subpart = part.split('=', 1)
|
||||
if len(subpart) == 2:
|
||||
self.data[subpart[0].lower()] = subpart[1].strip()
|
||||
|
||||
self.HostName = self["hostname"]
|
||||
self.SharedAccessKey = self["sharedaccesskey"]
|
||||
if self.data:
|
||||
self.HostName = self["hostname"]
|
||||
self.SharedAccessKey = self["sharedaccesskey"]
|
||||
|
||||
def __getitem__(self, key):
|
||||
return self.data[key]
|
||||
|
|
@ -19,11 +23,13 @@ class IoTHubConnectionString(ConnectionString):
|
|||
def __init__(self, value):
|
||||
ConnectionString.__init__(self, value)
|
||||
|
||||
self.SharedAccessKeyName = self["sharedaccesskeyname"]
|
||||
if self.value:
|
||||
self.SharedAccessKeyName = self["sharedaccesskeyname"]
|
||||
|
||||
|
||||
class DeviceConnectionString(ConnectionString):
|
||||
def __init__(self, value):
|
||||
ConnectionString.__init__(self, value)
|
||||
|
||||
self.DeviceId = self["deviceid"]
|
||||
if self.value:
|
||||
self.DeviceId = self["deviceid"]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,266 @@
|
|||
import docker
|
||||
from enum import Enum
|
||||
import os
|
||||
import zipfile
|
||||
from .moduletype import ModuleType
|
||||
|
||||
class Docker:
|
||||
|
||||
def __init__(self, envvars, utility, output):
|
||||
self.envvars = envvars
|
||||
self.envvars.check()
|
||||
self.utility = utility
|
||||
self.utility.set_config()
|
||||
self.output = output
|
||||
|
||||
if self.envvars.DOCKER_HOST:
|
||||
self.docker_client = docker.DockerClient(
|
||||
base_url=self.envvars.DOCKER_HOST)
|
||||
self.docker_api = docker.APIClient(
|
||||
base_url=self.envvars.DOCKER_HOST)
|
||||
else:
|
||||
self.docker_client = docker.from_env()
|
||||
self.docker_api = docker.APIClient()
|
||||
|
||||
def init_registry(self):
|
||||
|
||||
self.output.header("INITIALIZING CONTAINER REGISTRY")
|
||||
self.output.info("REGISTRY: " + self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
|
||||
if "localhost" in self.envvars.CONTAINER_REGISTRY_SERVER:
|
||||
self.init_local_registry()
|
||||
|
||||
self.login_registry()
|
||||
self.output.line()
|
||||
|
||||
def init_local_registry(self):
|
||||
|
||||
parts = self.envvars.CONTAINER_REGISTRY_SERVER.split(":")
|
||||
|
||||
if len(parts) < 2:
|
||||
self.output.error("You must specific a port for your local registry server. Expected: 'localhost:5000'. Found: " +
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
sys.exit()
|
||||
|
||||
port = parts[1]
|
||||
ports = {'{0}/tcp'.format(port): int(port)}
|
||||
|
||||
try:
|
||||
self.output.info("Looking for local 'registry' container")
|
||||
self.docker_client.containers.get("registry")
|
||||
self.output.info("Found local 'registry' container")
|
||||
except docker.errors.NotFound:
|
||||
self.output.info("Local 'registry' container not found")
|
||||
|
||||
try:
|
||||
self.output.info("Looking for local 'registry' image")
|
||||
self.docker_client.images.get("registry:2")
|
||||
self.output.info("Local 'registry' image found")
|
||||
except docker.errors.ImageNotFound:
|
||||
self.output.info("Local 'registry' image not found")
|
||||
self.output.info("Pulling 'registry' image")
|
||||
self.docker_client.images.pull("registry", tag="2")
|
||||
|
||||
self.output.info("Running registry container")
|
||||
self.docker_client.containers.run(
|
||||
"registry:2", detach=True, name="registry", ports=ports, restart_policy={"Name": "always"})
|
||||
|
||||
def login_registry(self):
|
||||
try:
|
||||
|
||||
if "localhost" in self.envvars.CONTAINER_REGISTRY_SERVER:
|
||||
client_login_status = self.docker_client.login(
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
|
||||
api_login_status = self.docker_api.login(
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
else:
|
||||
|
||||
client_login_status = self.docker_client.login(registry=self.envvars.CONTAINER_REGISTRY_SERVER,
|
||||
username=self.envvars.CONTAINER_REGISTRY_USERNAME,
|
||||
password=self.envvars.CONTAINER_REGISTRY_PASSWORD)
|
||||
|
||||
api_login_status = self.docker_api.login(registry=self.envvars.CONTAINER_REGISTRY_SERVER,
|
||||
username=self.envvars.CONTAINER_REGISTRY_USERNAME,
|
||||
password=self.envvars.CONTAINER_REGISTRY_PASSWORD)
|
||||
|
||||
self.output.info("Successfully logged into container registry: " +
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
|
||||
except Exception as ex:
|
||||
self.output.error(
|
||||
"Could not login to Container Registry. 1. Make sure Docker is running locally. 2. Verify your credentials in CONTAINER_REGISTRY_ environment variables. 2. If you are using WSL, then please set DOCKER_HOST Environment Variable. See the projects readme for full instructions.")
|
||||
self.output.error(str(ex))
|
||||
sys.exit(-1)
|
||||
|
||||
def setup_registry(self):
|
||||
self.output.header("SETTING UP CONTAINER REGISTRY")
|
||||
self.init_registry()
|
||||
self.output.info("PUSHING EDGE IMAGES TO CONTAINER REGISTRY")
|
||||
image_names = ["azureiotedge-agent", "azureiotedge-hub",
|
||||
"azureiotedge-simulated-temperature-sensor"]
|
||||
|
||||
for image_name in image_names:
|
||||
|
||||
microsoft_image_name = "microsoft/{0}:{1}".format(
|
||||
image_name, self.envvars.RUNTIME_TAG)
|
||||
|
||||
container_registry_image_name = "{0}/{1}:{2}".format(
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER, image_name, self.envvars.RUNTIME_TAG)
|
||||
|
||||
# Pull image from Microsoft Docker Hub
|
||||
try:
|
||||
self.output.info(
|
||||
"PULLING IMAGE: '{0}'".format(microsoft_image_name))
|
||||
image_pull = self.docker_client.images.pull(
|
||||
microsoft_image_name)
|
||||
self.output.info(
|
||||
"SUCCESSFULLY PULLED IMAGE: '{0}'".format(microsoft_image_name))
|
||||
self.output.info(str(image_pull))
|
||||
except docker.errors.APIError as e:
|
||||
self.output.error(
|
||||
"ERROR WHILE PULLING IMAGE: '{0}'".format(microsoft_image_name))
|
||||
self.output.error(e)
|
||||
|
||||
# Tagging Image with Container Registry Name
|
||||
try:
|
||||
tag_result = self.docker_api.tag(
|
||||
image=microsoft_image_name, repository=container_registry_image_name)
|
||||
except docker.errors.APIError as e:
|
||||
self.output.error(
|
||||
"ERROR WHILE TAGGING IMAGE: '{0}'".format(microsoft_image_name))
|
||||
self.output.error(e)
|
||||
|
||||
# Push Image to Container Registry
|
||||
try:
|
||||
self.output.info("PUSHING IMAGE: '{0}'".format(
|
||||
container_registry_image_name))
|
||||
|
||||
for line in self.docker_client.images.push(repository=container_registry_image_name, tag=self.envvars.RUNTIME_TAG, stream=True, auth_config={"username": self.envvars.CONTAINER_REGISTRY_USERNAME, "password": self.envvars.CONTAINER_REGISTRY_PASSWORD}):
|
||||
self.output.procout(self.utility.decode(line))
|
||||
|
||||
self.output.info("SUCCESSFULLY PUSHED IMAGE: '{0}'".format(
|
||||
container_registry_image_name))
|
||||
except docker.errors.APIError as e:
|
||||
self.output.error("ERROR WHILE PUSHING IMAGE: '{0}'".format(
|
||||
container_registry_image_name))
|
||||
self.output.error(e)
|
||||
|
||||
self.setup_registry_in_config(image_names)
|
||||
|
||||
self.utility.set_config(force=True)
|
||||
|
||||
self.output.footer("Container Registry Setup Complete")
|
||||
|
||||
def setup_registry_in_config(self, image_names):
|
||||
self.output.info(
|
||||
"Replacing 'microsoft/' with '{CONTAINER_REGISTRY_SERVER}/' in config files.")
|
||||
|
||||
# Replace microsoft/ with ${CONTAINER_REGISTRY_SERVER}
|
||||
for config_file in self.utility.get_config_files():
|
||||
config_file_contents = self.utility.get_file_contents(config_file)
|
||||
for image_name in image_names:
|
||||
config_file_contents = config_file_contents.replace(
|
||||
"microsoft/" + image_name, "${CONTAINER_REGISTRY_SERVER}/" + image_name)
|
||||
|
||||
with open(config_file, "w") as config_file_build:
|
||||
config_file_build.write(config_file_contents)
|
||||
|
||||
def remove_modules(self):
|
||||
self.output.info(
|
||||
"Removing Edge Modules Containers and Images from Docker")
|
||||
|
||||
modules_in_config = self.utility.get_modules_in_config(ModuleType.User)
|
||||
|
||||
for module in modules_in_config:
|
||||
self.output.info("Searching for {0} Containers".format(module))
|
||||
containers = self.docker_client.containers.list(
|
||||
filters={"name": module})
|
||||
for container in containers:
|
||||
self.output.info("Removing Container: " + str(container))
|
||||
container.remove(force=True)
|
||||
|
||||
self.output.info("Searching for {0} Images".format(module))
|
||||
for image in self.docker_client.images.list():
|
||||
if module in str(image):
|
||||
self.output.info(
|
||||
"Removing Module Image: " + str(image))
|
||||
self.docker_client.images.remove(
|
||||
image=image.id, force=True)
|
||||
|
||||
def remove_containers(self):
|
||||
self.output.info("Removing Containers....")
|
||||
containers = self.docker_client.containers.list(all=True)
|
||||
self.output.info("Found {0} Containers".format(len(containers)))
|
||||
for container in containers:
|
||||
self.output.info("Removing Container: {0}:{1}".format(
|
||||
container.id, container.name))
|
||||
container.remove(force=True)
|
||||
self.output.info("Containers Removed")
|
||||
|
||||
def remove_images(self):
|
||||
self.output.info("Removing Dangling Images....")
|
||||
images = self.docker_client.images.list(
|
||||
all=True, filters={"dangling": True})
|
||||
self.output.info("Found {0} Images".format(len(images)))
|
||||
|
||||
for image in images:
|
||||
self.output.info("Removing Image: {0}".format(str(image.id)))
|
||||
self.docker_client.images.remove(image=image.id, force=True)
|
||||
self.output.info("Images Removed")
|
||||
|
||||
self.output.info("Removing Images....")
|
||||
images = self.docker_client.images.list()
|
||||
self.output.info("Found {0} Images".format(len(images)))
|
||||
|
||||
for image in images:
|
||||
self.output.info("Removing Image: {0}".format(str(image.id)))
|
||||
self.docker_client.images.remove(image=image.id, force=True)
|
||||
self.output.info("Images Removed")
|
||||
|
||||
def handle_logs_cmd(self, show, save):
|
||||
|
||||
# Create LOGS_PATH dir if it doesn't exist
|
||||
if save and not os.path.exists(self.envvars.LOGS_PATH):
|
||||
os.makedirs(self.envvars.LOGS_PATH)
|
||||
|
||||
modules_in_config = self.utility.get_modules_in_config(ModuleType.Both)
|
||||
|
||||
for module in modules_in_config:
|
||||
if show:
|
||||
try:
|
||||
command = self.envvars.LOGS_CMD.format(module)
|
||||
os.system(command)
|
||||
except Exception as ex:
|
||||
self.output.error(
|
||||
"Error while trying to open module log '{0}' with command '{1}'. Try iotedgedev docker --save-logs instead.".format(module, command))
|
||||
self.output.error(str(ex))
|
||||
if save:
|
||||
try:
|
||||
self.utility.exe_proc(["docker", "logs", module, ">",
|
||||
os.path.join(self.envvars.LOGS_PATH, module + ".log")], True)
|
||||
except Exception as ex:
|
||||
self.output.error(
|
||||
"Error while trying to save module log file '{0}'".format(module))
|
||||
self.output.error(str(ex))
|
||||
|
||||
if save:
|
||||
self.zip_logs()
|
||||
|
||||
def zip_logs(self):
|
||||
log_files = [os.path.join(self.envvars.LOGS_PATH, f)
|
||||
for f in os.listdir(self.envvars.LOGS_PATH) if f.endswith(".log")]
|
||||
zip_path = os.path.join(self.envvars.LOGS_PATH, 'edge-logs.zip')
|
||||
|
||||
self.output.info("Creating {0} file".format(zip_path))
|
||||
|
||||
zipf = zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DEFLATED)
|
||||
|
||||
for log_file in log_files:
|
||||
self.output.info("Adding {0} to zip". format(log_file))
|
||||
zipf.write(log_file)
|
||||
|
||||
zipf.close()
|
||||
|
||||
self.output.info("Log files successfully saved to: " + zip_path)
|
||||
|
||||
|
|
@ -0,0 +1,138 @@
|
|||
from dotenv import load_dotenv
|
||||
import os
|
||||
import platform
|
||||
import socket
|
||||
import sys
|
||||
from .connectionstring import IoTHubConnectionString, DeviceConnectionString
|
||||
|
||||
class EnvVars:
|
||||
def __init__(self, output):
|
||||
self.output = output
|
||||
self.checked = False
|
||||
|
||||
def load_dotenv(self):
|
||||
dotenv_file = self.get_dotenv_file()
|
||||
dotenv_path = os.path.join(os.getcwd(), dotenv_file)
|
||||
|
||||
try:
|
||||
if os.path.exists(dotenv_path):
|
||||
load_dotenv(dotenv_path)
|
||||
self.output.info(
|
||||
"Environment Variables loaded from: {0} ({1})".format(dotenv_file, dotenv_path))
|
||||
else:
|
||||
self.output.info(
|
||||
"{0} file not found on disk. Without a file on disk, you must specify all Environment Variables at the system level. ({1})".format(dotenv_file, dotenv_path))
|
||||
except Exception as ex:
|
||||
self.output.error("Error while trying to load .env file: {0}. {1}".format(
|
||||
dotenv_path, str(ex)))
|
||||
|
||||
def get_dotenv_file(self):
|
||||
default_dotenv_file = ".env"
|
||||
|
||||
if not "DOTENV_FILE" in os.environ:
|
||||
return default_dotenv_file
|
||||
else:
|
||||
dotenv_file_from_environ = os.environ["DOTENV_FILE"].strip(
|
||||
"\"").strip("'")
|
||||
if dotenv_file_from_environ:
|
||||
return dotenv_file_from_environ
|
||||
|
||||
return default_dotenv_file
|
||||
|
||||
def check(self):
|
||||
if not self.checked:
|
||||
self.load_dotenv()
|
||||
|
||||
try:
|
||||
try:
|
||||
self.IOTHUB_CONNECTION_STRING = self.get_envvar(
|
||||
"IOTHUB_CONNECTION_STRING")
|
||||
self.IOTHUB_CONNECTION_INFO = IoTHubConnectionString(
|
||||
self.IOTHUB_CONNECTION_STRING)
|
||||
except Exception as ex:
|
||||
self.output.error("Unable to parse IOTHUB_CONNECTION_STRING Environment Variable. Please ensure that you have the right connection string set.")
|
||||
self.output.error(str(ex))
|
||||
sys.exit(-1)
|
||||
|
||||
try:
|
||||
self.DEVICE_CONNECTION_STRING = self.get_envvar(
|
||||
"DEVICE_CONNECTION_STRING")
|
||||
self.DEVICE_CONNECTION_INFO = DeviceConnectionString(
|
||||
self.DEVICE_CONNECTION_STRING)
|
||||
except Exception as ex:
|
||||
self.output.error("Unable to parse DEVICE_CONNECTION_STRING Environment Variable. Please ensure that you have the right connection string set.")
|
||||
self.output.error(str(ex))
|
||||
sys.exit(-1)
|
||||
|
||||
|
||||
self.RUNTIME_HOST_NAME = self.get_envvar("RUNTIME_HOST_NAME")
|
||||
if self.RUNTIME_HOST_NAME == ".":
|
||||
self.set_envvar("RUNTIME_HOST_NAME", socket.gethostname())
|
||||
|
||||
self.RUNTIME_HOME_DIR = self.get_envvar("RUNTIME_HOME_DIR")
|
||||
if self.RUNTIME_HOME_DIR == ".":
|
||||
self.set_envvar("RUNTIME_HOME_DIR", self.get_runtime_home_dir())
|
||||
|
||||
|
||||
self.RUNTIME_CONFIG_DIR = self.get_envvar("RUNTIME_CONFIG_DIR")
|
||||
if self.RUNTIME_CONFIG_DIR == ".":
|
||||
self.set_envvar("RUNTIME_CONFIG_DIR", self.get_runtime_config_dir())
|
||||
|
||||
self.ACTIVE_MODULES = self.get_envvar("ACTIVE_MODULES")
|
||||
self.ACTIVE_DOCKER_DIRS = self.get_envvar("ACTIVE_DOCKER_DIRS")
|
||||
self.CONTAINER_REGISTRY_SERVER = self.get_envvar(
|
||||
"CONTAINER_REGISTRY_SERVER", False)
|
||||
self.CONTAINER_REGISTRY_USERNAME = self.get_envvar(
|
||||
"CONTAINER_REGISTRY_USERNAME", False)
|
||||
self.CONTAINER_REGISTRY_PASSWORD = self.get_envvar(
|
||||
"CONTAINER_REGISTRY_PASSWORD", False)
|
||||
self.CONTAINER_TAG = self.get_envvar("CONTAINER_TAG", False)
|
||||
self.RUNTIME_TAG = self.get_envvar("RUNTIME_TAG")
|
||||
self.RUNTIME_VERBOSITY = self.get_envvar("RUNTIME_VERBOSITY")
|
||||
self.MODULES_CONFIG_FILE = self.get_envvar(
|
||||
"MODULES_CONFIG_FILE")
|
||||
self.RUNTIME_CONFIG_FILE = self.get_envvar(
|
||||
"RUNTIME_CONFIG_FILE")
|
||||
self.LOGS_PATH = self.get_envvar("LOGS_PATH")
|
||||
self.MODULES_PATH = self.get_envvar("MODULES_PATH")
|
||||
self.IOT_REST_API_VERSION = self.get_envvar(
|
||||
"IOT_REST_API_VERSION")
|
||||
self.DOTNET_VERBOSITY = self.get_envvar("DOTNET_VERBOSITY")
|
||||
self.LOGS_CMD = self.get_envvar("LOGS_CMD")
|
||||
if "DOCKER_HOST" in os.environ:
|
||||
self.DOCKER_HOST = self.get_envvar("DOCKER_HOST", False)
|
||||
else:
|
||||
self.DOCKER_HOST = None
|
||||
except Exception as ex:
|
||||
self.output.error(
|
||||
"Environment variables not configured correctly. Run `iotedgedev project --create [name]` to create a new project with sample .env file. Please see README for variable configuration options. Tip: You might just need to restart your command prompt to refresh your Environment Variables.")
|
||||
self.output.error("Variable that caused exception: " + str(ex))
|
||||
sys.exit(-1)
|
||||
|
||||
self.checked = True
|
||||
|
||||
def get_envvar(self, key, required=True):
|
||||
val = os.environ[key].strip()
|
||||
if not val and required:
|
||||
self.output.error("Environment Variable {0} not set. Either add to .env file or to your system's Environment Variables".format(key))
|
||||
sys.exit(-1)
|
||||
|
||||
return val
|
||||
|
||||
def set_envvar(self, key, value):
|
||||
os.environ[key] = value
|
||||
|
||||
def get_runtime_home_dir(self):
|
||||
plat = platform.system().lower()
|
||||
if plat == "linux" or plat == "darwin":
|
||||
return "/var/lib/azure-iot-edge"
|
||||
else:
|
||||
return os.environ["PROGRAMDATA"].replace("\\", "\\\\") + "\\\\azure-iot-edge\\\data"
|
||||
|
||||
def get_runtime_config_dir(self):
|
||||
plat = platform.system().lower()
|
||||
|
||||
if plat == "linux" or plat == "darwin":
|
||||
return "/etc/azure-iot-edge"
|
||||
else:
|
||||
return os.environ["PROGRAMDATA"].replace("\\", "\\\\") + "\\\\azure-iot-edge\\\\config"
|
||||
|
|
@ -1,785 +0,0 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
|
||||
from __future__ import absolute_import
|
||||
import requests
|
||||
import uuid
|
||||
import docker
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import json
|
||||
import click
|
||||
import zipfile
|
||||
import socket
|
||||
import platform
|
||||
from base64 import b64encode, b64decode
|
||||
from hashlib import sha256
|
||||
from time import time
|
||||
import fnmatch
|
||||
import inspect
|
||||
from hmac import HMAC
|
||||
from shutil import copyfile
|
||||
from enum import Enum
|
||||
from distutils.util import strtobool
|
||||
from dotenv import load_dotenv
|
||||
from .connectionstring import IoTHubConnectionString, DeviceConnectionString
|
||||
if sys.version_info.major >= 3:
|
||||
from urllib.parse import quote, urlencode
|
||||
else:
|
||||
from urllib import quote, urlencode
|
||||
|
||||
#print("Python Version: " + sys.version)
|
||||
|
||||
|
||||
class Output:
|
||||
|
||||
def info(self, text):
|
||||
click.secho(text, fg='yellow')
|
||||
|
||||
def error(self, text):
|
||||
click.secho(text, fg='red')
|
||||
|
||||
def header(self, text):
|
||||
click.secho("======== {0} ========".format(text).upper(), fg='white')
|
||||
|
||||
def footer(self, text):
|
||||
self.info(text.upper())
|
||||
self.line()
|
||||
|
||||
def procout(self, text):
|
||||
click.secho(text, dim=True)
|
||||
|
||||
def line(self):
|
||||
click.secho("")
|
||||
|
||||
|
||||
class EnvVars:
|
||||
def __init__(self, output):
|
||||
self.output = output
|
||||
self.checked = False
|
||||
|
||||
def load_dotenv(self):
|
||||
dotenv_file = self.get_dotenv_file()
|
||||
dotenv_path = os.path.join(os.getcwd(), dotenv_file)
|
||||
|
||||
try:
|
||||
if os.path.exists(dotenv_path):
|
||||
load_dotenv(dotenv_path)
|
||||
self.output.info(
|
||||
"Environment Variables loaded from: {0} ({1})".format(dotenv_file, dotenv_path))
|
||||
else:
|
||||
self.output.info(
|
||||
"{0} file not found on disk. Without a file on disk, you must specify all Environment Variables at the system level. ({1})".format(dotenv_file, dotenv_path))
|
||||
except Exception as e:
|
||||
self.output.error("ERROR: Error while trying to load .env file: {0}. {1}".format(
|
||||
dotenv_path, str(e)))
|
||||
|
||||
def get_dotenv_file(self):
|
||||
default_dotenv_file = ".env"
|
||||
|
||||
if not "DOTENV_FILE" in os.environ:
|
||||
return default_dotenv_file
|
||||
else:
|
||||
dotenv_file_from_environ = os.environ["DOTENV_FILE"].strip(
|
||||
"\"").strip("'")
|
||||
if dotenv_file_from_environ:
|
||||
return dotenv_file_from_environ
|
||||
|
||||
return default_dotenv_file
|
||||
|
||||
def check(self):
|
||||
if not self.checked:
|
||||
self.load_dotenv()
|
||||
|
||||
try:
|
||||
try:
|
||||
self.IOTHUB_CONNECTION_STRING = self.get_envvar(
|
||||
"IOTHUB_CONNECTION_STRING")
|
||||
self.IOTHUB_CONNECTION_INFO = IoTHubConnectionString(
|
||||
self.IOTHUB_CONNECTION_STRING)
|
||||
except:
|
||||
self.output.error("ERROR: Unable to parse IOTHUB_CONNECTION_STRING Environment Variable. Please ensure that you have the right connection string set.")
|
||||
sys.exit(-1)
|
||||
|
||||
try:
|
||||
self.DEVICE_CONNECTION_STRING = self.get_envvar(
|
||||
"DEVICE_CONNECTION_STRING")
|
||||
self.DEVICE_CONNECTION_INFO = DeviceConnectionString(
|
||||
self.DEVICE_CONNECTION_STRING)
|
||||
except:
|
||||
self.output.error("ERROR: Unable to parse DEVICE_CONNECTION_STRING Environment Variable. Please ensure that you have the right connection string set.")
|
||||
sys.exit(-1)
|
||||
|
||||
|
||||
self.RUNTIME_HOST_NAME = self.get_envvar("RUNTIME_HOST_NAME")
|
||||
if self.RUNTIME_HOST_NAME == ".":
|
||||
self.set_envvar("RUNTIME_HOST_NAME", socket.gethostname())
|
||||
|
||||
self.RUNTIME_HOME_DIR = self.get_envvar("RUNTIME_HOME_DIR")
|
||||
if self.RUNTIME_HOME_DIR == ".":
|
||||
self.set_envvar("RUNTIME_HOME_DIR", self.get_runtime_home_dir())
|
||||
|
||||
|
||||
self.RUNTIME_CONFIG_DIR = self.get_envvar("RUNTIME_CONFIG_DIR")
|
||||
if self.RUNTIME_CONFIG_DIR == ".":
|
||||
self.set_envvar("RUNTIME_CONFIG_DIR", self.get_runtime_config_dir())
|
||||
|
||||
self.ACTIVE_MODULES = self.get_envvar("ACTIVE_MODULES")
|
||||
self.ACTIVE_DOCKER_DIRS = self.get_envvar("ACTIVE_DOCKER_DIRS")
|
||||
self.CONTAINER_REGISTRY_SERVER = self.get_envvar(
|
||||
"CONTAINER_REGISTRY_SERVER")
|
||||
self.CONTAINER_REGISTRY_USERNAME = self.get_envvar(
|
||||
"CONTAINER_REGISTRY_USERNAME")
|
||||
self.CONTAINER_REGISTRY_PASSWORD = self.get_envvar(
|
||||
"CONTAINER_REGISTRY_PASSWORD")
|
||||
self.CONTAINER_TAG = self.get_envvar("CONTAINER_TAG")
|
||||
self.RUNTIME_TAG = self.get_envvar("RUNTIME_TAG")
|
||||
self.RUNTIME_VERBOSITY = self.get_envvar("RUNTIME_VERBOSITY")
|
||||
self.MODULES_CONFIG_FILE = self.get_envvar(
|
||||
"MODULES_CONFIG_FILE")
|
||||
self.RUNTIME_CONFIG_FILE = self.get_envvar(
|
||||
"RUNTIME_CONFIG_FILE")
|
||||
self.LOGS_PATH = self.get_envvar("LOGS_PATH")
|
||||
self.MODULES_PATH = self.get_envvar("MODULES_PATH")
|
||||
self.IOT_REST_API_VERSION = self.get_envvar(
|
||||
"IOT_REST_API_VERSION")
|
||||
self.DOTNET_VERBOSITY = self.get_envvar("DOTNET_VERBOSITY")
|
||||
self.LOGS_CMD = self.get_envvar("LOGS_CMD")
|
||||
if "DOCKER_HOST" in os.environ:
|
||||
self.DOCKER_HOST = self.get_envvar("DOCKER_HOST")
|
||||
else:
|
||||
self.DOCKER_HOST = None
|
||||
except Exception as e:
|
||||
self.output.error(
|
||||
"Environment variables not configured correctly. Run `iotedgedev project --create [name]` to create a new project with sample .env file. Please see README for variable configuration options. Tip: You might just need to restart your command prompt to refresh your Environment Variables.")
|
||||
self.output.error("Variable that caused exception: " + str(e))
|
||||
sys.exit(-1)
|
||||
|
||||
self.checked = True
|
||||
|
||||
def get_envvar(self, key):
|
||||
return os.environ[key].strip()
|
||||
|
||||
def set_envvar(self, key, value):
|
||||
os.environ[key] = value
|
||||
|
||||
def get_runtime_home_dir(self):
|
||||
plat = platform.system().lower()
|
||||
if plat == "linux" or plat == "darwin":
|
||||
return "/var/lib/azure-iot-edge"
|
||||
else:
|
||||
return os.environ["PROGRAMDATA"].replace("\\", "\\\\") + "\\\\azure-iot-edge\\\data"
|
||||
|
||||
def get_runtime_config_dir(self):
|
||||
plat = platform.system().lower()
|
||||
|
||||
if plat == "linux" or plat == "darwin":
|
||||
return "/etc/azure-iot-edge"
|
||||
else:
|
||||
return os.environ["PROGRAMDATA"].replace("\\", "\\\\") + "\\\\azure-iot-edge\\\\config"
|
||||
|
||||
|
||||
class Utility:
|
||||
def __init__(self, envvars, output):
|
||||
self.envvars = envvars
|
||||
self.envvars.check()
|
||||
self.output = output
|
||||
self.config_set = False
|
||||
|
||||
def exe_proc(self, params, shell=False):
|
||||
proc = subprocess.Popen(
|
||||
params, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=shell)
|
||||
|
||||
stdout_data, stderr_data = proc.communicate()
|
||||
if stdout_data != "":
|
||||
self.output.procout(self.decode(stdout_data))
|
||||
|
||||
if proc.returncode != 0:
|
||||
self.output.error(self.decode(stderr_data))
|
||||
sys.exit()
|
||||
|
||||
def find_files(self, directory, pattern):
|
||||
# find all files in directory that match the pattern.
|
||||
for root, dirs, files in os.walk(directory):
|
||||
for basename in files:
|
||||
if fnmatch.fnmatch(basename, pattern):
|
||||
filename = os.path.join(root, basename)
|
||||
yield filename
|
||||
|
||||
def get_iot_hub_sas_token(self, uri, key, policy_name, expiry=3600):
|
||||
ttl = time() + expiry
|
||||
sign_key = "%s\n%d" % ((quote(uri)), int(ttl))
|
||||
signature = b64encode(
|
||||
HMAC(b64decode(key), sign_key.encode("utf-8"), sha256).digest())
|
||||
|
||||
rawtoken = {
|
||||
"sr": uri,
|
||||
"sig": signature,
|
||||
"se": str(int(ttl))
|
||||
}
|
||||
|
||||
if policy_name is not None:
|
||||
rawtoken["skn"] = policy_name
|
||||
|
||||
return "SharedAccessSignature " + urlencode(rawtoken)
|
||||
|
||||
def get_file_contents(self, file):
|
||||
with open(file, "r") as file:
|
||||
return file.read()
|
||||
|
||||
def decode(self, val):
|
||||
return val.decode("utf-8").strip()
|
||||
|
||||
def get_config_files(self):
|
||||
# config files are in root of project
|
||||
|
||||
return [os.path.join(os.getcwd(), f) for f in os.listdir(os.getcwd()) if f.endswith("template.json")]
|
||||
|
||||
def get_active_modules(self):
|
||||
return [module.strip()
|
||||
for module in self.envvars.ACTIVE_MODULES.split(",") if module]
|
||||
|
||||
def get_modules_in_config(self, moduleType):
|
||||
modules_config = json.load(open(self.envvars.MODULES_CONFIG_FILE))
|
||||
|
||||
props = modules_config["moduleContent"]["$edgeAgent"]["properties.desired"]
|
||||
|
||||
system_modules = props["systemModules"]
|
||||
user_modules = props["modules"]
|
||||
|
||||
if moduleType == ModuleType.System:
|
||||
return system_modules
|
||||
elif moduleType == ModuleType.User:
|
||||
return user_modules
|
||||
else:
|
||||
return_modules = {}
|
||||
return_modules.update(system_modules)
|
||||
return_modules.update(user_modules)
|
||||
return return_modules
|
||||
|
||||
def set_config(self, force=False):
|
||||
|
||||
if not self.config_set or force:
|
||||
self.envvars.check()
|
||||
self.output.header("PROCESSING CONFIG FILES")
|
||||
|
||||
config_output_dir = ".config"
|
||||
|
||||
# Create config dir if it doesn't exist
|
||||
if not os.path.exists(config_output_dir):
|
||||
os.makedirs(config_output_dir)
|
||||
|
||||
config_files = self.get_config_files()
|
||||
|
||||
if len(config_files) == 0:
|
||||
self.output.info(
|
||||
"Unable to find config files in project root directory")
|
||||
sys.exit()
|
||||
|
||||
# Expand envars and rewrite to .config/
|
||||
for config_file in config_files:
|
||||
|
||||
build_config_file = os.path.join(
|
||||
config_output_dir, os.path.basename(config_file).replace(".template", ""))
|
||||
|
||||
self.output.info("Expanding '{0}' to '{1}'".format(
|
||||
os.path.basename(config_file), build_config_file))
|
||||
|
||||
config_file_expanded = os.path.expandvars(
|
||||
self.get_file_contents(config_file))
|
||||
|
||||
with open(build_config_file, "w") as config_file_build:
|
||||
config_file_build.write(config_file_expanded)
|
||||
|
||||
self.output.line()
|
||||
|
||||
self.config_set = True
|
||||
|
||||
|
||||
class Project:
|
||||
def __init__(self, output):
|
||||
self.output = output
|
||||
|
||||
def create(self, name):
|
||||
self.output.header("CREATING AZURE IOT EDGE PROJECT")
|
||||
|
||||
try:
|
||||
template_zip = os.path.join(os.path.split(
|
||||
__file__)[0], "template", "template.zip")
|
||||
except Exception as e:
|
||||
self.output.error("Error while trying to load template.zip")
|
||||
self.output.error(e)
|
||||
|
||||
if name == ".":
|
||||
name = ""
|
||||
|
||||
zipf = zipfile.ZipFile(template_zip)
|
||||
zipf.extractall(name)
|
||||
|
||||
os.rename(os.path.join(name, ".env.tmp"), os.path.join(name, ".env"))
|
||||
self.output.footer("Azure IoT Edge project created")
|
||||
|
||||
|
||||
class Runtime:
|
||||
def __init__(self, envvars, utility, output, dock):
|
||||
self.envvars = envvars
|
||||
self.envvars.check()
|
||||
self.utility = utility
|
||||
self.utility.set_config()
|
||||
self.dock = dock
|
||||
self.output = output
|
||||
|
||||
def start(self):
|
||||
self.output.header("Starting Edge Runtime")
|
||||
self.utility.exe_proc(["iotedgectl", "--verbose",
|
||||
self.envvars.RUNTIME_VERBOSITY, "start"])
|
||||
|
||||
def stop(self):
|
||||
self.output.header("Stopping Edge Runtime")
|
||||
self.utility.exe_proc(["iotedgectl", "--verbose",
|
||||
self.envvars.RUNTIME_VERBOSITY, "stop"])
|
||||
|
||||
def setup(self):
|
||||
self.output.header("Setting Up Edge Runtime")
|
||||
self.utility.exe_proc(["iotedgectl", "--verbose", self.envvars.RUNTIME_VERBOSITY,
|
||||
"setup", "--config-file", self.envvars.RUNTIME_CONFIG_FILE])
|
||||
|
||||
def status(self):
|
||||
self.output.header("Getting Edge Runtime Status")
|
||||
self.utility.exe_proc(["iotedgectl", "--verbose", self.envvars.RUNTIME_VERBOSITY,
|
||||
"status"])
|
||||
|
||||
def restart(self):
|
||||
self.stop()
|
||||
self.dock.remove_modules()
|
||||
self.setup()
|
||||
self.start()
|
||||
|
||||
|
||||
class Modules:
|
||||
def __init__(self, envvars, utility, output, dock):
|
||||
self.envvars = envvars
|
||||
self.envvars.check()
|
||||
self.utility = utility
|
||||
self.utility.set_config()
|
||||
self.output = output
|
||||
self.dock = dock
|
||||
self.dock.init_registry()
|
||||
|
||||
def build(self):
|
||||
self.output.header("BUILDING MODULES")
|
||||
|
||||
# Get all the modules to build as specified in config.
|
||||
modules_to_process = self.utility.get_active_modules()
|
||||
|
||||
for module in os.listdir(self.envvars.MODULES_PATH):
|
||||
|
||||
if len(
|
||||
modules_to_process) == 0 or modules_to_process[0] == "*" or module in modules_to_process:
|
||||
|
||||
module_dir = os.path.join(self.envvars.MODULES_PATH, module)
|
||||
|
||||
self.output.info("BUILDING MODULE: {0}".format(module_dir))
|
||||
|
||||
# Find first proj file in module dir and use it.
|
||||
project_files = [os.path.join(module_dir, f) for f in os.listdir(
|
||||
module_dir) if f.endswith("proj")]
|
||||
|
||||
if len(project_files) == 0:
|
||||
self.output.error("No project file found for module.")
|
||||
continue
|
||||
|
||||
self.utility.exe_proc(["dotnet", "build", project_files[0],
|
||||
"-v", self.envvars.DOTNET_VERBOSITY])
|
||||
|
||||
# Get all docker files in project
|
||||
docker_files = self.utility.find_files(
|
||||
module_dir, "Dockerfile*")
|
||||
|
||||
# Filter by Docker Dirs in envvars
|
||||
docker_dirs_process = [docker_dir.strip()
|
||||
for docker_dir in self.envvars.ACTIVE_DOCKER_DIRS.split(",") if docker_dir]
|
||||
|
||||
# Process each Dockerfile found
|
||||
for docker_file in docker_files:
|
||||
|
||||
docker_file_parent_folder = os.path.basename(
|
||||
os.path.dirname(docker_file))
|
||||
|
||||
if len(
|
||||
docker_dirs_process) == 0 or docker_dirs_process[0] == "*" or docker_file_parent_folder in docker_dirs_process:
|
||||
|
||||
self.output.info(
|
||||
"PROCESSING DOCKER FILE: " + docker_file)
|
||||
|
||||
docker_file_name = os.path.basename(docker_file)
|
||||
|
||||
# assume /Docker/{runtime}/Dockerfile folder structure
|
||||
# image name will be the same as the module folder name, filter-module
|
||||
# tag will be {runtime}{ext}{container_tag}, i.e. linux-x64-debug-jong
|
||||
# runtime is the Dockerfile immediate parent folder name
|
||||
# ext is Dockerfile extension for example with Dockerfile.debug, debug is the mod
|
||||
# CONTAINER_TAG is env var
|
||||
|
||||
# i.e. when found: filter-module/Docker/linux-x64/Dockerfile.debug and CONTAINER_TAG = jong
|
||||
# we'll get: filtermodule:linux-x64-debug-jong
|
||||
|
||||
runtime = os.path.basename(
|
||||
os.path.dirname(docker_file))
|
||||
ext = "" if os.path.splitext(docker_file)[
|
||||
1] == "" else "-" + os.path.splitext(docker_file)[1][1:]
|
||||
container_tag = "" if self.envvars.CONTAINER_TAG == "" else "-" + \
|
||||
self.envvars.CONTAINER_TAG
|
||||
|
||||
tag_name = runtime + ext + container_tag
|
||||
|
||||
# construct the build output path
|
||||
build_path = os.path.join(
|
||||
os.getcwd(), "build", "modules", module, runtime)
|
||||
if not os.path.exists(build_path):
|
||||
os.makedirs(build_path)
|
||||
|
||||
# dotnet publish
|
||||
self.output.info(
|
||||
"PUBLISHING PROJECT: " + project_files[0])
|
||||
|
||||
self.utility.exe_proc(["dotnet", "publish", project_files[0], "-f", "netcoreapp2.0",
|
||||
"-o", build_path, "-v", self.envvars.DOTNET_VERBOSITY])
|
||||
|
||||
# copy Dockerfile to publish dir
|
||||
build_dockerfile = os.path.join(
|
||||
build_path, docker_file_name)
|
||||
|
||||
copyfile(docker_file, build_dockerfile)
|
||||
|
||||
image_destination_name = "{0}/{1}:{2}".format(
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER, module, tag_name).lower()
|
||||
|
||||
self.output.info(
|
||||
"BUILDING DOCKER IMAGE: " + image_destination_name)
|
||||
|
||||
# cd to the build output to build the docker image
|
||||
project_dir = os.getcwd()
|
||||
os.chdir(build_path)
|
||||
|
||||
# BUILD DOCKER IMAGE
|
||||
build_result = self.dock.docker_client.images.build(
|
||||
tag=image_destination_name, path=".", dockerfile=docker_file_name)
|
||||
self.output.info(
|
||||
"DOCKER IMAGE DETAILS: {0}".format(build_result))
|
||||
|
||||
# CD BACK UP
|
||||
os.chdir(project_dir)
|
||||
|
||||
# PUSH TO CONTAINER REGISTRY
|
||||
self.output.info(
|
||||
"PUSHING DOCKER IMAGE TO: " + image_destination_name)
|
||||
|
||||
for line in self.dock.docker_client.images.push(repository=image_destination_name, tag=tag_name, stream=True, auth_config={
|
||||
"username": self.envvars.CONTAINER_REGISTRY_USERNAME, "password": self.envvars.CONTAINER_REGISTRY_PASSWORD}):
|
||||
self.output.procout(self.utility.decode(line))
|
||||
|
||||
self.output.footer("BUILD COMPLETE")
|
||||
|
||||
def deploy(self):
|
||||
self.output.header("DEPLOYING MODULES")
|
||||
self.deploy_device_configuration(
|
||||
self.envvars.IOTHUB_CONNECTION_INFO.HostName, self.envvars.IOTHUB_CONNECTION_INFO.SharedAccessKey,
|
||||
self.envvars.DEVICE_CONNECTION_INFO.DeviceId, self.envvars.MODULES_CONFIG_FILE,
|
||||
self.envvars.IOTHUB_CONNECTION_INFO.SharedAccessKeyName, self.envvars.IOT_REST_API_VERSION)
|
||||
self.output.footer("DEPLOY COMPLETE")
|
||||
|
||||
def deploy_device_configuration(
|
||||
self, host_name, iothub_key, device_id, config_file, iothub_policy_name, api_version):
|
||||
resource_uri = host_name
|
||||
token_expiration_period = 60
|
||||
deploy_uri = "https://{0}/devices/{1}/applyConfigurationContent?api-version={2}".format(
|
||||
resource_uri, device_id, api_version)
|
||||
iot_hub_sas_token = self.utility.get_iot_hub_sas_token(
|
||||
resource_uri, iothub_key, iothub_policy_name, token_expiration_period)
|
||||
|
||||
deploy_response = requests.post(deploy_uri,
|
||||
headers={
|
||||
"Authorization": iot_hub_sas_token,
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
data=self.utility.get_file_contents(
|
||||
config_file)
|
||||
)
|
||||
|
||||
if deploy_response.status_code == 204:
|
||||
self.output.info(
|
||||
"Edge Device configuration successfully deployed to '{0}'.".format(device_id))
|
||||
else:
|
||||
self.output.info(deploy_uri)
|
||||
self.output.info(deploy_response.status_code)
|
||||
self.output.info(deploy_response.text)
|
||||
self.output.error(
|
||||
"There was an error deploying the configuration. Please make sure your IOTHUB_CONNECTION_STRING and DEVICE_CONNECTION_STRING Environment Variables are correct.")
|
||||
|
||||
|
||||
class Docker:
|
||||
|
||||
def __init__(self, envvars, utility, output):
|
||||
self.envvars = envvars
|
||||
self.envvars.check()
|
||||
self.utility = utility
|
||||
self.utility.set_config()
|
||||
self.output = output
|
||||
|
||||
if self.envvars.DOCKER_HOST:
|
||||
self.docker_client = docker.DockerClient(
|
||||
base_url=self.envvars.DOCKER_HOST)
|
||||
self.docker_api = docker.APIClient(
|
||||
base_url=self.envvars.DOCKER_HOST)
|
||||
else:
|
||||
self.docker_client = docker.from_env()
|
||||
self.docker_api = docker.APIClient()
|
||||
|
||||
def init_registry(self):
|
||||
|
||||
self.output.header("INITIALIZING CONTAINER REGISTRY")
|
||||
self.output.info("REGISTRY: " + self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
|
||||
if "localhost" in self.envvars.CONTAINER_REGISTRY_SERVER:
|
||||
self.init_local_registry()
|
||||
|
||||
self.login_registry()
|
||||
self.output.line()
|
||||
|
||||
def init_local_registry(self):
|
||||
|
||||
parts = self.envvars.CONTAINER_REGISTRY_SERVER.split(":")
|
||||
|
||||
if len(parts) < 2:
|
||||
self.output.error("You must specific a port for your local registry server. Expected: 'localhost:5000'. Found: " +
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
sys.exit()
|
||||
|
||||
port = parts[1]
|
||||
ports = {'{0}/tcp'.format(port): int(port)}
|
||||
|
||||
try:
|
||||
self.output.info("Looking for local 'registry' container")
|
||||
self.docker_client.containers.get("registry")
|
||||
self.output.info("Found local 'registry' container")
|
||||
except docker.errors.NotFound:
|
||||
self.output.info("Local 'registry' container not found")
|
||||
|
||||
try:
|
||||
self.output.info("Looking for local 'registry' image")
|
||||
self.docker_client.images.get("registry:2")
|
||||
self.output.info("Local 'registry' image found")
|
||||
except docker.errors.ImageNotFound:
|
||||
self.output.info("Local 'registry' image not found")
|
||||
self.output.info("Pulling 'registry' image")
|
||||
self.docker_client.images.pull("registry", tag="2")
|
||||
|
||||
self.output.info("Running registry container")
|
||||
self.docker_client.containers.run(
|
||||
"registry:2", detach=True, name="registry", ports=ports, restart_policy={"Name": "always"})
|
||||
|
||||
def login_registry(self):
|
||||
try:
|
||||
|
||||
if "localhost" in self.envvars.CONTAINER_REGISTRY_SERVER:
|
||||
client_login_status = self.docker_client.login(
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
|
||||
api_login_status = self.docker_api.login(
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
else:
|
||||
|
||||
client_login_status = self.docker_client.login(registry=self.envvars.CONTAINER_REGISTRY_SERVER,
|
||||
username=self.envvars.CONTAINER_REGISTRY_USERNAME,
|
||||
password=self.envvars.CONTAINER_REGISTRY_PASSWORD)
|
||||
|
||||
api_login_status = self.docker_api.login(registry=self.envvars.CONTAINER_REGISTRY_SERVER,
|
||||
username=self.envvars.CONTAINER_REGISTRY_USERNAME,
|
||||
password=self.envvars.CONTAINER_REGISTRY_PASSWORD)
|
||||
|
||||
self.output.info("Successfully logged into container registry: " +
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER)
|
||||
|
||||
except Exception as ex:
|
||||
self.output.error(
|
||||
"ERROR: Could not login to Container Registry. 1. Make sure Docker is running locally. 2. Verify your credentials in CONTAINER_REGISTRY_ environment variables. 2. If you are using WSL, then please set DOCKER_HOST Environment Variable. See the projects readme for full instructions.")
|
||||
self.output.error(str(ex))
|
||||
sys.exit(-1)
|
||||
|
||||
def setup_registry(self):
|
||||
self.output.header("SETTING UP CONTAINER REGISTRY")
|
||||
self.init_registry()
|
||||
self.output.info("PUSHING EDGE IMAGES TO CONTAINER REGISTRY")
|
||||
image_names = ["azureiotedge-agent", "azureiotedge-hub",
|
||||
"azureiotedge-simulated-temperature-sensor"]
|
||||
|
||||
for image_name in image_names:
|
||||
|
||||
microsoft_image_name = "microsoft/{0}:{1}".format(
|
||||
image_name, self.envvars.RUNTIME_TAG)
|
||||
|
||||
container_registry_image_name = "{0}/{1}:{2}".format(
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER, image_name, self.envvars.RUNTIME_TAG)
|
||||
|
||||
# Pull image from Microsoft Docker Hub
|
||||
try:
|
||||
self.output.info(
|
||||
"PULLING IMAGE: '{0}'".format(microsoft_image_name))
|
||||
image_pull = self.docker_client.images.pull(
|
||||
microsoft_image_name)
|
||||
self.output.info(
|
||||
"SUCCESSFULLY PULLED IMAGE: '{0}'".format(microsoft_image_name))
|
||||
self.output.info(str(image_pull))
|
||||
except docker.errors.APIError as e:
|
||||
self.output.error(
|
||||
"ERROR WHILE PULLING IMAGE: '{0}'".format(microsoft_image_name))
|
||||
self.output.error(e)
|
||||
|
||||
# Tagging Image with Container Registry Name
|
||||
try:
|
||||
tag_result = self.docker_api.tag(
|
||||
image=microsoft_image_name, repository=container_registry_image_name)
|
||||
except docker.errors.APIError as e:
|
||||
self.output.error(
|
||||
"ERROR WHILE TAGGING IMAGE: '{0}'".format(microsoft_image_name))
|
||||
self.output.error(e)
|
||||
|
||||
# Push Image to Container Registry
|
||||
try:
|
||||
self.output.info("PUSHING IMAGE: '{0}'".format(
|
||||
container_registry_image_name))
|
||||
|
||||
for line in self.docker_client.images.push(repository=container_registry_image_name, tag=self.envvars.RUNTIME_TAG, stream=True, auth_config={"username": self.envvars.CONTAINER_REGISTRY_USERNAME, "password": self.envvars.CONTAINER_REGISTRY_PASSWORD}):
|
||||
self.output.procout(self.utility.decode(line))
|
||||
|
||||
self.output.info("SUCCESSFULLY PUSHED IMAGE: '{0}'".format(
|
||||
container_registry_image_name))
|
||||
except docker.errors.APIError as e:
|
||||
self.output.error("ERROR WHILE PUSHING IMAGE: '{0}'".format(
|
||||
container_registry_image_name))
|
||||
self.output.error(e)
|
||||
|
||||
self.setup_registry_in_config(image_names)
|
||||
|
||||
self.utility.set_config(force=True)
|
||||
|
||||
self.output.footer("Container Registry Setup Complete")
|
||||
|
||||
def setup_registry_in_config(self, image_names):
|
||||
self.output.info(
|
||||
"Replacing 'microsoft/' with '{CONTAINER_REGISTRY_SERVER}/' in config files.")
|
||||
|
||||
# Replace microsoft/ with ${CONTAINER_REGISTRY_SERVER}
|
||||
for config_file in self.utility.get_config_files():
|
||||
config_file_contents = self.utility.get_file_contents(config_file)
|
||||
for image_name in image_names:
|
||||
config_file_contents = config_file_contents.replace(
|
||||
"microsoft/" + image_name, "${CONTAINER_REGISTRY_SERVER}/" + image_name)
|
||||
|
||||
with open(config_file, "w") as config_file_build:
|
||||
config_file_build.write(config_file_contents)
|
||||
|
||||
def remove_modules(self):
|
||||
self.output.info(
|
||||
"Removing Edge Modules Containers and Images from Docker")
|
||||
|
||||
modules_in_config = self.utility.get_modules_in_config(ModuleType.User)
|
||||
|
||||
for module in modules_in_config:
|
||||
self.output.info("Searching for {0} Containers".format(module))
|
||||
containers = self.docker_client.containers.list(
|
||||
filters={"name": module})
|
||||
for container in containers:
|
||||
self.output.info("Removing Container: " + str(container))
|
||||
container.remove(force=True)
|
||||
|
||||
self.output.info("Searching for {0} Images".format(module))
|
||||
for image in self.docker_client.images.list():
|
||||
if module in str(image):
|
||||
self.output.info(
|
||||
"Removing Module Image: " + str(image))
|
||||
self.docker_client.images.remove(
|
||||
image=image.id, force=True)
|
||||
|
||||
def remove_containers(self):
|
||||
self.output.info("Removing Containers....")
|
||||
containers = self.docker_client.containers.list(all=True)
|
||||
self.output.info("Found {0} Containers".format(len(containers)))
|
||||
for container in containers:
|
||||
self.output.info("Removing Container: {0}:{1}".format(
|
||||
container.id, container.name))
|
||||
container.remove(force=True)
|
||||
self.output.info("Containers Removed")
|
||||
|
||||
def remove_images(self):
|
||||
self.output.info("Removing Dangling Images....")
|
||||
images = self.docker_client.images.list(
|
||||
all=True, filters={"dangling": True})
|
||||
self.output.info("Found {0} Images".format(len(images)))
|
||||
|
||||
for image in images:
|
||||
self.output.info("Removing Image: {0}".format(str(image.id)))
|
||||
self.docker_client.images.remove(image=image.id, force=True)
|
||||
self.output.info("Images Removed")
|
||||
|
||||
self.output.info("Removing Images....")
|
||||
images = self.docker_client.images.list()
|
||||
self.output.info("Found {0} Images".format(len(images)))
|
||||
|
||||
for image in images:
|
||||
self.output.info("Removing Image: {0}".format(str(image.id)))
|
||||
self.docker_client.images.remove(image=image.id, force=True)
|
||||
self.output.info("Images Removed")
|
||||
|
||||
def handle_logs_cmd(self, show, save):
|
||||
|
||||
# Create LOGS_PATH dir if it doesn't exist
|
||||
if save and not os.path.exists(self.envvars.LOGS_PATH):
|
||||
os.makedirs(self.envvars.LOGS_PATH)
|
||||
|
||||
modules_in_config = self.utility.get_modules_in_config(ModuleType.Both)
|
||||
|
||||
for module in modules_in_config:
|
||||
if show:
|
||||
try:
|
||||
command = self.envvars.LOGS_CMD.format(module)
|
||||
os.system(command)
|
||||
except Exception as e:
|
||||
self.output.error(
|
||||
"Error while trying to open module log '{0}' with command '{1}'. Try iotedgedev docker --save-logs instead.".format(module, command))
|
||||
self.output.error(e)
|
||||
if save:
|
||||
try:
|
||||
self.utility.exe_proc(["docker", "logs", module, ">",
|
||||
os.path.join(self.envvars.LOGS_PATH, module + ".log")], True)
|
||||
except Exception as e:
|
||||
self.output.error(
|
||||
"Error while trying to save module log file '{0}'".format(module))
|
||||
self.output.error(e)
|
||||
|
||||
if save:
|
||||
self.zip_logs()
|
||||
|
||||
def zip_logs(self):
|
||||
log_files = [os.path.join(self.envvars.LOGS_PATH, f)
|
||||
for f in os.listdir(self.envvars.LOGS_PATH) if f.endswith(".log")]
|
||||
zip_path = os.path.join(self.envvars.LOGS_PATH, 'edge-logs.zip')
|
||||
|
||||
self.output.info("Creating {0} file".format(zip_path))
|
||||
|
||||
zipf = zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DEFLATED)
|
||||
|
||||
for log_file in log_files:
|
||||
self.output.info("Adding {0} to zip". format(log_file))
|
||||
zipf.write(log_file)
|
||||
|
||||
zipf.close()
|
||||
|
||||
self.output.info("Log files successfully saved to: " + zip_path)
|
||||
|
||||
|
||||
class ModuleType(Enum):
|
||||
System = 1
|
||||
User = 2
|
||||
Both = 3
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
class IoTHub:
|
||||
def __init__(self, envvars, output, utility):
|
||||
self.envvars = envvars
|
||||
self.envvars.check()
|
||||
self.output = output
|
||||
self.utility = utility
|
||||
|
||||
def monitor_events(self):
|
||||
try:
|
||||
self.utility.call_proc(['iothub-explorer', '--login', self.envvars.IOTHUB_CONNECTION_STRING, 'monitor-events', self.envvars.DEVICE_CONNECTION_INFO.DeviceId], shell=True)
|
||||
except Exception as ex:
|
||||
self.output.error("Problem while trying to call iothub-explorer. Please ensure that you have installed the iothub-explorer npm package with: npm i -g iothub-explorer.")
|
||||
self.output.error(str(ex))
|
||||
|
|
@ -0,0 +1,165 @@
|
|||
import os
|
||||
import requests
|
||||
from shutil import copyfile
|
||||
|
||||
class Modules:
|
||||
def __init__(self, envvars, utility, output, dock):
|
||||
self.envvars = envvars
|
||||
self.envvars.check()
|
||||
self.utility = utility
|
||||
self.utility.set_config()
|
||||
self.output = output
|
||||
self.dock = dock
|
||||
self.dock.init_registry()
|
||||
|
||||
def build(self):
|
||||
self.output.header("BUILDING MODULES")
|
||||
|
||||
# Get all the modules to build as specified in config.
|
||||
modules_to_process = self.utility.get_active_modules()
|
||||
|
||||
for module in os.listdir(self.envvars.MODULES_PATH):
|
||||
|
||||
if len(
|
||||
modules_to_process) == 0 or modules_to_process[0] == "*" or module in modules_to_process:
|
||||
|
||||
module_dir = os.path.join(self.envvars.MODULES_PATH, module)
|
||||
|
||||
self.output.info("BUILDING MODULE: {0}".format(module_dir))
|
||||
|
||||
# Find first proj file in module dir and use it.
|
||||
project_files = [os.path.join(module_dir, f) for f in os.listdir(
|
||||
module_dir) if f.endswith("proj")]
|
||||
|
||||
if len(project_files) == 0:
|
||||
self.output.error("No project file found for module.")
|
||||
continue
|
||||
|
||||
self.utility.exe_proc(["dotnet", "build", project_files[0],
|
||||
"-v", self.envvars.DOTNET_VERBOSITY])
|
||||
|
||||
# Get all docker files in project
|
||||
docker_files = self.utility.find_files(
|
||||
module_dir, "Dockerfile*")
|
||||
|
||||
# Filter by Docker Dirs in envvars
|
||||
docker_dirs_process = [docker_dir.strip()
|
||||
for docker_dir in self.envvars.ACTIVE_DOCKER_DIRS.split(",") if docker_dir]
|
||||
|
||||
# Process each Dockerfile found
|
||||
for docker_file in docker_files:
|
||||
|
||||
docker_file_parent_folder = os.path.basename(
|
||||
os.path.dirname(docker_file))
|
||||
|
||||
if len(
|
||||
docker_dirs_process) == 0 or docker_dirs_process[0] == "*" or docker_file_parent_folder in docker_dirs_process:
|
||||
|
||||
self.output.info(
|
||||
"PROCESSING DOCKER FILE: " + docker_file)
|
||||
|
||||
docker_file_name = os.path.basename(docker_file)
|
||||
|
||||
# assume /Docker/{runtime}/Dockerfile folder structure
|
||||
# image name will be the same as the module folder name, filter-module
|
||||
# tag will be {runtime}{ext}{container_tag}, i.e. linux-x64-debug-jong
|
||||
# runtime is the Dockerfile immediate parent folder name
|
||||
# ext is Dockerfile extension for example with Dockerfile.debug, debug is the mod
|
||||
# CONTAINER_TAG is env var
|
||||
|
||||
# i.e. when found: filter-module/Docker/linux-x64/Dockerfile.debug and CONTAINER_TAG = jong
|
||||
# we'll get: filtermodule:linux-x64-debug-jong
|
||||
|
||||
runtime = os.path.basename(
|
||||
os.path.dirname(docker_file))
|
||||
ext = "" if os.path.splitext(docker_file)[
|
||||
1] == "" else "-" + os.path.splitext(docker_file)[1][1:]
|
||||
container_tag = "" if self.envvars.CONTAINER_TAG == "" else "-" + \
|
||||
self.envvars.CONTAINER_TAG
|
||||
|
||||
tag_name = runtime + ext + container_tag
|
||||
|
||||
# construct the build output path
|
||||
build_path = os.path.join(
|
||||
os.getcwd(), "build", "modules", module, runtime)
|
||||
if not os.path.exists(build_path):
|
||||
os.makedirs(build_path)
|
||||
|
||||
# dotnet publish
|
||||
self.output.info(
|
||||
"PUBLISHING PROJECT: " + project_files[0])
|
||||
|
||||
self.utility.exe_proc(["dotnet", "publish", project_files[0], "-f", "netcoreapp2.0",
|
||||
"-o", build_path, "-v", self.envvars.DOTNET_VERBOSITY])
|
||||
|
||||
# copy Dockerfile to publish dir
|
||||
build_dockerfile = os.path.join(
|
||||
build_path, docker_file_name)
|
||||
|
||||
copyfile(docker_file, build_dockerfile)
|
||||
|
||||
image_destination_name = "{0}/{1}:{2}".format(
|
||||
self.envvars.CONTAINER_REGISTRY_SERVER, module, tag_name).lower()
|
||||
|
||||
self.output.info(
|
||||
"BUILDING DOCKER IMAGE: " + image_destination_name)
|
||||
|
||||
# cd to the build output to build the docker image
|
||||
project_dir = os.getcwd()
|
||||
os.chdir(build_path)
|
||||
|
||||
# BUILD DOCKER IMAGE
|
||||
build_result = self.dock.docker_client.images.build(
|
||||
tag=image_destination_name, path=".", dockerfile=docker_file_name)
|
||||
self.output.info(
|
||||
"DOCKER IMAGE DETAILS: {0}".format(build_result))
|
||||
|
||||
# CD BACK UP
|
||||
os.chdir(project_dir)
|
||||
|
||||
# PUSH TO CONTAINER REGISTRY
|
||||
self.output.info(
|
||||
"PUSHING DOCKER IMAGE TO: " + image_destination_name)
|
||||
|
||||
for line in self.dock.docker_client.images.push(repository=image_destination_name, tag=tag_name, stream=True, auth_config={
|
||||
"username": self.envvars.CONTAINER_REGISTRY_USERNAME, "password": self.envvars.CONTAINER_REGISTRY_PASSWORD}):
|
||||
self.output.procout(self.utility.decode(line))
|
||||
|
||||
self.output.footer("BUILD COMPLETE")
|
||||
|
||||
def deploy(self):
|
||||
self.output.header("DEPLOYING MODULES")
|
||||
self.deploy_device_configuration(
|
||||
self.envvars.IOTHUB_CONNECTION_INFO.HostName, self.envvars.IOTHUB_CONNECTION_INFO.SharedAccessKey,
|
||||
self.envvars.DEVICE_CONNECTION_INFO.DeviceId, self.envvars.MODULES_CONFIG_FILE,
|
||||
self.envvars.IOTHUB_CONNECTION_INFO.SharedAccessKeyName, self.envvars.IOT_REST_API_VERSION)
|
||||
self.output.footer("DEPLOY COMPLETE")
|
||||
|
||||
def deploy_device_configuration(
|
||||
self, host_name, iothub_key, device_id, config_file, iothub_policy_name, api_version):
|
||||
resource_uri = host_name
|
||||
token_expiration_period = 60
|
||||
deploy_uri = "https://{0}/devices/{1}/applyConfigurationContent?api-version={2}".format(
|
||||
resource_uri, device_id, api_version)
|
||||
iot_hub_sas_token = self.utility.get_iot_hub_sas_token(
|
||||
resource_uri, iothub_key, iothub_policy_name, token_expiration_period)
|
||||
|
||||
deploy_response = requests.post(deploy_uri,
|
||||
headers={
|
||||
"Authorization": iot_hub_sas_token,
|
||||
"Content-Type": "application/json"
|
||||
},
|
||||
data=self.utility.get_file_contents(
|
||||
config_file)
|
||||
)
|
||||
|
||||
if deploy_response.status_code == 204:
|
||||
self.output.info(
|
||||
"Edge Device configuration successfully deployed to '{0}'.".format(device_id))
|
||||
else:
|
||||
self.output.info(deploy_uri)
|
||||
self.output.info(deploy_response.status_code)
|
||||
self.output.info(deploy_response.text)
|
||||
self.output.error(
|
||||
"There was an error deploying the configuration. Please make sure your IOTHUB_CONNECTION_STRING and DEVICE_CONNECTION_STRING Environment Variables are correct.")
|
||||
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
from enum import Enum
|
||||
|
||||
class ModuleType(Enum):
|
||||
System = 1
|
||||
User = 2
|
||||
Both = 3
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
import click
|
||||
|
||||
class Output:
|
||||
|
||||
def info(self, text):
|
||||
click.secho(text, fg='yellow')
|
||||
|
||||
def error(self, text):
|
||||
click.secho("ERROR: " + text, fg='red')
|
||||
|
||||
def header(self, text):
|
||||
click.secho("======== {0} ========".format(text).upper(), fg='white')
|
||||
|
||||
def footer(self, text):
|
||||
self.info(text.upper())
|
||||
self.line()
|
||||
|
||||
def procout(self, text):
|
||||
click.secho(text, dim=True)
|
||||
|
||||
def line(self):
|
||||
click.secho("")
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
import os
|
||||
import zipfile
|
||||
|
||||
class Project:
|
||||
def __init__(self, output):
|
||||
self.output = output
|
||||
|
||||
def create(self, name):
|
||||
self.output.header("CREATING AZURE IOT EDGE PROJECT")
|
||||
|
||||
try:
|
||||
template_zip = os.path.join(os.path.split(
|
||||
__file__)[0], "template", "template.zip")
|
||||
except Exception as ex:
|
||||
self.output.error("Error while trying to load template.zip")
|
||||
self.output.error(str(ex))
|
||||
|
||||
if name == ".":
|
||||
name = ""
|
||||
|
||||
zipf = zipfile.ZipFile(template_zip)
|
||||
zipf.extractall(name)
|
||||
|
||||
os.rename(os.path.join(name, ".env.tmp"), os.path.join(name, ".env"))
|
||||
self.output.footer("Azure IoT Edge project created")
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
class Runtime:
|
||||
def __init__(self, envvars, utility, output, dock):
|
||||
self.envvars = envvars
|
||||
self.envvars.check()
|
||||
self.utility = utility
|
||||
self.utility.set_config()
|
||||
self.dock = dock
|
||||
self.output = output
|
||||
|
||||
def start(self):
|
||||
self.output.header("Starting Edge Runtime")
|
||||
self.utility.exe_proc(["iotedgectl", "--verbose",
|
||||
self.envvars.RUNTIME_VERBOSITY, "start"])
|
||||
|
||||
def stop(self):
|
||||
self.output.header("Stopping Edge Runtime")
|
||||
self.utility.exe_proc(["iotedgectl", "--verbose",
|
||||
self.envvars.RUNTIME_VERBOSITY, "stop"])
|
||||
|
||||
def setup(self):
|
||||
self.output.header("Setting Up Edge Runtime")
|
||||
self.utility.exe_proc(["iotedgectl", "--verbose", self.envvars.RUNTIME_VERBOSITY,
|
||||
"setup", "--config-file", self.envvars.RUNTIME_CONFIG_FILE])
|
||||
|
||||
def status(self):
|
||||
self.output.header("Getting Edge Runtime Status")
|
||||
self.utility.exe_proc(["iotedgectl", "--verbose", self.envvars.RUNTIME_VERBOSITY,
|
||||
"status"])
|
||||
|
||||
def restart(self):
|
||||
self.stop()
|
||||
self.dock.remove_modules()
|
||||
self.setup()
|
||||
self.start()
|
||||
|
||||
|
|
@ -6,4 +6,5 @@
|
|||
build
|
||||
.env
|
||||
venv
|
||||
logs
|
||||
logs
|
||||
.config
|
||||
Двоичные данные
iotedgedev/template/template.zip
Двоичные данные
iotedgedev/template/template.zip
Двоичный файл не отображается.
|
|
@ -0,0 +1,135 @@
|
|||
from base64 import b64encode, b64decode
|
||||
import fnmatch
|
||||
from hashlib import sha256
|
||||
from hmac import HMAC
|
||||
import json
|
||||
import os
|
||||
import requests
|
||||
import subprocess
|
||||
import sys
|
||||
from time import time
|
||||
if sys.version_info.major >= 3:
|
||||
from urllib.parse import quote, urlencode
|
||||
else:
|
||||
from urllib import quote, urlencode
|
||||
from .moduletype import ModuleType
|
||||
|
||||
class Utility:
|
||||
def __init__(self, envvars, output):
|
||||
self.envvars = envvars
|
||||
self.envvars.check()
|
||||
self.output = output
|
||||
self.config_set = False
|
||||
|
||||
def exe_proc(self, params, shell=False):
|
||||
proc = subprocess.Popen(
|
||||
params, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=shell)
|
||||
|
||||
stdout_data, stderr_data = proc.communicate()
|
||||
if stdout_data != "":
|
||||
self.output.procout(self.decode(stdout_data))
|
||||
|
||||
if proc.returncode != 0:
|
||||
self.output.error(self.decode(stderr_data))
|
||||
sys.exit()
|
||||
|
||||
def call_proc(self, params, shell=False):
|
||||
subprocess.check_call(params, shell=shell)
|
||||
|
||||
def find_files(self, directory, pattern):
|
||||
# find all files in directory that match the pattern.
|
||||
for root, dirs, files in os.walk(directory):
|
||||
for basename in files:
|
||||
if fnmatch.fnmatch(basename, pattern):
|
||||
filename = os.path.join(root, basename)
|
||||
yield filename
|
||||
|
||||
def get_iot_hub_sas_token(self, uri, key, policy_name, expiry=3600):
|
||||
ttl = time() + expiry
|
||||
sign_key = "%s\n%d" % ((quote(uri)), int(ttl))
|
||||
signature = b64encode(
|
||||
HMAC(b64decode(key), sign_key.encode("utf-8"), sha256).digest())
|
||||
|
||||
rawtoken = {
|
||||
"sr": uri,
|
||||
"sig": signature,
|
||||
"se": str(int(ttl))
|
||||
}
|
||||
|
||||
if policy_name is not None:
|
||||
rawtoken["skn"] = policy_name
|
||||
|
||||
return "SharedAccessSignature " + urlencode(rawtoken)
|
||||
|
||||
def get_file_contents(self, file):
|
||||
with open(file, "r") as file:
|
||||
return file.read()
|
||||
|
||||
def decode(self, val):
|
||||
return val.decode("utf-8").strip()
|
||||
|
||||
def get_config_files(self):
|
||||
# config files are in root of project
|
||||
|
||||
return [os.path.join(os.getcwd(), f) for f in os.listdir(os.getcwd()) if f.endswith("template.json")]
|
||||
|
||||
def get_active_modules(self):
|
||||
return [module.strip()
|
||||
for module in self.envvars.ACTIVE_MODULES.split(",") if module]
|
||||
|
||||
def get_modules_in_config(self, moduleType):
|
||||
modules_config = json.load(open(self.envvars.MODULES_CONFIG_FILE))
|
||||
|
||||
props = modules_config["moduleContent"]["$edgeAgent"]["properties.desired"]
|
||||
|
||||
system_modules = props["systemModules"]
|
||||
user_modules = props["modules"]
|
||||
|
||||
if moduleType == ModuleType.System:
|
||||
return system_modules
|
||||
elif moduleType == ModuleType.User:
|
||||
return user_modules
|
||||
else:
|
||||
return_modules = {}
|
||||
return_modules.update(system_modules)
|
||||
return_modules.update(user_modules)
|
||||
return return_modules
|
||||
|
||||
def set_config(self, force=False):
|
||||
|
||||
if not self.config_set or force:
|
||||
self.envvars.check()
|
||||
self.output.header("PROCESSING CONFIG FILES")
|
||||
|
||||
config_output_dir = ".config"
|
||||
|
||||
# Create config dir if it doesn't exist
|
||||
if not os.path.exists(config_output_dir):
|
||||
os.makedirs(config_output_dir)
|
||||
|
||||
config_files = self.get_config_files()
|
||||
|
||||
if len(config_files) == 0:
|
||||
self.output.info(
|
||||
"Unable to find config files in project root directory")
|
||||
sys.exit()
|
||||
|
||||
# Expand envars and rewrite to .config/
|
||||
for config_file in config_files:
|
||||
|
||||
build_config_file = os.path.join(
|
||||
config_output_dir, os.path.basename(config_file).replace(".template", ""))
|
||||
|
||||
self.output.info("Expanding '{0}' to '{1}'".format(
|
||||
os.path.basename(config_file), build_config_file))
|
||||
|
||||
config_file_expanded = os.path.expandvars(
|
||||
self.get_file_contents(config_file))
|
||||
|
||||
with open(build_config_file, "w") as config_file_build:
|
||||
config_file_build.write(config_file_expanded)
|
||||
|
||||
self.output.line()
|
||||
|
||||
self.config_set = True
|
||||
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
{
|
||||
"name": "azure-iot-edge-dev-tool",
|
||||
"version": "1.0.0",
|
||||
"description": "The **Azure IoT Edge Dev Tool** greatly simplifies [Azure IoT Edge](https://azure.microsoft.com/en-us/services/iot-edge/) development down to simple CLI commands driven by Environment Variables.",
|
||||
"main": "index.js",
|
||||
"directories": {
|
||||
"test": "tests"
|
||||
},
|
||||
"scripts": {
|
||||
"test": "echo \"Error: no test specified\" && exit 1"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/jonbgallant/azure-iot-edge-dev-tool.git"
|
||||
},
|
||||
"author": "",
|
||||
"license": "MIT",
|
||||
"bugs": {
|
||||
"url": "https://github.com/jonbgallant/azure-iot-edge-dev-tool/issues"
|
||||
},
|
||||
"homepage": "https://github.com/jonbgallant/azure-iot-edge-dev-tool#readme",
|
||||
"dependencies": {
|
||||
"iothub-explorer": "^1.1.20"
|
||||
}
|
||||
}
|
||||
|
|
@ -9,7 +9,6 @@ import os
|
|||
import shutil
|
||||
from click.testing import CliRunner
|
||||
|
||||
from iotedgedev import iotedgedev
|
||||
from iotedgedev import cli
|
||||
|
||||
from dotenv import load_dotenv, find_dotenv
|
||||
|
|
@ -34,8 +33,8 @@ class TestIotedgedev(unittest.TestCase):
|
|||
shutil.copyfile('.env', os.path.join(os.getcwd(), project, '.env'))
|
||||
os.chdir(project)
|
||||
|
||||
except Exception as e:
|
||||
print(e)
|
||||
except Exception as ex:
|
||||
print(str(ex))
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(self):
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче