microsoft
/
data-accelerator
зеркало из https://github.com/microsoft/data-accelerator.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
Страница: Cloud deployment
Страницы
Arm Parameters Azure deployment Cloud Deployment On Linux Cloud Simulator Cloud deployment Configuring the Arm template Create new metric Creating your first pipeline in 5 minutes! Data Accelerator with Databricks Data Accelerator Data Accumulator Diagnose issues using Telemetry FAQ Find Applications For Your Environment Home Inviting others and RBAC Live query Local Cloud Debugging Local Tutorial Add an Alert Local Tutorial Adding SQL to your flow and outputs to Metrics dashboard Local Tutorial Advanced Aggregate alerts Local Tutorial Creating your first Flow in local mode Local Tutorial Custom schema Local Tutorial Debugging using Spark logs Local Tutorial Extending with UDF UDAF custom code Local Tutorial Outputs to disk Local Tutorial Reference data Local Tutorial Scaling the docker host Local Tutorial Tag Aggregate to metrics Local Tutorial Tag Rules output to local file Local create metric Local mode with Docker Local running sample Output data to Azure SQL Database Run Data Accelerator Flows on Databricks Scale Schedule batch job Set up aggregate alert Set up new outputs Set up simple alert Spark logs Tagged data flowing to CosmosDB Tagging aggregate rules Tagging simple rules Tutorials Upgrade Existing Data Accelerator Environment to v1.1 Upgrade Existing Data Accelerator Environment to v1.2 Use Input in different tenant Windowing functions functions readme reference data sql query
26 Cloud deployment
Carl Brochu редактировал(а) эту страницу 2019-10-01 11:12:37 -07:00
Содержание

Unleash the full power of Data Accelerator by deploying it into your subscription in Azure by following the instructions below and get started setting up end to end data pipelines!

Prerequisites

  • Install Azure CLI from here
  • Download the scripts and templates locally via this link: template
  • (For databricks environment only; this is not required for HDInsight environments), Install Databricks CLI from here.

Deployment

  1. Open common.parameters.txt under DeploymentCloud/Deployment.DataX, provide TenantId and SubscriptionId. In order to setup Data Accelerator with Databricks environment, also set useDatabricks = y (See FAQ section below on how to use other parameters.)

  2. For Windows OS, open a command prompt as an admin under the downloaded folder DeploymentCloud/Deployment.DataX and run :

 deploy.bat
  1. If you are not the admin of the tenant (typically when using AAD account), then please copy over the DeploymentCloud folder to your admin's machine and ask your admin to run the following command:
runAdminSteps.bat
  1. For

  • Data Accelerator with HDInsight environment - Once the deployment finishes, open the Data Accelerator Portal via http://name.azurewebsites.net (Url available from the command prompt or via the Azure Portal under the deployed App Service. To find this, go to App Services in http://portal.azure.com, click on the app service called "dx*", and open the URL)

  • Data Accelerator with Databricks environment - If useDatabricks was set to y in common.parameters.txt file before running deploy.bat then the above steps will setup the azure resources required by Data Accelerator including a Databricks resource. To finish setting up databricks resource you will further need to generate databricks token, create a secret scope, upload jars to DBFS which are required to run spark jobs and finally create a databricks cluster for live query. Please refer Data Accelerator with Databricks Environment Setup for complete instructions.

Cloud deployment on Linux and Mac

  • The ARM deployment for DataX currently support only Windows. If you are looking for multiplatform support (Linux and Mac) please reach out to discuss options.

For details, please see Cloud Deployment On Linux

Troubleshooting and FAQ

Data Accelerator

Install

Docs