1. Adding data generation details.

2. Adding data generation source code.

Docs/PostDeploymentInstructions.md

@@ -1,3 +1,12 @@
+<h1 id="overview">Overview</h1>
+The following document describes details on the **Extract-Transform-Load On Azure Solution** deployed via [*Cortana Intelligence Solutions (CIS)*](https://start.cortanaintelligence.com/). It covers the following: 
+
+  * [Architecture](#architecture): A high level description of deployed components, building-blocks and resulting outputs. 
+  * [Data Flow](#dataflow): Describes the datasets created and transforms applied over various services to generate the star-schema model of the source OLTP data.
+  * [Dataset](#datasets): Overview of the Adventure Works OLTP dataset: our source OLTP database for this solution.
+  * [Monitoring](#monitor): Details on monitoring and setting up alarms for your warehousing pipeline.
+  * [Visualizing with Power BI](#pbi-setup): A wallk through on sourcing the OLAP data to visualize a sample *Reseller Sales Dashboard* using Power BI.
+
 <h1 id="architecture">Architecture</h1>
 In this solution, we demonstrate how a hybrid EDW scenario can be implemented on Azure using: 
 * **Azure SQL Data Warehouse** as a Data mart to vend business-line specific data.
@@ -9,7 +18,7 @@ Our scenario includes an Extract-Load-and-Transform (ELT) model. Firstly, we ext
 
 ![High Level Pipeline Architecture](https://github.com/Azure/etlorchestration-cortana-intelligence-preconfigured-solution/blob/master/Docs/figures/Architecture.png)
 
-<h1>Data Flow</h1>
+<h1 id="dataflow">Data Flow</h1>
 The following steps are performed as outlined in the chart above: 
 * **[1->2]** Normalized OLTP data is cloned to Azure Blob storage every hour. Data copied is partitioned by time slice at a 1 hour granularity.
 * **[3->4->5]** Hive external tables are created against the cloned source OLTP data and used to generate dimensions which are writtern back to a Hive transactional table. Refer [here](#batch-loads) for details of the transforms applied. In the incremental pipeline, deltas are reconciled using the procedure outlined [here](#incremental-loads).
@@ -19,10 +28,13 @@ The following steps are performed as outlined in the chart above:
 
 ![Pipeline Data Flow Chart](https://github.com/Azure/etlorchestration-cortana-intelligence-preconfigured-solution/blob/master/Docs/figures/DataFlowChart.png)
 
-<h1>Dataset</h1>
+<h1 id="datasets">Dataset</h1>
 The data used as our OLTP source models a fictious company named 'Adventure Works Cycles'; a large, multinational manufacturing company. The company manufactures and sells metal and composite bicycles to North American, European and Asian commercial markets. Refer [here](https://technet.microsoft.com/en-us/library/ms124825(v=sql.100).aspx) for deeper look at the various business scenarios addressed by this dataset.
 
-To simulate incremental inserts, we deploy a data generator to simulate sales orders being produced in real-time. This will be deployed as a webjob in your subscription.
+### Synthetic Data Generation 
+To simulate incremental inserts, we deploy a data generator to simulate sales orders being produced in real-time. This will be deployed as a webjob in your subscription as part of the solution. You can view the status of this webjob by referring to the [CIS deployment page](https://start.cortanaintelligence.com/Deployments?type=avhivedw). You can also view the [source code here](https://github.com/Azure/etlorchestration-cortana-intelligence-preconfigured-solution/tree/master/Src/SalesDataGenerator/DataGenerator).
+
+![Dataset Figure 1](https://github.com/Azure/etlorchestration-cortana-intelligence-preconfigured-solution/blob/master/Docs/figures/DATASET-1.png)
 
 <h1 id="monitor">Monitoring your Warehousing Pipeline</h1>
 
@@ -83,7 +95,7 @@ You can set up alarms to send email notifications when pipeline activities fail
 <h1>Visualize Using Power BI</h1>
 The generated Fact and Dimension tables can be visualized in Power BI by connecting to the SQL Data Warehouse instance. Refer [this sample Power BI Desktop file](https://github.com/Azure/etlorchestration-cortana-intelligence-preconfigured-solution/tree/master/Power-BI-Templates/AzureEtlOrchestrationSampleDashboard.pbix). See [PBI section](#pbi-setup) for details on wiring it up with your Data Warehouse instance. 
 
-#### Power BI Dashboard <a id="pbi-setup"/>
+<h1 id="pbi-setup">Power BI Dashboard </h1>
 
 Power BI can connect to our data mart hosted on Azure SQL Data Warehouse to visualize the generated Facts and Dimensions. This section describes how to set up the sample Power BI dashboard to visualize the results of the pipeline.
 

Src/SalesDataGenerator/DataGenerator/App.config

@@ -0,0 +1,24 @@
+<?xml version="1.0" encoding="utf-8"?>
+<configuration>
+  <configSections>
+    <!-- For more information on Entity Framework configuration, visit http://go.microsoft.com/fwlink/?LinkID=237468 -->
+    <section name="entityFramework" type="System.Data.Entity.Internal.ConfigFile.EntityFrameworkSection, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
+  </configSections>
+  <startup>
+    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.2" />
+  </startup>
+  <appSettings>
+    <add key="SqlDwServerName" value="" />
+    <add key="SqlDbServerName" value="" />
+    <add key="SqlDwDatabaseName" value="" />
+    <add key="SqlUser" value="" />
+    <add key="SqlPassword" value="" />
+    <add key="ClientSettingsProvider.ServiceUri" value="" />
+  </appSettings>
+  <entityFramework>
+    <defaultConnectionFactory type="System.Data.Entity.Infrastructure.SqlConnectionFactory, EntityFramework" />
+    <providers>
+      <provider invariantName="System.Data.SqlClient" type="System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer" />
+    </providers>
+  </entityFramework>
+</configuration>

Src/SalesDataGenerator/DataGenerator/BackoffPolicies/FixedBackoffPolicy.cs

@@ -0,0 +1,39 @@
+// --------------------------------------------------------------------------------------------------------------------
+// <copyright file="FixedBackoffPolicy.cs" company="Microsoft">
+//   Copyright (c) Microsoft.  All rights reserved.
+// </copyright>
+// <summary>
+//   Backoff policy with fixed interval waits.
+// </summary>
+// --------------------------------------------------------------------------------------------------------------------
+
+namespace DataGenerator.BackoffPolicies
+{
+    using System.Threading;
+
+    /// <summary>
+    ///     Backoff policy with fixed interval waits.
+    /// </summary>
+    internal class FixedBackoffPolicy : IBackoffPolicy
+    {
+        /// <summary>Interval wait period in milliseconds.</summary>
+        private readonly int _millisecondTimeout;
+
+        /// <summary>
+        ///     Initializes an instance of FixedBackoffPolicy.
+        /// </summary>
+        /// <param name="millisecondTimeout">Interval wait period in milliseconds.</param>
+        public FixedBackoffPolicy(int millisecondTimeout)
+        {
+            _millisecondTimeout = millisecondTimeout;
+        }
+
+        /// <summary>
+        ///     Await a fixed time interval.
+        /// </summary>
+        public void Wait()
+        {
+            Thread.Sleep(_millisecondTimeout);
+        }
+    }
+}

Src/SalesDataGenerator/DataGenerator/BackoffPolicies/IBackoffPolicy.cs

@@ -0,0 +1,22 @@
+// --------------------------------------------------------------------------------------------------------------------
+// <copyright file="IBackoffPolicy.cs" company="Microsoft">
+//   Copyright (c) Microsoft.  All rights reserved.
+// </copyright>
+// <summary>
+//   Interface for backoff strategy on repeated loops.
+// </summary>
+// --------------------------------------------------------------------------------------------------------------------
+
+namespace DataGenerator.BackoffPolicies
+{
+    /// <summary>
+    ///     Back off policy interface.
+    /// </summary>
+    internal interface IBackoffPolicy
+    {
+        /// <summary>
+        ///     Await a period based on a strategy.
+        /// </summary>
+        void Wait();
+    }
+}

Src/SalesDataGenerator/DataGenerator/DAL/AdventureWorksModel.cs

@@ -0,0 +1,61 @@
+// --------------------------------------------------------------------------------------------------------------------
+// <copyright file="AdventureWorksModel.cs" company="Microsoft">
+//   Copyright (c) Microsoft.  All rights reserved.
+// </copyright>
+// <summary>
+//   Adventure Works SQL Database OLTP Model.
+// </summary>
+// --------------------------------------------------------------------------------------------------------------------
+namespace DataGenerator.DAL
+{
+    using System.Data.Entity;
+
+    /// <summary>
+    /// Adventure Works SQL Database OLTP Model.
+    /// </summary>
+    public class AdventureWorksModel : DbContext
+    {
+        public AdventureWorksModel(string connectionString)
+            : base(connectionString)
+        {
+        }
+
+        public virtual DbSet<SalesOrderDetail> SalesOrderDetails { get; set; }
+        public virtual DbSet<SalesOrderHeader> SalesOrderHeaders { get; set; }
+
+        protected override void OnModelCreating(DbModelBuilder modelBuilder)
+        {
+            modelBuilder.Entity<SalesOrderDetail>()
+                .Property(e => e.UnitPrice)
+                .HasPrecision(19, 4);
+
+            modelBuilder.Entity<SalesOrderDetail>()
+                .Property(e => e.UnitPriceDiscount)
+                .HasPrecision(19, 4);
+
+            modelBuilder.Entity<SalesOrderDetail>()
+                .Property(e => e.LineTotal)
+                .HasPrecision(38, 6);
+
+            modelBuilder.Entity<SalesOrderHeader>()
+                .Property(e => e.CreditCardApprovalCode)
+                .IsUnicode(false);
+
+            modelBuilder.Entity<SalesOrderHeader>()
+                .Property(e => e.SubTotal)
+                .HasPrecision(19, 4);
+
+            modelBuilder.Entity<SalesOrderHeader>()
+                .Property(e => e.TaxAmt)
+                .HasPrecision(19, 4);
+
+            modelBuilder.Entity<SalesOrderHeader>()
+                .Property(e => e.Freight)
+                .HasPrecision(19, 4);
+
+            modelBuilder.Entity<SalesOrderHeader>()
+                .Property(e => e.TotalDue)
+                .HasPrecision(19, 4);
+        }
+    }
+}

Src/SalesDataGenerator/DataGenerator/DAL/SalesOrderDetail.cs

@@ -0,0 +1,56 @@
+// --------------------------------------------------------------------------------------------------------------------
+// <copyright file="AdventureWorksModel.cs" company="Microsoft">
+//   Copyright (c) Microsoft.  All rights reserved.
+// </copyright>
+// <summary>
+//   Adventure Works SQL Database OLTP Model.
+// </summary>
+// --------------------------------------------------------------------------------------------------------------------
+namespace DataGenerator.DAL
+{
+    using System;
+    using System.ComponentModel.DataAnnotations;
+    using System.ComponentModel.DataAnnotations.Schema;
+
+    /// <summary>
+    /// Sales Order Detail DAO.
+    /// </summary>
+    [Table("Sales.SalesOrderDetail")]
+    public class SalesOrderDetail
+    {
+        [Key]
+        [Column(Order = 0)]
+        [DatabaseGenerated(DatabaseGeneratedOption.None)]
+        public int SalesOrderID { get; set; }
+
+        [Key]
+        [Column(Order = 1)]
+        [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
+        public int SalesOrderDetailID { get; set; }
+
+        [StringLength(25)]
+        public string CarrierTrackingNumber { get; set; }
+
+        public short OrderQty { get; set; }
+
+        public int ProductID { get; set; }
+
+        public int SpecialOfferID { get; set; }
+
+        [Column(TypeName = "money")]
+        public decimal UnitPrice { get; set; }
+
+        [Column(TypeName = "money")]
+        public decimal UnitPriceDiscount { get; set; }
+
+        [Column(TypeName = "numeric")]
+        [DatabaseGenerated(DatabaseGeneratedOption.Computed)]
+        public decimal LineTotal { get; set; }
+
+        public Guid rowguid { get; set; }
+
+        public DateTime ModifiedDate { get; set; }
+
+        public virtual SalesOrderHeader SalesOrderHeader { get; set; }
+    }
+}

Src/SalesDataGenerator/DataGenerator/DAL/SalesOrderHeader.cs

@@ -0,0 +1,96 @@
+// --------------------------------------------------------------------------------------------------------------------
+// <copyright file="AdventureWorksModel.cs" company="Microsoft">
+//   Copyright (c) Microsoft.  All rights reserved.
+// </copyright>
+// <summary>
+//   Adventure Works SQL Database OLTP Model.
+// </summary>
+// --------------------------------------------------------------------------------------------------------------------
+namespace DataGenerator.DAL
+{
+    using System;
+    using System.Collections.Generic;
+    using System.ComponentModel.DataAnnotations;
+    using System.ComponentModel.DataAnnotations.Schema;
+    using System.Diagnostics.CodeAnalysis;
+
+    /// <summary>
+    /// Sales Order Header DAO.
+    /// </summary>
+    [Table("Sales.SalesOrderHeader")]
+    public class SalesOrderHeader
+    {
+        [SuppressMessage("Microsoft.Usage", "CA2214:DoNotCallOverridableMethodsInConstructors")]
+        public SalesOrderHeader()
+        {
+            SalesOrderDetails = new HashSet<SalesOrderDetail>();
+        }
+
+        [Key]
+        public int SalesOrderID { get; set; }
+
+        public byte RevisionNumber { get; set; }
+
+        public DateTime OrderDate { get; set; }
+
+        public DateTime DueDate { get; set; }
+
+        public DateTime? ShipDate { get; set; }
+
+        public byte Status { get; set; }
+
+        public bool OnlineOrderFlag { get; set; }
+
+        [DatabaseGenerated(DatabaseGeneratedOption.Computed)]
+        [StringLength(25)]
+        public string SalesOrderNumber { get; set; }
+
+        [StringLength(25)]
+        public string PurchaseOrderNumber { get; set; }
+
+        [StringLength(15)]
+        public string AccountNumber { get; set; }
+
+        public int CustomerID { get; set; }
+
+        public int? SalesPersonID { get; set; }
+
+        public int? TerritoryID { get; set; }
+
+        public int BillToAddressID { get; set; }
+
+        public int ShipToAddressID { get; set; }
+
+        public int ShipMethodID { get; set; }
+
+        public int? CreditCardID { get; set; }
+
+        [StringLength(15)]
+        public string CreditCardApprovalCode { get; set; }
+
+        public int? CurrencyRateID { get; set; }
+
+        [Column(TypeName = "money")]
+        public decimal SubTotal { get; set; }
+
+        [Column(TypeName = "money")]
+        public decimal TaxAmt { get; set; }
+
+        [Column(TypeName = "money")]
+        public decimal Freight { get; set; }
+
+        [Column(TypeName = "money")]
+        [DatabaseGenerated(DatabaseGeneratedOption.Computed)]
+        public decimal TotalDue { get; set; }
+
+        [StringLength(128)]
+        public string Comment { get; set; }
+
+        public Guid rowguid { get; set; }
+
+        public DateTime ModifiedDate { get; set; }
+
+        [SuppressMessage("Microsoft.Usage", "CA2227:CollectionPropertiesShouldBeReadOnly")]
+        public virtual ICollection<SalesOrderDetail> SalesOrderDetails { get; set; }
+    }
+}

Src/SalesDataGenerator/DataGenerator/DataGenerators/IDataGenerator.cs

@@ -0,0 +1,21 @@
+// --------------------------------------------------------------------------------------------------------------------
+// <copyright file="IDataGenerator.cs" company="Microsoft">
+//   Copyright (c) Microsoft.  All rights reserved.
+// </copyright>
+// <summary>
+//   Data generator interface.
+// </summary>
+// --------------------------------------------------------------------------------------------------------------------
+namespace DataGenerator.DataGenerators
+{
+    /// <summary>
+    /// 
+    /// </summary>
+    interface IDataGenerator
+    {
+        /// <summary>
+        /// 
+        /// </summary>
+        void Generate();
+    }
+}

Src/SalesDataGenerator/DataGenerator/DataGenerators/ResellerSaleDataGenerator.cs

@@ -0,0 +1,219 @@
+// --------------------------------------------------------------------------------------------------------------------
+// <copyright file="ResellerSaleDataGenerator.cs" company="Microsoft">
+//   Copyright (c) Microsoft.  All rights reserved.
+// </copyright>
+// <summary>
+//   Data generator to simulate reseller sales data.
+// </summary>
+// --------------------------------------------------------------------------------------------------------------------
+namespace DataGenerator.DataGenerators
+{
+    using System;
+    using System.Linq;
+    using DAL;
+
+    /// <summary>
+    /// Data generator to simulate reseller sales data.
+    /// </summary>
+    public class ResellerSaleDataGenerator : IDataGenerator
+    {
+        private readonly int[] _addressIds = Enumerable.Range(1, 34).ToArray();
+        private readonly int[] _creditCardIds = Enumerable.Range(1, 30).ToArray();
+        private readonly int[] _currencyRateIds = Enumerable.Range(1, 30).ToArray();
+        private readonly int[] _customerIds = Enumerable.Range(1, 100).ToArray();
+        private readonly int[] _salesPersonIds = Enumerable.Range(274, 16).ToArray();
+        private readonly int[] _salesTerritoryIds = Enumerable.Range(1, 10).ToArray();
+        private readonly int[] _shipMethodIds = Enumerable.Range(1, 5).ToArray();
+
+        private readonly SpecialOfferProduct[] _specialOfferProducts =
+        {
+            new SpecialOfferProduct(Enumerable.Range(707, 100).ToArray(), 1)
+        };
+
+        private readonly string _sqlDwConnectionString;
+
+        public ResellerSaleDataGenerator(string sqlDwConnectionString)
+        {
+            _sqlDwConnectionString = sqlDwConnectionString;
+        }
+
+        /// <summary>
+        /// Generate Reseller sales order data and insert to SQL DB.
+        /// </summary>
+        public void Generate()
+        {
+            using (var db = new AdventureWorksModel(_sqlDwConnectionString))
+            {
+                var soh = salesOrderHeader();
+                var sod = salesOrderDetail();
+                soh.SalesOrderDetails.Add(sod);
+                db.SalesOrderHeaders.Add(soh);
+                db.SaveChanges();
+                Console.WriteLine(
+                    $"Inserted SalesOrderHeader Id: {soh.SalesOrderID} and SalesOrderDetail Id: {sod.SalesOrderDetailID}..");
+            }
+        }
+
+        private SalesOrderDetail salesOrderDetail()
+        {
+            var unitPrice = randomDecimalBetween(0m, 3000m);
+            var sop = specialOfferProduct();
+            var productId = RandomFromArray(sop.ProductIds);
+            var specialOfferId = sop.SpecialOfferId;
+
+            var sod = new SalesOrderDetail
+            {
+                CarrierTrackingNumber = idString(10),
+                OrderQty = (short) randomIntBetween(1, 6),
+                ProductID = productId,
+                SpecialOfferID = specialOfferId,
+                UnitPrice = unitPrice,
+                UnitPriceDiscount = 0.1m,
+                rowguid = Guid.NewGuid(),
+                ModifiedDate = DateTime.Now
+            };
+
+            return sod;
+        }
+
+        private SalesOrderHeader salesOrderHeader()
+        {
+            var subTotal = randomDecimalBetween(0.0m, 5000m);
+            var orderDate = randomDateBetween(new DateTime(2005, 1, 27), new DateTime(2012, 12, 4));
+            var soh = new SalesOrderHeader
+            {
+                Status = status(),
+                Freight = randomDecimalBetween(0.0m, 1000m),
+                SubTotal = subTotal,
+                AccountNumber = idString(15),
+                BillToAddressID = addressId(),
+                CreditCardApprovalCode = idString(15),
+                CurrencyRateID = currencyRateId(),
+                CustomerID = customerId(),
+                ShipToAddressID = addressId(),
+                SalesPersonID = salesPersonId(),
+                PurchaseOrderNumber = idString(25),
+                OnlineOrderFlag = onlineOrderFlag(),
+                ShipMethodID = shipMethodId(),
+                CreditCardID = creditCardId(),
+                TerritoryID = salesTerritoryId(),
+                TaxAmt = 0.1m*subTotal,
+                OrderDate = orderDate,
+                ShipDate = randomDateBetween(orderDate, orderDate.AddDays(20)),
+                DueDate = randomDateBetween(orderDate, orderDate.AddDays(20)),
+                rowguid = Guid.NewGuid(),
+                ModifiedDate = DateTime.Now
+            };
+
+            return soh;
+        }
+
+        private int salesTerritoryId()
+        {
+            return RandomFromArray(_salesTerritoryIds);
+        }
+
+        private int creditCardId()
+        {
+            return RandomFromArray(_creditCardIds);
+        }
+
+        private int shipMethodId()
+        {
+            return RandomFromArray(_shipMethodIds);
+        }
+
+        private bool onlineOrderFlag()
+        {
+            return new Random().NextDouble() >= 0.5;
+        }
+
+        private SpecialOfferProduct specialOfferProduct()
+        {
+            return RandomFromArray(_specialOfferProducts);
+        }
+
+        private int salesPersonId()
+        {
+            return RandomFromArray(_salesPersonIds);
+        }
+
+        private int customerId()
+        {
+            return RandomFromArray(_customerIds);
+        }
+
+        private int currencyRateId()
+        {
+            return RandomFromArray(_currencyRateIds);
+        }
+
+        private int addressId()
+        {
+            return RandomFromArray(_addressIds);
+        }
+
+        private string idString(int maxLength)
+        {
+            return Guid.NewGuid().ToString().Substring(0, maxLength);
+        }
+
+        private byte status()
+        {
+            return (byte) RandomEnum<StatusEnum>();
+        }
+
+        private T RandomEnum<T>()
+        {
+            var values = (T[]) Enum.GetValues(typeof(T));
+            return values[new Random().Next(0, values.Length)];
+        }
+
+        private T RandomFromArray<T>(T[] candidatesArray)
+        {
+            return candidatesArray[new Random().Next(0, candidatesArray.Length)];
+        }
+
+        private static decimal randomDecimalBetween(decimal minValue, decimal maxValue)
+        {
+            var next = (decimal) new Random().NextDouble();
+
+            return minValue + next*(maxValue - minValue);
+        }
+
+        private static int randomIntBetween(int minValue, int maxValue)
+        {
+            var next = new Random().NextDouble();
+
+            return minValue + (int) Math.Round(next*(maxValue - minValue));
+        }
+
+        private static DateTime randomDateBetween(DateTime start, DateTime end)
+        {
+            var range = (end - start).Days;
+            return start.AddDays(new Random().Next(range));
+        }
+
+        private enum StatusEnum
+        {
+            InProcess = 1,
+            Approved = 2,
+            BackOrdered = 3,
+            Rejected = 4,
+            Shipped = 5,
+            Canceled = 5
+        }
+
+        private class SpecialOfferProduct
+        {
+            public SpecialOfferProduct(int[] productIds, int specialOfferId)
+            {
+                ProductIds = productIds;
+                SpecialOfferId = specialOfferId;
+            }
+
+            public int[] ProductIds { get; }
+            public int SpecialOfferId { get; }
+        }
+    }
+}

Src/SalesDataGenerator/DataGenerator/Program.cs

@@ -0,0 +1,79 @@
+// --------------------------------------------------------------------------------------------------------------------
+// <copyright file="AdventureWorksModel.cs" company="Microsoft">
+//   Copyright (c) Microsoft.  All rights reserved.
+// </copyright>
+// <summary>
+//   Adventure Works SQL Database OLTP Model.
+// </summary>
+// --------------------------------------------------------------------------------------------------------------------
+
+namespace DataGenerator
+{
+    using System;
+    using System.Configuration;
+    using System.Diagnostics;
+    using System.IO;
+    using BackoffPolicies;
+    using DataGenerators;
+
+    internal class Program
+    {
+        private static readonly string SqlServer = ConfigurationManager.AppSettings["SqlDbServerName"];
+        private static readonly string SqlUser = ConfigurationManager.AppSettings["SqlUser"];
+        private static readonly string SqlPassword = ConfigurationManager.AppSettings["SqlPassword"];
+
+        private static readonly string DisableFilePath =
+            Environment.ExpandEnvironmentVariables(
+                "%WEBROOT_PATH%\\app_data\\jobs\\continuous\\%WEBJOBS_NAME%\\disable.job");
+
+        /// <summary>
+        ///     User's sql database connection string.
+        /// </summary>
+        private static readonly string SqlDwConnectionString = $"data source=etlupdatedkwow6bo27zvssrv.database.windows.net;initial catalog=AdventureWorks2012;persist security info=True;user id=username;password=Pa$$w0rd12!;MultipleActiveResultSets=True;App=EntityFramework";
+        //$"data source={SqlServer}.database.windows.net;initial catalog=AdventureWorks2012;persist security info=True;user id={SqlUser};password={SqlPassword};MultipleActiveResultSets=True;App=EntityFramework";
+
+        private static void Main(string[] args)
+        {
+            const int backoffMillisecondTimeout = 5000;
+            var dataGenerator = new ResellerSaleDataGenerator(SqlDwConnectionString);
+            var backoffPolicy = new FixedBackoffPolicy(backoffMillisecondTimeout);
+            var dataGeneratorTimeout = new TimeSpan(2, 0, 0);
+
+            if (IsJobDisabled())
+            {
+                return;
+            }
+            RepeatTillTimeout(dataGenerator.Generate, dataGeneratorTimeout, backoffPolicy);
+            DisableJob();
+        }
+
+        private static bool IsJobDisabled()
+        {
+            return File.Exists(DisableFilePath);
+        }
+
+        private static void DisableJob()
+        {
+            File.Create(DisableFilePath).Dispose();
+        }
+
+        private static void RepeatTillTimeout(VoidFunction voidFunction, TimeSpan timeout, IBackoffPolicy backoffPolicy)
+        {
+            var stopwatch = Stopwatch.StartNew();
+            while (stopwatch.Elapsed < timeout)
+            {
+                try
+                {
+                    voidFunction();
+                }
+                catch (Exception e)
+                {
+                    Trace.WriteLine(e);
+                }
+                backoffPolicy.Wait();
+            }
+        }
+
+        private delegate void VoidFunction();
+    }
+}

Src/SalesDataGenerator/DataGenerator/Properties/AssemblyInfo.cs

@@ -0,0 +1,36 @@
+using System.Reflection;
+using System.Runtime.CompilerServices;
+using System.Runtime.InteropServices;
+
+// General Information about an assembly is controlled through the following 
+// set of attributes. Change these attribute values to modify the information
+// associated with an assembly.
+[assembly: AssemblyTitle("DataGenerator")]
+[assembly: AssemblyDescription("")]
+[assembly: AssemblyConfiguration("")]
+[assembly: AssemblyCompany("")]
+[assembly: AssemblyProduct("DataGenerator")]
+[assembly: AssemblyCopyright("Copyright ©  2016")]
+[assembly: AssemblyTrademark("")]
+[assembly: AssemblyCulture("")]
+
+// Setting ComVisible to false makes the types in this assembly not visible 
+// to COM components.  If you need to access a type in this assembly from 
+// COM, set the ComVisible attribute to true on that type.
+[assembly: ComVisible(false)]
+
+// The following GUID is for the ID of the typelib if this project is exposed to COM
+[assembly: Guid("9eb22ba6-bc39-441c-817a-1f461a7c6cee")]
+
+// Version information for an assembly consists of the following four values:
+//
+//      Major Version
+//      Minor Version 
+//      Build Number
+//      Revision
+//
+// You can specify all the values or you can default the Build and Revision Numbers 
+// by using the '*' as shown below:
+// [assembly: AssemblyVersion("1.0.*")]
+[assembly: AssemblyVersion("1.0.0.0")]
+[assembly: AssemblyFileVersion("1.0.0.0")]

Src/SalesDataGenerator/DataGenerator/Properties/webjob-publish-settings.json

@@ -0,0 +1,9 @@
+{
+  "$schema": "http://schemastore.org/schemas/json/webjob-publish-settings.json",
+  "webJobName": "DataGenerator",
+  "startTime": null,
+  "endTime": null,
+  "jobRecurrenceFrequency": null,
+  "interval": null,
+  "runMode": "Continuous"
+}

Src/SalesDataGenerator/DataGenerator/packages.config

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="utf-8"?>
+<packages>
+  <package id="EntityFramework" version="6.1.3" targetFramework="net45" />
+  <package id="Hyak.Common" version="1.0.2" targetFramework="net45" />
+  <package id="Microsoft.Azure.Common" version="2.0.4" targetFramework="net45" />
+  <package id="Microsoft.Azure.Common.Dependencies" version="1.0.0" targetFramework="net45" />
+  <package id="Microsoft.Azure.Management.DataFactories" version="4.2.0" targetFramework="net45" />
+  <package id="Microsoft.Bcl" version="1.1.9" targetFramework="net45" />
+  <package id="Microsoft.Bcl.Async" version="1.0.168" targetFramework="net45" />
+  <package id="Microsoft.Bcl.Build" version="1.0.14" targetFramework="net45" />
+  <package id="Microsoft.Net.Http" version="2.2.22" targetFramework="net45" />
+  <package id="Microsoft.Web.WebJobs.Publish" version="1.0.12" targetFramework="net45" />
+  <package id="Newtonsoft.Json" version="6.0.4" targetFramework="net45" />
+</packages>