v0.3 release -support for alternative telemetry ingestion endpoints. (#8)

* Support for custom telemetry endpoints.

* Documentation update.

SETUP.md

@@ -1,13 +1,9 @@
 # Application Insights for Kubernetes
-Application Insights for Kubernetes is a monitoring solution that allows you to collect Application Insights telemetry pertaining to 
-incoming and outgoing requests to and from pods running in your Kubernetes cluster without the need for instrumenting the 
-application with an SDK. We utilize service mesh technology called Istio to collect data, so the only requirement is that your 
-Kubernetes deployment is configured to run with Istio (there are few simple requirements, see below for details).
+Application Insights for Kubernetes is a monitoring solution that allows you to collect Application Insights telemetry pertaining to incoming and outgoing requests to and from pods running in your Kubernetes cluster without the need for instrumenting the application with an SDK. We utilize service mesh technology called Istio to collect data, so the only requirement is that your Kubernetes deployment is configured to run with Istio (there are few simple requirements, see below for details).
 
 *If you are not familiar with Application Insights, see [here](https://docs.microsoft.com/azure/azure-monitor/app/app-insights-overview).*
 
-Since service mesh lifts data off the wire, we can not intercept encrypted traffic. For traffic that doesn't leave the cluster, 
-use plain unencrypted protocol (e.g. HTTP). For external traffic that must be encrypted, consider setting up SSL termination at the ingress controller.
+Since service mesh lifts data off the wire, we can not intercept encrypted traffic. For traffic that doesn't leave the cluster, use plain unencrypted protocol (e.g. HTTP). For external traffic that must be encrypted, consider setting up SSL termination at the ingress controller.
 
 Applications running outside of the service mesh are not affected.
 
@@ -34,9 +30,7 @@ kubectl label namespace <my-app-namespace> istio-injection=enabled
 ```
 
 ### Deploy your application
-- Deploy your application to *my-app-namespace* namespace. If the application is already deployed, and you have followed the automatic 
-sidecar injection method described above, you need to recreate pods to ensure Istio injects its sidecar; either initiate a 
-rolling update or simply delete individual pods and wait for them to be recreated.
+- Deploy your application to *my-app-namespace* namespace. If the application is already deployed, and you have followed the automatic sidecar injection method described above, you need to recreate pods to ensure Istio injects its sidecar; either initiate a rolling update or simply delete individual pods and wait for them to be recreated.
 - Ensure your application complies with Istio requirements listed [here](https://istio.io/docs/setup/kubernetes/prepare/requirements/).
 
 ### Deploy Application Insights for Kubernetes
@@ -46,6 +40,7 @@ rolling update or simply delete individual pods and wait for them to be recreate
     - edit the value of *ISTIO_MIXER_PLUGIN_AI_INSTRUMENTATIONKEY* environment variable to contain the instrumentation key of the Application Insights resource in Azure Portal to contain the telemetry.
     - *if required*, edit the value of *ISTIO_MIXER_PLUGIN_WATCHLIST_NAMESPACES* environment variable to contain a comma-separated list of namespaces for which you would like to enable monitoring. Leave it blank to 
 monitor all namespaces.
+    - *if running in a government cloud*, or in other scenarios where the default Application Insights endpoint must be overridden, edit the values of *ISTIO_MIXER_PLUGIN_TELEMETRY_CHANNEL_ENDPOINT* and *ISTIO_MIXER_PLUGIN_QUICKPULSE_SERVICE_ENDPOINT* environment variables. See [here](https://docs.microsoft.com/en-us/azure/azure-monitor/app/custom-endpoints#regions-that-require-endpoint-modification) for detailed information about available endpoints.
 4. Apply *every* YAML file found under *src/kubernetes/* by running the following (you must still be inside */src/kubernetes/*):
    ```
    kubectl apply -f .
@@ -102,7 +97,7 @@ each containing its own namespace and instrumentation key.
 - *Are Windows containers supported?* Since only Istio is supported currently, and Istio does not offer support for Windows containers - the answer is no. As soon as supported service meshes are compatible
 with Windows containers this scenario will be enabled.
 - *Distributed tracing support* Support for distributed tracing is coming soon. Currently, only Application Map is supported, not full distributed tracing chains.
-- *Support for other protocols – Redis, mongoDB, HTTPS, etc.* Since telemetry is lifted off the wire, encrypted protocols (like HTTPS) cannot be supported. For internal traffic use plain unecnrypted protocols (like HTTP); 
+- *Support for other protocols - Redis, mongoDB, HTTPS, etc.* Since telemetry is lifted off the wire, encrypted protocols (like HTTPS) cannot be supported. For internal traffic use plain unecnrypted protocols (like HTTP); 
 for external traffic that must be encrypted consider setting up SSL termination at the ingress gateway. Support for some of the most popular non-HTTP protocols is coming.
 
 ## Uninstall

src/ConsoleHost/IstioMixerPlugin.config

@@ -27,6 +27,20 @@ This is a configuration file for Istio Mixer Plugin.
     <MaxOtherItemsPerSecond>%ISTIO_MIXER_PLUGIN_AI_ADAPTIVE_SAMPLING_LIMIT%</MaxOtherItemsPerSecond>
   </AdaptiveSampling>
 
+  <Endpoints>
+    <!--
+      Leave empty for default endpoints
+      
+      See https://docs.microsoft.com/en-us/azure/azure-monitor/app/custom-endpoints#regions-that-require-endpoint-modification for available endpoints
+    -->
+
+    <!--Endpoint for telemetry ingestion-->
+    <TelemetryChannelEndpoint>%ISTIO_MIXER_PLUGIN_TELEMETRY_CHANNEL_ENDPOINT%</TelemetryChannelEndpoint>
+    
+    <!--Endpoint for live metrics stream ingestion-->
+    <QuickPulseServiceEndpoint>%ISTIO_MIXER_PLUGIN_QUICKPULSE_SERVICE_ENDPOINT%</QuickPulseServiceEndpoint>
+  </Endpoints>
+
   <ClusterInfoListener>
     <!-- this setting is for internal purposes in starting the webserver meant to process incoming cluster related data
     Do not modify unless so instructed-->

src/IstioMixerPlugin.sln

@@ -14,6 +14,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution
 		ApplicationInsightsIstioPluginStrongNameKey.snk = ApplicationInsightsIstioPluginStrongNameKey.snk
 		AuthenticodeWhitelist.txt = AuthenticodeWhitelist.txt
 		build-docker.cmd = build-docker.cmd
+		changes.md = changes.md
 		Dockerfile = Dockerfile
 		NuGet.config = NuGet.config
 	EndProjectSection

src/Library/Configuration.cs

@@ -98,6 +98,38 @@
             }
         }
 
+        public string Endpoints_TelemetryChannelEndpoint
+        {
+            get
+            {
+                try
+                {
+                    return this.configuration.Element("Endpoints")?.Element("TelemetryChannelEndpoint")?.Value;
+                }
+                catch (Exception e)
+                {
+                    throw new ArgumentException(
+                        FormattableString.Invariant($"Could not find or convert the data field {MethodBase.GetCurrentMethod().Name} in configuration. {this.configuration.Value}"), e);
+                }
+            }
+        }
+
+        public string Endpoints_QuickPulseServiceEndpoint
+        {
+            get
+            {
+                try
+                {
+                    return this.configuration.Element("Endpoints")?.Element("QuickPulseServiceEndpoint")?.Value;
+                }
+                catch (Exception e)
+                {
+                    throw new ArgumentException(
+                        FormattableString.Invariant($"Could not find or convert the data field {MethodBase.GetCurrentMethod().Name} in configuration. {this.configuration.Value}"), e);
+                }
+            }
+        }
+
         public string[] Watchlist_Namespaces
         {
             get

src/Library/Library.cs

@@ -22,9 +22,6 @@
 
         private readonly TelemetryGenerator telemetryGenerator;
 
-        private readonly string instrumentationKey;
-        private readonly string liveMetricsStreamAuthenticationApiKey;
-        
         private readonly TimeSpan statsTracingTimeout = TimeSpan.FromMinutes(1);
         
         /// <summary>
@@ -42,9 +39,6 @@
         {
             this.config = new Configuration(configuration);
 
-            this.instrumentationKey = this.config.InstrumentationKey;
-            this.liveMetricsStreamAuthenticationApiKey = this.config.LiveMetricsStreamAuthenticationApiKey;
-            
             Diagnostics.LogInfo(
                 FormattableString.Invariant($"Loaded configuration. {Environment.NewLine}{configuration}"));
 
@@ -53,9 +47,14 @@
             try
             {
                 var activeConfiguration = TelemetryConfiguration.Active;
-                activeConfiguration.InstrumentationKey = this.instrumentationKey;
+                activeConfiguration.InstrumentationKey = this.config.InstrumentationKey;
 
                 var channel = new ServerTelemetryChannel();
+                string telemetryChannelEndpoint = this.config.Endpoints_TelemetryChannelEndpoint;
+                if(!string.IsNullOrWhiteSpace(telemetryChannelEndpoint))
+                {
+                    channel.EndpointAddress = telemetryChannelEndpoint;
+                }
                 channel.Initialize(activeConfiguration);
                 activeConfiguration.TelemetryChannel = channel;
 
@@ -76,7 +75,13 @@
 
                 builder.Build();
 
-                var quickPulseModule = new QuickPulseTelemetryModule() { AuthenticationApiKey = this.liveMetricsStreamAuthenticationApiKey };
+                var quickPulseModule = new QuickPulseTelemetryModule() {AuthenticationApiKey = this.config.LiveMetricsStreamAuthenticationApiKey};
+                string quickPulseServiceEndpoint = this.config.Endpoints_QuickPulseServiceEndpoint;
+                if (!string.IsNullOrWhiteSpace(quickPulseServiceEndpoint))
+                {
+                    quickPulseModule.QuickPulseServiceEndpoint = quickPulseServiceEndpoint;
+                }
+
                 quickPulseModule.Initialize(activeConfiguration);
                 quickPulseModule.RegisterTelemetryProcessor(processor);
 

src/LibraryTest/Library/ConfigurationTests.cs

@@ -30,6 +30,8 @@ namespace Microsoft.IstioMixerPlugin.LibraryTest.Library
             Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_AI_ADAPTIVE_SAMPLING_LIMIT", "25", EnvironmentVariableTarget.Process);
             Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_WATCHLIST_NAMESPACES", null, EnvironmentVariableTarget.Process);
             Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_WATCHLIST_NAMESPACES_IGNORED", null, EnvironmentVariableTarget.Process);
+            Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_TELEMETRY_CHANNEL_ENDPOINT", null, EnvironmentVariableTarget.Process);
+            Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_QUICKPULSE_SERVICE_ENDPOINT", null, EnvironmentVariableTarget.Process);
 
             // ACT
             var config = new Configuration(defaultConfig);
@@ -42,6 +44,8 @@ namespace Microsoft.IstioMixerPlugin.LibraryTest.Library
             Assert.AreEqual("%ISTIO_MIXER_PLUGIN_AI_LIVE_METRICS_STREAM_AUTH_KEY%", config.LiveMetricsStreamAuthenticationApiKey);
             Assert.AreEqual("%ISTIO_MIXER_PLUGIN_WATCHLIST_NAMESPACES%", config.Watchlist_Namespaces.Single());
             Assert.AreEqual("%ISTIO_MIXER_PLUGIN_WATCHLIST_NAMESPACES_IGNORED%", config.Watchlist_IgnoredNamespaces.Single());
+            Assert.AreEqual("%ISTIO_MIXER_PLUGIN_TELEMETRY_CHANNEL_ENDPOINT%", config.Endpoints_TelemetryChannelEndpoint);
+            Assert.AreEqual("%ISTIO_MIXER_PLUGIN_QUICKPULSE_SERVICE_ENDPOINT%", config.Endpoints_QuickPulseServiceEndpoint);
 
             Assert.AreEqual(true, config.AdaptiveSampling_Enabled);
             Assert.AreEqual(10, config.AdaptiveSampling_MaxEventsPerSecond);
@@ -51,6 +55,8 @@ namespace Microsoft.IstioMixerPlugin.LibraryTest.Library
             Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_AI_ADAPTIVE_SAMPLING_LIMIT", null, EnvironmentVariableTarget.Process);
             Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_WATCHLIST_NAMESPACES", null, EnvironmentVariableTarget.Process);
             Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_WATCHLIST_NAMESPACES_IGNORED", null, EnvironmentVariableTarget.Process);
+            Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_TELEMETRY_CHANNEL_ENDPOINT", null, EnvironmentVariableTarget.Process);
+            Environment.SetEnvironmentVariable("ISTIO_MIXER_PLUGIN_QUICKPULSE_SERVICE_ENDPOINT", null, EnvironmentVariableTarget.Process);
         }
 
         [TestMethod]
@@ -68,6 +74,10 @@ namespace Microsoft.IstioMixerPlugin.LibraryTest.Library
       <!--Telemetry items other than events are counted together-->
       <MaxOtherItemsPerSecond>%ISTIO_MIXER_PLUGIN_AI_ADAPTIVE_SAMPLING_LIMIT%</MaxOtherItemsPerSecond>
     </AdaptiveSampling>
+    <Endpoints>
+        <TelemetryChannelEndpoint>%TelemetryChannelEndpoint%</TelemetryChannelEndpoint>
+        <QuickPulseServiceEndpoint>%QuickPulseEndpoint%</QuickPulseServiceEndpoint>
+    </Endpoints>
 </Configuration>
 ";
 
@@ -75,11 +85,15 @@ namespace Microsoft.IstioMixerPlugin.LibraryTest.Library
             string host = Guid.NewGuid().ToString();
             string port = rand.Next().ToString();
             string ikey = Guid.NewGuid().ToString();
-            
+            string telemetryChannelEndpoint = Guid.NewGuid().ToString();
+            string quickPulseEndpoint = Guid.NewGuid().ToString();
+
             Environment.SetEnvironmentVariable("Input_Host", host);
             Environment.SetEnvironmentVariable("Input_Port", port);
             Environment.SetEnvironmentVariable("ConfigTestInstrumentationKey", ikey);
-            
+            Environment.SetEnvironmentVariable("TelemetryChannelEndpoint", telemetryChannelEndpoint);
+            Environment.SetEnvironmentVariable("QuickPulseEndpoint", quickPulseEndpoint);
+
             // ACT
             var configuration = new Configuration(config);
 
@@ -87,6 +101,8 @@ namespace Microsoft.IstioMixerPlugin.LibraryTest.Library
             Assert.AreEqual(host, configuration.Host);
             Assert.AreEqual(port, configuration.Port.ToString());
             Assert.AreEqual(ikey, configuration.InstrumentationKey);
+            Assert.AreEqual(telemetryChannelEndpoint, configuration.Endpoints_TelemetryChannelEndpoint);
+            Assert.AreEqual(quickPulseEndpoint, configuration.Endpoints_QuickPulseServiceEndpoint);
         }
 
         [TestMethod]

src/LibraryTest/Library/LibraryTests.cs

@@ -36,6 +36,9 @@ namespace Microsoft.IstioMixerPlugin.LibraryTest.Library
       <!--Telemetry items other than events are counted together-->
       <MaxOtherItemsPerSecond>%ISTIO_MIXER_PLUGIN_AI_ADAPTIVE_SAMPLING_LIMIT%</MaxOtherItemsPerSecond>
     </AdaptiveSampling>
+    <Endpoints>
+      <TelemetryChannelEndpoint></TelemetryChannelEndpoint>
+    </Endpoints>
 </Configuration>
 ";
             var lib = new Library(config);

src/changes.md

@@ -1,8 +1,9 @@
 # Change log
 Image location: *mcr.microsoft.com/applicationinsights/istiomixeradapter:ver*
 
-## vNext
-- support for *version* label
+## v0.3
+- support for *version* label, which is now included in reported telemetry
+- support for redirection to a different telemetry ingestion endpoint
 
 ## v0.2
 Public preview

src/kubernetes/application-insights-istio-mixer-adapter-deployment.yaml

@@ -35,6 +35,10 @@ spec:
           value: Info # Log level: Info is the default, Trace for detailed tracing, Error for errors only
         - name: ISTIO_MIXER_PLUGIN_AI_ADAPTIVE_SAMPLING_LIMIT
           value: "1000"
+        - name: ISTIO_MIXER_PLUGIN_TELEMETRY_CHANNEL_ENDPOINT
+          value: "" # telemetry ingestion endpoint. See https://docs.microsoft.com/en-us/azure/azure-monitor/app/custom-endpoints#regions-that-require-endpoint-modification for available options
+        - name: ISTIO_MIXER_PLUGIN_QUICKPULSE_SERVICE_ENDPOINT
+          value: "" # live metrics stream (QuickPulse) endpoint. See https://docs.microsoft.com/en-us/azure/azure-monitor/app/custom-endpoints#regions-that-require-endpoint-modification for available options
 ---
 apiVersion: v1
 kind: Service