Update docs for Azure CNI subnet requirements (#2992)

docs/kubernetes/features.md

@@ -204,7 +204,7 @@ you can define stricter policies. Good resources to get information about that a
 
 *Note: Custom VNET for Kubernetes Windows cluster has a [known issue](https://github.com/Azure/acs-engine/issues/1767).*
 
-ACS Engine supports deploying into an existing VNET. Operators must specify the ARM path/id of Subnets for the `masterProfile` and  any `agentPoolProfiles`, as well as the first IP address to use for IP static IP allocation in `firstConsecutiveStaticIP`. Additionally, to prevent source address NAT'ing within the VNET, we assign to the `vnetCidr` property in `masterProfile` the CIDR block that represents the usable address space in the existing VNET.
+ACS Engine supports deploying into an existing VNET. Operators must specify the ARM path/id of Subnets for the `masterProfile` and  any `agentPoolProfiles`, as well as the first IP address to use for static IP allocation in `firstConsecutiveStaticIP`. Please note that in any azure subnet, the first four and the last ip address is reserved and can not be used. Additionally, each pod now gets the IP address from the Subnet. As a result, enough IP addresses (equal to `ipAddressCount` for each node) should be available beyond `firstConsecutiveStaticIP`. By default, the `ipAddressCount` has a value of 31, 1 for the node and 30 for pods, (note that the number of pods can be changed via `KubeletConfig["--max-pods"]`). `ipAddressCount` can be changed if desired. Furthermore, to prevent source address NAT'ing within the VNET, we assign to the `vnetCidr` property in `masterProfile` the CIDR block that represents the usable address space in the existing VNET. Therefore, it is recommended to use a large subnet size such as `/16`.
 
 Depending upon the size of the VNET address space, during deployment, it is possible to experience IP address assignment collision between the required Kubernetes static IPs (one each per master and one for the API server load balancer, if more than one masters) and Azure CNI-assigned dynamic IPs (one for each NIC on the agent nodes). In practice, the larger the VNET the less likely this is to happen; some detail, and then a guideline.
 

examples/kubernetes-config/kubernetes-private-cluster-single-master.json

@@ -9,7 +9,7 @@
           "jumpboxProfile": {
             "name": "my-jb",
             "vmSize": "Standard_D2_v2",
-            "diskSizeGB": 30,
+            "osDiskSizeGB": 30,
             "username": "azureuser",
             "publicKey": ""
           }

examples/kubernetes-config/kubernetes-private-cluster.json

@@ -9,7 +9,7 @@
           "jumpboxProfile": {
             "name": "my-jb",
             "vmSize": "Standard_D2_v2",
-            "diskSizeGB": 30,
+            "osDiskSizeGB": 30,
             "username": "azureuser",
             "publicKey": ""
           }

pkg/acsengine/defaults.go

@@ -730,10 +730,8 @@ func setAgentNetworkDefaults(a *api.Properties) {
 
 			// Allocate IP addresses for pods if VNET integration is enabled.
 			if a.OrchestratorProfile.IsAzureCNI() {
-				if a.OrchestratorProfile.OrchestratorType == api.Kubernetes {
-					agentPoolMaxPods, _ := strconv.Atoi(profile.KubernetesConfig.KubeletConfig["--max-pods"])
-					profile.IPAddressCount += agentPoolMaxPods
-				}
+				agentPoolMaxPods, _ := strconv.Atoi(profile.KubernetesConfig.KubeletConfig["--max-pods"])
+				profile.IPAddressCount += agentPoolMaxPods
 			}
 		}
 	}