Merge pull request #43 from Azure/fix-learn-links
Fix links to Microsoft Learn + Minor Formatting Updates
This commit is contained in:
Ayobami Ayodeji
GitHub
Коммит
94f0521613
|
|
@ -1,7 +1,6 @@
|
|||
---
|
||||
name: General feedback
|
||||
about: For positive or negative feedback on Microsoft docs or Reference Implementation,
|
||||
success stories, etc
|
||||
about: For positive or negative feedback on Microsoft Learn or Reference Implementation, success stories, etc
|
||||
title: ''
|
||||
labels: feedback
|
||||
assignees: mosabami
|
||||
|
|
|
|||
|
|
@ -2,19 +2,19 @@
|
|||
|
||||
Azure Landing Zone Accelerators are architectural guidance, reference architecture, reference implementations and automation packaged to deploy workload platforms on Azure at Scale and aligned with industry proven practices.
|
||||
|
||||
AKS Landing Zone Accelerator represents the strategic design path and target technical state for an Azure Kubernetes Service (AKS) deployment. This solution provides an architectural approach and reference implementation to prepare landing zone subscriptions for a scalable Azure Kubernetes Service (AKS) cluster. For the architectural guidance, check out [AKS Landing Zone Accelerator](https://docs.microsoft.com/azure/cloud-adoption-framework/scenarios/aks/enterprise-scale-landing-zone) in Microsoft Docs.
|
||||
AKS Landing Zone Accelerator represents the strategic design path and target technical state for an Azure Kubernetes Service (AKS) deployment. This solution provides an architectural approach and reference implementation to prepare landing zone subscriptions for a scalable Azure Kubernetes Service (AKS) cluster. For the architectural guidance, check out [AKS landing zone accelerator](https://learn.microsoft.com/azure/cloud-adoption-framework/scenarios/app-platform/aks/landing-zone-accelerator) in Microsoft Learn.
|
||||
|
||||
Below is a picture of what a golden state looks like and open source software like flux and traefik integrate well within the AKS ecosystem.
|
||||
|
||||
|
||||
|
||||
The AKS Landing Zone Accelerator is only concerned with what gets deployed in the landing zone subscription highlighted by the red box in the picture above. It is assumed that an appropriate platform foundation is already setup which may or may not be the [official ESLZ](https://docs.microsoft.com/azure/cloud-adoption-framework/ready/enterprise-scale/architecture) platform foundation. This means that policies and governance should already be in place or should be setup after this implementation and are not a part of the scope this reference implementaion. The policies applied to management groups in the hierarchy above the subscription will trickle down to the AKS Landing Zone Accelerator landing zone subscription.
|
||||
The AKS Landing Zone Accelerator is only concerned with what gets deployed in the landing zone subscription highlighted by the red box in the picture above. It is assumed that an appropriate platform foundation is already setup which may or may not be the [official ESLZ](https://learn.microsoft.com/azure/cloud-adoption-framework/ready/landing-zone/) platform foundation. This means that policies and governance should already be in place or should be setup after this implementation and are not a part of the scope this reference implementaion. The policies applied to management groups in the hierarchy above the subscription will trickle down to the AKS Landing Zone Accelerator landing zone subscription.
|
||||
|
||||
---
|
||||
|
||||
## Choosing a Deployment Model
|
||||
|
||||
The reference implementations are spread across three repos that all build on top of the [AKS Secure Baseline](https://docs.microsoft.com/azure/architecture/reference-architectures/containers/aks/secure-baseline-aks) and Azure Landing Zones.
|
||||
The reference implementations are spread across three repos that all build on top of the [AKS baseline reference architecture](https://learn.microsoft.com/azure/architecture/reference-architectures/containers/aks/baseline-aks) and Azure Landing Zones.
|
||||
|
||||
1. This one
|
||||
1. The [AKS Construction Helper](https://github.com/Azure/Aks-Construction)
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@
|
|||
|
||||
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
|
||||
|
||||
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](<https://docs.microsoft.com/en-us/previous-versions/tn-archive/cc751383(v=technet.10)>), please report it to us as described below.
|
||||
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](<https://learn.microsoft.com/previous-versions/tn-archive/cc751383(v=technet.10)>), please report it to us as described below.
|
||||
|
||||
## Reporting Security Issues
|
||||
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ To easily modify manifest files, you will connect to the control plane using Rem
|
|||
Prerequisites
|
||||
To get started, you need to have done the following steps:
|
||||
|
||||
1. Install an OpenSSH compatible SSH client (PuTTY is not supported). https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse
|
||||
1. Install an OpenSSH compatible SSH client (PuTTY is not supported). <https://learn.microsoft.com/windows-server/administration/openssh/openssh_install_firstuse>
|
||||
2. Install Visual Studio Code.
|
||||
|
||||
When the Remote-SSH vs code extension is installed you should see the following icon in the lower left screen of your vs code windows
|
||||
|
|
|
|||
|
|
@ -386,7 +386,7 @@
|
|||
},
|
||||
{
|
||||
"name": "ubuntu-security-patches",
|
||||
"description": "This address lets the Linux cluster nodes download the required security patches and updates per https://docs.microsoft.com/azure/aks/limit-egress-traffic#optional-recommended-fqdn--application-rules-for-aks-clusters.",
|
||||
"description": "This address lets the Linux cluster nodes download the required security patches and updates per https://learn.microsoft.com/azure/aks/limit-egress-traffic#optional-recommended-fqdn--application-rules-for-aks-clusters.",
|
||||
"sourceIpGroups": [
|
||||
"[resourceId('Microsoft.Network/ipGroups', variables('aksIpGroupName'))]"
|
||||
],
|
||||
|
|
@ -404,7 +404,7 @@
|
|||
},
|
||||
{
|
||||
"name": "azure-monitor",
|
||||
"description": "All required for Azure Monitor for containers per https://docs.microsoft.com/azure/aks/limit-egress-traffic#azure-monitor-for-containers",
|
||||
"description": "All required for Azure Monitor for containers per https://learn.microsoft.com/azure/aks/limit-egress-traffic#azure-monitor-for-containers",
|
||||
"sourceIpGroups": [
|
||||
"[resourceId('Microsoft.Network/ipGroups', variables('aksIpGroupName'))]"
|
||||
],
|
||||
|
|
@ -424,7 +424,7 @@
|
|||
},
|
||||
{
|
||||
"name": "azure-policy",
|
||||
"description": "All required for Azure Policy per https://docs.microsoft.com/azure/aks/limit-egress-traffic#azure-policy",
|
||||
"description": "All required for Azure Policy per https://learn.microsoft.com/azure/aks/limit-egress-traffic#azure-policy",
|
||||
"sourceIpGroups": [
|
||||
"[resourceId('Microsoft.Network/ipGroups', variables('aksIpGroupName'))]"
|
||||
],
|
||||
|
|
|
|||
|
|
@ -1,30 +1,33 @@
|
|||
# Deploying the Workload
|
||||
A suggested example workload for the cluster is detailed in this MS Learning Workshop https://docs.microsoft.com/en-us/learn/modules/aks-workshop/.
|
||||
|
||||
To deploy this workload, you will need to be able to access the Azure Container Registry that was deployed as part of the supporting infrastructure for AKS. The container registry was configured to only be accessible from a build agent on the private network.
|
||||
|
||||
If you use the Dev Server for this, the following tools must be installed:
|
||||
|
||||
1. Azure CLI
|
||||
|
||||
```bash
|
||||
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
|
||||
```
|
||||
|
||||
2. Docker CLI
|
||||
|
||||
```bash
|
||||
apt install docker.io
|
||||
```
|
||||
|
||||
You will need to clone the following repos:
|
||||
|
||||
1. The public repo for the Fruit Smoothie API.
|
||||
1. The public repo for the Fruit Smoothie API.
|
||||
|
||||
```bash
|
||||
git clone https://github.com/MicrosoftDocs/mslearn-aks-workshop-ratings-api.git
|
||||
```
|
||||
|
||||
2. The public repo for the Fruit Smootie Web Frontend:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/MicrosoftDocs/mslearn-aks-workshop-ratings-web.git
|
||||
```
|
||||
|
||||
3. This repo, for the application code - /Enterprise-Scale-for-AKS/Scenarios/Secure-Baseline/Apps/RatingsApp
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -16,5 +16,5 @@ if($isInstalled){
|
|||
New-AzADGroup -DisplayName $aksops -MailNickname $aksops
|
||||
}
|
||||
else {
|
||||
Write-Output "Azuer Powershell not installed. Installation steps in: https://docs.microsoft.com/en-us/powershell/azure/install-az-ps"
|
||||
Write-Output "Azure PowerShell not installed. Installation steps in: https://learn.microsoft.com/powershell/azure/install-az-ps"
|
||||
}
|
||||
|
|
|
|||
|
|
@ -4,15 +4,15 @@ This is the starting point for the instructions on deploying the [AKS Baseline p
|
|||
|
||||
## Steps
|
||||
|
||||
1. Latest [Azure CLI installed](https://docs.microsoft.com/cli/azure/install-azure-cli?view=azure-cli-latest) (must be at least 2.37), or you can perform this from Azure Cloud Shell by clicking below.
|
||||
1. Latest [Azure CLI installed](https://learn.microsoft.com/cli/azure/install-azure-cli?view=azure-cli-latest) (must be at least 2.37), or you can perform this from Azure Cloud Shell by clicking below.
|
||||
1. An Azure subscription.
|
||||
|
||||
The subscription used in this deployment cannot be a [free account](https://azure.microsoft.com/free); it must be a standard EA, pay-as-you-go, or Visual Studio benefit subscription. This is because the resources deployed here are beyond the quotas of free subscriptions.
|
||||
|
||||
> :warning: The user or service principal initiating the deployment process _must_ have the following minimal set of Azure Role-Based Access Control (RBAC) roles:
|
||||
>
|
||||
> * [Contributor role](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#contributor) is _required_ at the subscription level to have the ability to create resource groups and perform deployments.
|
||||
> * [User Access Administrator role](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator) is _required_ at the subscription level since you'll be performing role assignments to managed identities across various resource groups.
|
||||
> * [Contributor role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#contributor) is _required_ at the subscription level to have the ability to create resource groups and perform deployments.
|
||||
> * [User Access Administrator role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator) is _required_ at the subscription level since you'll be performing role assignments to managed identities across various resource groups.
|
||||
|
||||
1. **This step only applies if you are creating a new AAD group for this deployment. If you have one already existing and you are a part of it, you can skip this prerequisite, and the remaining steps in this page, move on to the next page by clicking on the link at the bottom**.
|
||||
|
||||
|
|
@ -20,8 +20,8 @@ This is the starting point for the instructions on deploying the [AKS Baseline p
|
|||
|
||||
> :warning: The user or service principal initiating the deployment process _must_ have the following minimal set of Azure AD permissions assigned:
|
||||
>
|
||||
> * Azure AD [User Administrator](https://docs.microsoft.com/azure/active-directory/users-groups-roles/directory-assign-admin-roles#user-administrator-permissions) is _required_ to create a "break glass" AKS admin Active Directory Security Group and User. Alternatively, you could get your Azure AD admin to create this for you when instructed to do so.
|
||||
> * If you are not part of the User Administrator group in the tenant associated to your Azure subscription, please consider [creating a new tenant](https://docs.microsoft.com/azure/active-directory/fundamentals/active-directory-access-create-new-tenant#create-a-new-tenant-for-your-organization) to use while evaluating this implementation. The Azure AD tenant backing your cluster's API RBAC does NOT need to be the same tenant associated with your Azure subscription.
|
||||
> * Azure AD [User Administrator](https://learn.microsoft.com/azure/active-directory/users-groups-roles/directory-assign-admin-roles#user-administrator-permissions) is _required_ to create a "break glass" AKS admin Active Directory Security Group and User. Alternatively, you could get your Azure AD admin to create this for you when instructed to do so.
|
||||
> * If you are not part of the User Administrator group in the tenant associated to your Azure subscription, please consider [creating a new tenant](https://learn.microsoft.com/azure/active-directory/fundamentals/active-directory-access-create-new-tenant#create-a-new-tenant-for-your-organization) to use while evaluating this implementation. The Azure AD tenant backing your cluster's API RBAC does NOT need to be the same tenant associated with your Azure subscription.
|
||||
|
||||
# Create Azure Active Directory Groups for AKS
|
||||
|
||||
|
|
@ -48,9 +48,11 @@ az ad group create --display-name $aksops --mail-nickname $aksops
|
|||
```
|
||||
|
||||
# [PowerShell](#tab/PowerShell)
|
||||
Running the command to create the new AAD groups requires the New-AzADGroup cmdlet. More details can be found [here](https://docs.microsoft.com/en-us/powershell/azure/install-az-ps?view=azps-7.0.0).
|
||||
|
||||
Running the command to create the new AAD groups requires the New-AzADGroup cmdlet. More details can be found [here](https://learn.microsoft.com/powershell/azure/install-az-ps).
|
||||
|
||||
Install New-AzADGroup cmdlet
|
||||
|
||||
```azurepowershell
|
||||
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
|
||||
Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force
|
||||
|
|
|
|||
|
|
@ -25,13 +25,13 @@ if not enter the command below to enable it
|
|||
az feature register --namespace "Microsoft.ContainerService" --name "AKS-AzureKeyVaultSecretsProvider"
|
||||
```
|
||||
|
||||
It takes a few minutes for the status to show *Registered*. Verify the registration status by using the [az feature list](https://docs.microsoft.com/en-us/cli/azure/feature#az_feature_list) command:
|
||||
It takes a few minutes for the status to show *Registered*. Verify the registration status by using the [az feature list](https://learn.microsoft.com/cli/azure/feature#az_feature_list) command:
|
||||
|
||||
```bash
|
||||
az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-AzureKeyVaultSecretsProvider')].{Name:name,State:properties.state}"
|
||||
```
|
||||
|
||||
When ready, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [az provider register](https://docs.microsoft.com/en-us/cli/azure/provider#az_provider_register) command:
|
||||
When ready, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [az provider register](https://learn.microsoft.com/cli/azure/provider#az_provider_register) command:
|
||||
|
||||
```bash
|
||||
az provider register --namespace Microsoft.ContainerService
|
||||
|
|
@ -76,7 +76,7 @@ Review "**parameters-main.json**" file and update the values as required. Please
|
|||
> * Admin group which will grant the role "Azure Kubernetes Service Cluster Admin Role". The parameter name is: *aksadminaccessprincipalId*.
|
||||
> * Dev/User group which will grant "Azure Kubernetes Service Cluster User Role". The parameter name is: *aksadminaccessprincipalId*.
|
||||
|
||||
The Kubernetes community releases minor versions roughly every three months. AKS has it own supportability policy based in the community releases. Before proceeding with the deployment, check the latest version reviewing the [supportability doc](https://docs.microsoft.com/en-us/azure/aks/supported-kubernetes-versions). You can also check the latest version by using the following command:
|
||||
The Kubernetes community releases minor versions roughly every three months. AKS has it own supportability policy based in the community releases. Before proceeding with the deployment, check the latest version reviewing the [supportability doc](https://learn.microsoft.com/azure/aks/supported-kubernetes-versions). You can also check the latest version by using the following command:
|
||||
|
||||
```azurecli
|
||||
az aks get-versions -l <region>
|
||||
|
|
@ -128,8 +128,8 @@ For the purpose of this deployment when used with kubenet a UDR will be created
|
|||
|
||||
It's also possible to use an Azure external solution to watch the scaling operations and auto-update the routes using Azure Automation, Azure Functions or Logic Apps.
|
||||
|
||||
[Use kubenet networking with your own IP address ranges in Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/configure-kubenet)
|
||||
[Application Gateway infrastructure configuration](https://docs.microsoft.com/en-us/azure/application-gateway/configuration-infrastructure#supported-user-defined-routes)
|
||||
[Use kubenet networking with your own IP address ranges in Azure Kubernetes Service (AKS)](https://learn.microsoft.com/azure/aks/configure-kubenet)
|
||||
[Application Gateway infrastructure configuration](https://learn.microsoft.com/azure/application-gateway/configuration-infrastructure#supported-user-defined-routes)
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Deploy a Basic Workload using the Fruit Smoothie Ratings Application
|
||||
|
||||
This application is provided by Microsoft Learning and is used as part of a self-paced Kubernetes training [workshop](https://docs.microsoft.com/en-us/learn/modules/aks-workshop/). You may find reviewing that workshop helpful as it presents some alternative deployment options and features using different architecture requirements. The application consists of a web frontend, an API service and a MongoDB database.
|
||||
This application consists of a web frontend, an API service and a MongoDB database.
|
||||
|
||||
Because the infrastructure has been deployed in a private AKS cluster setup with private endpoints for the container registry and other components, you will need to perform the application container build and the publishing to the Container Registry from the Dev Jumpbox in the Hub VNET, connecting via the Bastion Host service. If your computer is connected to the hub network, you may be able to just use that as well. The rest of the steps can be performed on your local machine by using AKS Run commands which allow access into private clusters using RBAC. This will help with improving security and will provide a more user-friendly way of editing YAML files.
|
||||
|
||||
|
|
|
|||
|
|
@ -4,24 +4,26 @@ Congratulations on completing the previous labs. If everything went to plan, you
|
|||
|
||||
If you would like to use a different ingress controller instead of Application Gateway, for example - NGINX - you can complete these optional extra steps to deploy NGINX as your ingress into the cluster.
|
||||
|
||||
This will be using some steps from the quickstart guide that can be found here - https://docs.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli
|
||||
This will be using some steps from the quickstart guide that can be found here - <https://learn.microsoft.com/azure/aks/ingress-basic?tabs=azure-cli>
|
||||
|
||||
Before we get started, it is worth removing the existing ingress resource that is attached to your web service and is configured with Application Gateway.
|
||||
|
||||
1. If you used HTTPS
|
||||
|
||||
```bash
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "kubectl delete ingress ratings-web-https -n ratingsapp"
|
||||
```
|
||||
```bash
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "kubectl delete ingress ratings-web-https -n ratingsapp"
|
||||
```
|
||||
|
||||
2. If you used HTTP
|
||||
```bash
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "kubectl delete ingress ratings-web -n ratingsapp"
|
||||
```
|
||||
|
||||
```bash
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "kubectl delete ingress ratings-web -n ratingsapp"
|
||||
```
|
||||
|
||||
|
||||
## Deploy the Enterprise Scale AKS (ES AKS) Policy Initiative
|
||||
|
||||
The first step is to apply the ES AKS Policy Initative to the resource group that AKS is hosted in. By doing this, it allows us to set extra security controls on the cluster, to improve security and avoid any public IP addresses for the application/cluster. If you would like to learn more about Azure Policies in detail, you can visit this link - https://docs.microsoft.com/en-us/azure/governance/policy/overview
|
||||
The first step is to apply the ES AKS Policy Initative to the resource group that AKS is hosted in. By doing this, it allows us to set extra security controls on the cluster, to improve security and avoid any public IP addresses for the application/cluster. If you would like to learn more about Azure Policies in detail, you can visit this link - <https://learn.microsoft.com/azure/governance/policy/overview>
|
||||
|
||||
The policy initiative we will be deploying here is comprised of various different policy definitions that set security controls on our cluster. Some controls are enforced with a 'Deny' effect, meaning that you will not be able to create something if it's against the policy. Some controls are implemented with an 'audit' effect, meaning that you won't be stopped from doing anything, however an audit trail of the changes you're making, will be logged and can be viewed in the compliance part of Azure Policy.
|
||||
|
||||
|
|
@ -38,9 +40,11 @@ First of all, deploy the ES AKS Policy Initative:
|
|||
The next step is to assign the policy initiative to the resource group that contains the AKS cluster:
|
||||
|
||||
1. Edit the policy.bicep file and add your Subscription Id where required then deploy the initiative
|
||||
|
||||
```bash
|
||||
code policy.bicep
|
||||
```
|
||||
|
||||
```bash
|
||||
az deployment group create -g <AKSResourceGroup> -f policy.bicep
|
||||
```
|
||||
|
|
@ -49,10 +53,10 @@ Now that the policy is assigned, you can move onto the next step which is deploy
|
|||
|
||||
## Deploy the NGINX Ingress Controller
|
||||
|
||||
We will now proceed to deploy the NGINX ingress controller using the basic configuration outlined in the quickstart guide - https://docs.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli
|
||||
We will now proceed to deploy the NGINX ingress controller using the basic configuration outlined in the quickstart guide - https://learn.microsoft.com/azure/aks/ingress-basic?tabs=azure-cli
|
||||
|
||||
```bash
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx && helm repo update && helm install ingress-nginx ingress-nginx/ingress-nginx --set controller.service.annotations.'service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path=/healthz'"
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx && helm repo update && helm install ingress-nginx ingress-nginx/ingress-nginx --set controller.service.annotations.'service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path=/healthz'"
|
||||
```
|
||||
|
||||
You will see that this installation of the NGINX ingress controller has failed. This is because Azure Policy is blocking any public load balancers from being created. You should have recieved the error as shown in the image below:
|
||||
|
|
@ -66,7 +70,7 @@ To get around this issue, we will need to deploy the ingress controller as an in
|
|||
3. Run the following command, this time passing in your yaml configuration file stating that the NGINX Ingress resource should be internal.
|
||||
|
||||
```bash
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx && helm repo update && helm install ingress-nginx ingress-nginx/ingress-nginx --set controller.service.annotations.'service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path=/healthz' -f internal-ingress.yaml" --file internal-ingress.yaml
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx && helm repo update && helm install ingress-nginx ingress-nginx/ingress-nginx --set controller.service.annotations.'service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path=/healthz' -f internal-ingress.yaml" --file internal-ingress.yaml
|
||||
```
|
||||
|
||||
Now you have an NGINX ingress controller inside of your network with an internal IP address. The last step is to configure the ingress resource to instead use NGINX as the ingress controller.
|
||||
|
|
@ -74,13 +78,15 @@ Now you have an NGINX ingress controller inside of your network with an internal
|
|||
1. Deploy the 5b-http-ratings-web-ingress.yaml file. This will deploy a new ingress resource using NGINX as the Ingress Controller.
|
||||
|
||||
```bash
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "kubectl apply -f 5b-http-ratings-web-ingress.yaml -n ratingsapp" --file 5b-http-ratings-web-ingress.yaml
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "kubectl apply -f 5b-http-ratings-web-ingress.yaml -n ratingsapp" --file 5b-http-ratings-web-ingress.yaml
|
||||
```
|
||||
|
||||
Congratulations, you should now have deployed an internal ingress controller using NGINX and mapped it to the services previously created, in order to access your application. If you run a kubectl get ingress -n ratingsapp command, you should be able to retrieve the internal IP address used by the ingress controller to access your application.
|
||||
|
||||
```bash
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "kubectl get ingress --all-namespaces"
|
||||
az aks command invoke --resource-group $ClusterRGName --name $ClusterName --command "kubectl get ingress --all-namespaces"
|
||||
```
|
||||
|
||||
.
|
||||
You can also see the internal load balancer deployed in your AKS infrastructure resource group in Azure Portal.
|
||||
.
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ The code here is purposely written to avoid loops, complex variables and logic.
|
|||
|
||||
This section is organized using folders that match the steps outlined below. Make any necessary adjustments to the variables and settings within that folder to match the needs of your deployment.
|
||||
|
||||
1. Preqs - Clone this repo, install [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli), install [Bicep tools](https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/install)
|
||||
1. Preqs - Clone this repo, install [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli), install [Bicep tools](https://learn.microsoft.com/azure/azure-resource-manager/bicep/install)
|
||||
2. [Create or Import Azure Active Directory Groups for AKS Cluster Admins and AKS Cluster Users](./02-aad.md)
|
||||
3. [Creation of Hub Network & its respective Components](./03-network-hub.md)
|
||||
4. [Creation of Spoke Network & its respective Components](./04-network-lz.md)
|
||||
|
|
|
|||
|
|
@ -2,11 +2,11 @@
|
|||
|
||||
A deployment of AKS-hosted workloads typically requires a separation of duties and lifecycle management in different areas, such as prerequisites, the host network, the cluster infrastructure, the shared services and finally the workload itself. This reference implementation is no different. Also, be aware that our primary purpose is to illustrate the topology and decisions involved in the deployment of an AKS cluster. We feel a "step-by-step" flow will help you learn the pieces of the solution and will give you insight into the relationship between them. Ultimately, lifecycle/SDLC management of your cluster and its dependencies will depend on your situation (organizational structures, standards, processes and tools), and will be implemented as appropriate for your needs.
|
||||
|
||||
There are various ways to secure your AKS cluster. From a network security perspective, these can be classified into securing the control plane and securing the workload. When it comes to securing the controle plane, one of the best ways to do that is by using a private cluster, where the control plane or API server has internal IP addresses that are defined in the [RFC1918 - Address Allocation for Private Internet](https://datatracker.ietf.org/doc/html/rfc1918) document. By using a private cluster, you can ensure network traffic between your API server and your node pools remains on the private network only. For more details about private clusters, check out the [documentation](https://docs.microsoft.com/azure/aks/private-clusters).
|
||||
There are various ways to secure your AKS cluster. From a network security perspective, these can be classified into securing the control plane and securing the workload. When it comes to securing the controle plane, one of the best ways to do that is by using a private cluster, where the control plane or API server has internal IP addresses that are defined in the [RFC1918 - Address Allocation for Private Internet](https://datatracker.ietf.org/doc/html/rfc1918) document. By using a private cluster, you can ensure network traffic between your API server and your node pools remains on the private network only. For more details about private clusters, check out the [documentation](https://learn.microsoft.com/azure/aks/private-clusters).
|
||||
|
||||
When using a private cluster, the control plane can only be accessed from computers in the private network or peered networks. For this reason, in this reference implementation, we will be deploying a virtual machine in the Hub network through which we can connect to the control plane.
|
||||
|
||||
By the end of this, you would have deployed a secure AKS cluster, complient with Enterprise-Scale for AKS guidance and best practices. We will also be deploying a workload known as the Ratings app that is also featured in the [Azure Kubernetes Services Workshop](https://docs.microsoft.com/en-us/learn/modules/aks-workshop/). Check out the workshop for some intermediate level training on AKS.
|
||||
By the end of this, you would have deployed a secure AKS cluster, complient with Enterprise-Scale for AKS guidance and best practices. We will also be deploying a workload known as the Ratings app. Check out the [Introduction to Kubernetes on Azure](https://learn.microsoft.com/training/paths/intro-to-kubernetes-on-azure/) Training path on Microsoft Learn for some intermediate level training on AKS.
|
||||
|
||||
For this scenario, we have various IaC technology that you can choose from depending on your preference. At this time only the Terraform and Bicep versions are available. Below is an architectural diagram of this scenario.
|
||||
|
||||
|
|
@ -27,9 +27,9 @@ For this scenario, we have various IaC technology that you can choose from depen
|
|||
* Azure firewall
|
||||
* MongoDB
|
||||
* Helm
|
||||
* [Secret store CSI driver](https://docs.microsoft.com/azure/aks/csi-secrets-store-driver)
|
||||
* [Azure RBAC for Kubernetes Authorization](https://docs.microsoft.com/azure/aks/manage-azure-rbac)
|
||||
* [Azure Active Directory pod-managed identities](https://docs.microsoft.com/azure/aks/use-azure-ad-pod-identity)
|
||||
* [Secret store CSI driver](https://learn.microsoft.com/azure/aks/csi-secrets-store-driver)
|
||||
* [Azure RBAC for Kubernetes Authorization](https://learn.microsoft.com/azure/aks/manage-azure-rbac)
|
||||
* [Azure Active Directory pod-managed identities](https://learn.microsoft.com/azure/aks/use-azure-ad-pod-identity)
|
||||
|
||||
## A future workload for this scenario will include the following
|
||||
* Horizontal Pod Autoscaling
|
||||
|
|
|
|||
|
|
@ -4,15 +4,15 @@ This is the starting point for the instructions on deploying the [AKS Baseline p
|
|||
|
||||
## Steps
|
||||
|
||||
1. Latest [Azure CLI installed](https://docs.microsoft.com/cli/azure/install-azure-cli?view=azure-cli-latest) (must be at least 2.37), or you can perform this from Azure Cloud Shell by clicking below.
|
||||
1. Latest [Azure CLI installed](https://learn.microsoft.com/cli/azure/install-azure-cli?view=azure-cli-latest) (must be at least 2.37), or you can perform this from Azure Cloud Shell by clicking below.
|
||||
1. An Azure subscription.
|
||||
|
||||
The subscription used in this deployment cannot be a [free account](https://azure.microsoft.com/free); it must be a standard EA, pay-as-you-go, or Visual Studio benefit subscription. This is because the resources deployed here are beyond the quotas of free subscriptions.
|
||||
|
||||
> :warning: The user or service principal initiating the deployment process _must_ have the following minimal set of Azure Role-Based Access Control (RBAC) roles:
|
||||
>
|
||||
> * [Contributor role](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#contributor) is _required_ at the subscription level to have the ability to create resource groups and perform deployments.
|
||||
> * [User Access Administrator role](https://docs.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator) is _required_ at the subscription level since you'll be performing role assignments to managed identities across various resource groups.
|
||||
> * [Contributor role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#contributor) is _required_ at the subscription level to have the ability to create resource groups and perform deployments.
|
||||
> * [User Access Administrator role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator) is _required_ at the subscription level since you'll be performing role assignments to managed identities across various resource groups.
|
||||
|
||||
1. **This step only applies if you are creating a new AAD group for this deployment. If you have one already existing and you are a part of it, you can skip this prerequisite, and follow the import portion of the instructions below**.
|
||||
|
||||
|
|
@ -20,8 +20,8 @@ This is the starting point for the instructions on deploying the [AKS Baseline p
|
|||
|
||||
> :warning: The user or service principal initiating the deployment process _must_ have the following minimal set of Azure AD permissions assigned:
|
||||
>
|
||||
> * Azure AD [User Administrator](https://docs.microsoft.com/azure/active-directory/users-groups-roles/directory-assign-admin-roles#user-administrator-permissions) is _required_ to create a "break glass" AKS admin Active Directory Security Group and User. Alternatively, you could get your Azure AD admin to create this for you when instructed to do so.
|
||||
> * If you are not part of the User Administrator group in the tenant associated to your Azure subscription, please consider [creating a new tenant](https://docs.microsoft.com/azure/active-directory/fundamentals/active-directory-access-create-new-tenant#create-a-new-tenant-for-your-organization) to use while evaluating this implementation. The Azure AD tenant backing your cluster's API RBAC does NOT need to be the same tenant associated with your Azure subscription.
|
||||
> * Azure AD [User Administrator](https://learn.microsoft.com/azure/active-directory/users-groups-roles/directory-assign-admin-roles#user-administrator-permissions) is _required_ to create a "break glass" AKS admin Active Directory Security Group and User. Alternatively, you could get your Azure AD admin to create this for you when instructed to do so.
|
||||
> * If you are not part of the User Administrator group in the tenant associated to your Azure subscription, please consider [creating a new tenant](https://learn.microsoft.com/azure/active-directory/fundamentals/active-directory-access-create-new-tenant#create-a-new-tenant-for-your-organization) to use while evaluating this implementation. The Azure AD tenant backing your cluster's API RBAC does NOT need to be the same tenant associated with your Azure subscription.
|
||||
|
||||
## Create or Import Azure Active Directory Groups for AKS
|
||||
Before creating the Azure Active Directory integrated cluster, groups must be created that can be later mapped to the Built-In Roles of "Azure Kubernetes Service Cluster User Role" and "Azure Kubernetes Service RBAC Cluster Admin".
|
||||
|
|
|
|||
|
|
@ -70,40 +70,40 @@ if not enter the command below to enable it
|
|||
az feature register --namespace "Microsoft.ContainerService" --name "AKS-AzureKeyVaultSecretsProvider"
|
||||
```
|
||||
|
||||
It takes a few minutes for the status to show *Registered*. Verify the registration status by using the [az feature list](https://docs.microsoft.com/en-us/cli/azure/feature#az_feature_list) command:
|
||||
It takes a few minutes for the status to show *Registered*. Verify the registration status by using the [az feature list](https://learn.microsoft.com/cli/azure/feature#az_feature_list) command:
|
||||
|
||||
```bash
|
||||
az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-AzureKeyVaultSecretsProvider')].{Name:name,State:properties.state}"
|
||||
```
|
||||
|
||||
When ready, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [az provider register](https://docs.microsoft.com/en-us/cli/azure/provider#az_provider_register) command:
|
||||
When ready, refresh the registration of the *Microsoft.ContainerService* resource provider by using the [az provider register](https://learn.microsoft.com/cli/azure/provider#az_provider_register) command:
|
||||
|
||||
```bash
|
||||
az provider register --namespace Microsoft.ContainerService
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Enable Keyvault Secrets Provider for your cluster
|
||||
|
||||
```
|
||||
```bash
|
||||
az aks enable-addons --addons azure-keyvault-secrets-provider --name $AKSCLUSTERNAME --resource-group $AKSRESOURCEGROUP
|
||||
```
|
||||
|
||||
**IMPORTANT**: When completed, take note of the client-id created for the add-on:
|
||||
|
||||
...,
|
||||
"addonProfiles": {
|
||||
"azureKeyvaultSecretsProvider": {
|
||||
...,
|
||||
"identity": {
|
||||
"clientId": "<client-id>",
|
||||
...
|
||||
}
|
||||
```json
|
||||
"addonProfiles": {
|
||||
"azureKeyvaultSecretsProvider": {
|
||||
...,
|
||||
"identity": {
|
||||
"clientId": "<client-id>",
|
||||
...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Update the permissions on the Key Vault to allow access from the newly created identity. The object-type can be certificate, key or secret. In this case, it should be all 3. Run the command below 3 times, one for each of the options.
|
||||
```
|
||||
|
||||
```bash
|
||||
az keyvault set-policy -n $KV_NAME -g $KV_RESOURCEGROUP --<object type>-permissions get --spn <client-id>
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Deploy a Basic Workload using the Fruit Smoothie Ratings Application
|
||||
|
||||
This application is provided by Microsoft Learning and is used as part of a self-paced Kubernetes training [workshop](https://docs.microsoft.com/en-us/learn/modules/aks-workshop/). You may find reviewing that workshop helpful as it presents some alternative deployment options and features using different architecture requirements. The application consists of a web frontend, an API service and a MongoDB database.
|
||||
The application consists of a web frontend, an API service and a MongoDB database.
|
||||
|
||||
Because the infrastructure has been deployed in a private AKS cluster setup with private endpoints for the container registry and other components, you will need to perform the application container build and the publishing to the Container Registry from the Dev Jumpbox in the Hub VNET, connecting via the Bastion Host service. If your computer is connected to the hub network, you may be able to just use that as well. The rest of the steps can be performed on your local machine by using AKS Run commands which allow access into private clusters using RBAC. This will help with improving security and will provide a more user-friendly way of editing YAML files.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
# Open Service Mesh AKS add-on
|
||||
|
||||
The [Open Service Mesh AKS add-on](https://learn.microsoft.com/en-us/azure/aks/open-service-mesh-about) is well documented on how to install and/or enable the AKS add-on, as well as how to deploy a sample application or onboard an existing application. Please review the following links to get started securing your AKS workloads with Open Service Mesh.
|
||||
The [Open Service Mesh AKS add-on](https://learn.microsoft.com/azure/aks/open-service-mesh-about) is well documented on how to install and/or enable the AKS add-on, as well as how to deploy a sample application or onboard an existing application. Please review the following links to get started securing your AKS workloads with Open Service Mesh.
|
||||
|
||||
## Installation
|
||||
|
||||
[Install the Open Service Mesh add-on by using the Azure CLI](https://learn.microsoft.com/en-us/azure/aks/open-service-mesh-deploy-addon-az-cli)
|
||||
[Install the Open Service Mesh add-on by using the Azure CLI](https://learn.microsoft.com/azure/aks/open-service-mesh-deploy-addon-az-cli)
|
||||
|
||||
## Deploy a Sample Application
|
||||
|
||||
|
|
|
|||
|
|
@ -12,4 +12,4 @@ The topics described in these scenarios will provide guidance on how to protect
|
|||
|
||||
A [Service Mesh](https://en.wikipedia.org/wiki/Service_mesh) provides a way to make communications between service endpoints in your Kubernetes cluster secure by encrypting the communications by way of a proxy. This can be beneficial in several ways, first you can offload the need of your applications having to negotiate encryption as part of its code base, secondly a service mesh provides a single operational control experience to provide security policies and transport security observability across your whole cluster.
|
||||
|
||||
[OSM (Open Service Mesh)](openservicemesh.io), is an open-source service mesh, that is integrated with the AKS service as a [managed AKS add-on](https://learn.microsoft.com/en-us/azure/aks/open-service-mesh-about) providing a free fully supported service mesh experience.
|
||||
[OSM (Open Service Mesh)](openservicemesh.io), is an open-source service mesh, that is integrated with the AKS service as a [managed AKS add-on](https://learn.microsoft.com/azure/aks/open-service-mesh-about) providing a free fully supported service mesh experience.
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ AKS-HCI is a turn-key solution for Administrators to easily deploy, manage lifec
|
|||
|
||||
AKS-HCI is an Azure service that is hybrid by design. It leverages our experience with AKS, follows the AKS design patterns and best-practices, and uses code directly from AKS. This means that you can use AKS-HCI to develop applications on AKS and deploy them unchanged on-premises. It also means that any skills that you learn with AKS on Azure Stack HCI are transferable to AKS as well. With Azure Arc capability built-in, you can manage your fleet of clusters centrally from Azure, deploy applications and apply configuration using GitOps-based configuration management, view and monitor your clusters using Azure Monitor for containers, enforce threat protection using Azure Defender for Kubernetes, apply policies using Azure Policy for Kubernetes, and run Azure services like Arc-enabled Data Services on premises.
|
||||
|
||||
No matter how you choose to deploy AKS-HCI – wizard-driven workflow in [Windows Admin Center (WAC)](https://docs.microsoft.com/en-us/azure-stack/aks-hci/setup) or [PowerShell](https://docs.microsoft.com/en-us/azure-stack/aks-hci/kubernetes-walkthrough-powershell) – your cluster is ready to host workloads in less than an hour. Under the hood, the deployment takes care of everything that’s required to bring up Kubernetes and run applications. This includes core Kubernetes, container runtime, networking, storage, and security, and operators to manage underlying infrastructure. Scaling the cluster up or down by adding/removing nodes and cluster-updates/upgrades are equally quick and easy. So is ongoing local management through WAC or PowerShell.
|
||||
No matter how you choose to deploy AKS-HCI – wizard-driven workflow in [Windows Admin Center (WAC)](https://learn.microsoft.com/azure-stack/aks-hci/setup) or [PowerShell](https://learn.microsoft.com/azure-stack/aks-hci/kubernetes-walkthrough-powershell) – your cluster is ready to host workloads in less than an hour. Under the hood, the deployment takes care of everything that’s required to bring up Kubernetes and run applications. This includes core Kubernetes, container runtime, networking, storage, and security, and operators to manage underlying infrastructure. Scaling the cluster up or down by adding/removing nodes and cluster-updates/upgrades are equally quick and easy. So is ongoing local management through WAC or PowerShell.
|
||||
|
||||
AKS-HCI is the best platform for running .Net Core and Framework applications – whether your applications are based on Linux or Windows. The infrastructure required to run containers is included and fully supported. For Windows, AKS-HCI offers an industry-leading solution with advanced features like GMSA non-domain joined hosts, Active Directory integration, and WAC based application deployment, migration, and management. We want to ensure that AKS-HCI remains the best destination for Windows containers.
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ On Azure, You don't need to setup your own Prometheus server: Azure Container In
|
|||
|
||||
|
||||
Read the documentation on Prometheus Integration:
|
||||
https://docs.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-prometheus-integration
|
||||
<https://learn.microsoft.com/azure/azure-monitor/containers/container-insights-prometheus-integration>
|
||||
|
||||
|
||||
|
||||
|
|
@ -23,23 +23,24 @@ To enable scraping with Container insights, you simply need to deploy a ConfigMa
|
|||
You can find [an example, in this repository](./container-azm-ms-agentconfig.yaml) for testing purposes (we recommend to download the latest version from the documentation link above).
|
||||
|
||||
In this example (because Velero already exports the monitoring metrics), we simply *enable monitoring_kubernetes_pods* in the ConfigMap, and that's it !
|
||||
```bash
|
||||
monitor_kubernetes_pods = true
|
||||
## Restricts Kubernetes monitoring to namespaces for pods that have annotations set and are scraped using the monitor_kubernetes_pods setting.
|
||||
## This will take effect when monitor_kubernetes_pods is set to true
|
||||
# ex. monitor_kubernetes_pods_namespaces = ["velero"]
|
||||
|
||||
```yaml
|
||||
monitor_kubernetes_pods = true
|
||||
## Restricts Kubernetes monitoring to namespaces for pods that have annotations set and are scraped using the monitor_kubernetes_pods setting.
|
||||
## This will take effect when monitor_kubernetes_pods is set to true
|
||||
# ex. monitor_kubernetes_pods_namespaces = ["velero"]
|
||||
```
|
||||
|
||||
- Run the following command to deploy the configMap to the AKS cluster:
|
||||
```
|
||||
kubectl apply -f container-azm-ms-agentconfig.yaml
|
||||
- Run the following command to deploy the configMap to the AKS cluster:
|
||||
|
||||
``` bash
|
||||
kubectl apply -f container-azm-ms-agentconfig.yaml
|
||||
```
|
||||
|
||||
## Viewing Velero Metrics in Azure
|
||||
## Viewing Velero Metrics in Azure
|
||||
|
||||
You can access Insights on Velero Metrics, by viewing the *InsightsMetrics* of your AKS cluster.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -2,22 +2,22 @@
|
|||
|
||||
|
||||
|
||||
**First, check out** [Best practices for business continuity and disaster recovery in Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/operator-best-practices-multi-region)
|
||||
**First, check out** [Best practices for business continuity and disaster recovery in Azure Kubernetes Service (AKS)](https://learn.microsoft.com/azure/aks/operator-best-practices-multi-region)
|
||||
|
||||
## High Availability Considerations
|
||||
|
||||
|
||||
* **AKS Cluster Configuration**:
|
||||
- Enable [Uptime SLA](https://docs.microsoft.com/en-us/azure/aks/uptime-sla) for production workloads
|
||||
- Use [Availability Zones](https://docs.microsoft.com/en-us/azure/aks/availability-zones) (with Standard Load Balancer)
|
||||
- Use [multiple node pools](https://docs.microsoft.com/en-us/azure/aks/use-multiple-node-pools) spanning AZs
|
||||
- Enforce [Resource Quotas](https://docs.microsoft.com/en-us/azure/aks/operator-best-practices-scheduler#enforce-resource-quotas) and Plan for [pod disruption budgets](https://docs.microsoft.com/en-us/azure/aks/operator-best-practices-scheduler#plan-for-availability-using-pod-disruption-budgets)
|
||||
- Control Pod scheduling using [Taints & Tolerations](https://docs.microsoft.com/en-us/azure/aks/operator-best-practices-advanced-scheduler#provide-dedicated-nodes-using-taints-and-tolerations), & [Pod Affinity](https://docs.microsoft.com/en-us/azure/aks/operator-best-practices-advanced-scheduler#control-pod-scheduling-using-node-selectors-and-affinity)
|
||||
- Enable [Uptime SLA](https://learn.microsoft.com/azure/aks/uptime-sla) for production workloads
|
||||
- Use [Availability Zones](https://learn.microsoft.com/azure/aks/availability-zones) (with Standard Load Balancer)
|
||||
- Use [multiple node pools](https://learn.microsoft.com/azure/aks/use-multiple-node-pools) spanning AZs
|
||||
- Enforce [Resource Quotas](https://learn.microsoft.com/azure/aks/operator-best-practices-scheduler#enforce-resource-quotas) and Plan for [pod disruption budgets](https://learn.microsoft.com/azure/aks/operator-best-practices-scheduler#plan-for-availability-using-pod-disruption-budgets)
|
||||
- Control Pod scheduling using [Taints & Tolerations](https://learn.microsoft.com/azure/aks/operator-best-practices-advanced-scheduler#provide-dedicated-nodes-using-taints-and-tolerations), & [Pod Affinity](https://learn.microsoft.com/azure/aks/operator-best-practices-advanced-scheduler#control-pod-scheduling-using-node-selectors-and-affinity)
|
||||
|
||||
|
||||
|
||||
* **Applications**:
|
||||
- Configure applications [requests & limits](https://docs.microsoft.com/en-us/azure/aks/developer-best-practices-resource-management#define-pod-resource-requests-and-limits)
|
||||
- Configure applications [requests & limits](https://learn.microsoft.com/azure/aks/developer-best-practices-resource-management#define-pod-resource-requests-and-limits)
|
||||
- to ensure the PVs are located in the same zone as the pods:
|
||||
- Use Volume Binding Mode: WaitForFirstConsumer (In your storage classes)
|
||||
- Use StatefulSets
|
||||
|
|
@ -41,7 +41,7 @@ Checkout the repo section on [High Availability Baseline](https://github.com/Azu
|
|||
|
||||
|
||||
- It might be simpler for most cases
|
||||
- You can expose the storage account to both regions, in a secure manner, via [Azure private link](https://docs.microsoft.com/en-us/azure/private-link/private-link-overview)
|
||||
- You can expose the storage account to both regions, in a secure manner, via [Azure private link](https://learn.microsoft.com/azure/private-link/private-link-overview)
|
||||
- The seconday cluster should be configured to have readonly access to the backup storage
|
||||
- to enable Regional Disastery Recovery, Storage account should be configured to have regional redundancy (sku RA-GRS or RA-GZRS)
|
||||
|
||||
|
|
@ -50,7 +50,7 @@ Checkout the repo section on [High Availability Baseline](https://github.com/Azu
|
|||
|
||||
|
||||
- This scenario offers better security as it ensure a strict isolation of environments
|
||||
- Velero (blob) Container is configured with [Object Replication](https://docs.microsoft.com/en-us/azure/storage/blobs/object-replication-overview?tabs=powershell), to the secondary storage account
|
||||
- Velero (blob) Container is configured with [Object Replication](https://learn.microsoft.com/azure/storage/blobs/object-replication-overview?tabs=powershell), to the secondary storage account
|
||||
|
||||
|
||||
## Integrate Velero with your infrastructure:
|
||||
|
|
@ -72,7 +72,7 @@ Checkout the repo section on [High Availability Baseline](https://github.com/Azu
|
|||
|
||||
- Prepare Cluster Node Pools :
|
||||
- Create Nodes & re-deploy Node Configuration
|
||||
- Use Automated configuration using CICD or [GitOps!](https://docs.microsoft.com/en-us/azure/azure-arc/kubernetes/conceptual-gitops-flux2)
|
||||
- Use Automated configuration using CICD or [GitOps!](https://learn.microsoft.com/azure/azure-arc/kubernetes/conceptual-gitops-flux2)
|
||||
|
||||
|
||||
- Prepare Applications Persistent volumes :
|
||||
|
|
@ -92,7 +92,7 @@ Checkout the repo section on [High Availability Baseline](https://github.com/Azu
|
|||
|
||||
➡️ A tool such as Velero simplifies the process fo backup & restore for stateful applications
|
||||
|
||||
➡️ **Coming Soon!** Perform Backup for Persistent Volume of AKS clusters using [Azure Backup](https://azure.microsoft.com/en-us/updates/akspvbackupprivatepreview/)
|
||||
➡️ **Coming Soon!** Perform Backup for Persistent Volume of AKS clusters using [Azure Backup](https://azure.microsoft.com/updates/akspvbackupprivatepreview/)
|
||||
|
||||
:arrow_forward: [Deep Dive on Velero configuration for AKS](./velero_terraform_sample)
|
||||
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ Velero is a plugin based tool. You can use the following plugins to run Velero o
|
|||
<a href="https://github.com/vmware-tanzu/velero-plugin-for-csi" target="_blank">velero-plugin-for-csi</a>
|
||||
- **A volume snapshotter plugin** for CSI backed PVCs using the CSI beta snapshot APIs for Kubernetes.
|
||||
- See [how Velero supports CSI Snapshot API](https://velero.io/docs/v1.8/csi/)
|
||||
- It supports <a href="https://docs.microsoft.com/en-us/azure/aks/azure-disk-csi" target="_blank">Azure Disks</a> `disk.csi.azure.com`
|
||||
- It supports [Azure Disks](https://learn.microsoft.com/azure/aks/azure-disk-csi) `disk.csi.azure.com`
|
||||
- Volume snapshots are configured using a VolumeSnapshotClass:
|
||||
- <a href="https://github.com/kubernetes-sigs/azuredisk-csi-driver/blob/master/deploy/example/snapshot/storageclass-azuredisk-snapshot.yaml" target="_blank">Azure Disk VolumeSnapshotClass</a>
|
||||
- Limitation:
|
||||
|
|
@ -71,7 +71,7 @@ Velero’s backups are split into 2 pieces :
|
|||
- You can use CSI Snapshots to restore to a cluster in the same Region.
|
||||
- Regional volume snapshot with CSI Driver is coming soon ! --> to Restore to a cluster in a secondary region, use Restic for now
|
||||
|
||||
- Note on <a href="https://docs.microsoft.com/en-us/azure/aks/availability-zones#azure-disk-availability-zone-support" target="_blank">Azure Disk Availability Zone support</a>
|
||||
- Note on [Azure Disk Availability Zone support](https://learn.microsoft.com/azure/aks/availability-zones#azure-disk-availability-zone-support)
|
||||
- Volumes that use Azure managed LRS disks are not zone-redundant resources, those volumes cannot be attached across zones and must be co-located in the same zone as a given node hosting the target pod
|
||||
- Kubernetes is aware of Azure availability zones since version 1.12. You can deploy a PersistentVolumeClaim object referencing an Azure Managed Disk in a multi-zone AKS cluster and Kubernetes will take care of scheduling any pod that claims this PVC in the correct availability zone.
|
||||
- See How to use Availability Zones in your StorageClasses: https://kubernetes-sigs.github.io/cloud-provider-azure/topics/availability-zones/
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
This article has been moved to the Azure architecture center and be accessed here: [Blue-green deployment for AKS](https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/containers/blue-green-deployment-for-aks/blue-green-deployment-for-aks)
|
||||
This article has been moved to the Azure architecture center and be accessed here: [Blue-green deployment for AKS](https://learn.microsoft.com/azure/architecture/reference-architectures/containers/blue-green-deployment-for-aks/blue-green-deployment-for-aks)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Blue Green Deployment for AKS
|
||||
|
||||
This architectural pattern describes how to properly implement a Blue-Green deployment of an AKS cluster that follows the guiding tenets of the [Azure Well-Architected Framework](https://docs.microsoft.com/en-us/azure/architecture/framework/). For Blue Green deployment at the application level, refer to [this article](https://docs.microsoft.com/en-us/azure/architecture/example-scenario/blue-green-spring/blue-green-spring).
|
||||
This architectural pattern describes how to properly implement a Blue-Green deployment of an AKS cluster that follows the guiding tenets of the [Azure Well-Architected Framework](https://learn.microsoft.com/azure/architecture/framework/). For Blue Green deployment at the application level, refer to [this article](https://learn.microsoft.com/azure/architecture/example-scenario/blue-green-spring/blue-green-spring).
|
||||
The main purpose of this pattern is to provide a reliable and high availability solution when performing the following tasks:
|
||||
|
||||
- Kubernetes version update
|
||||
|
|
@ -11,7 +11,7 @@ The main purpose of this pattern is to provide a reliable and high availability
|
|||
In the above mentioned scenario the desired outcome is to apply these changes without affecting the applications and/workloads hosted in the AKS cluster.
|
||||
This pattern is also at the basis for the mission critical deployment of workloads on AKS, the main difference is that in that scenario, the resiliency and AKS distribution in multiple regions are the main drivers and elements of the solution.
|
||||
|
||||
The proposed pattern comes also with a Reference Architecture document in the Azure architecture center [Blue-green deployment for AKS](https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/containers/blue-green-deployment-for-aks/blue-green-deployment-for-aks).
|
||||
The proposed pattern comes also with a Reference Architecture document in the Azure architecture center [Blue-green deployment for AKS](https://learn.microsoft.com/azure/architecture/reference-architectures/containers/blue-green-deployment-for-aks/blue-green-deployment-for-aks).
|
||||
Deploy this scenario using the step by step guidance by clicking on the link below:
|
||||
|
||||
:arrow_forward: [Terraform](blue-green-deployment.md)
|
||||
|
|
|
|||
|
|
@ -4,9 +4,9 @@ In this walkthrough, we will explore the options of Azure Kubernetes Services (A
|
|||
It will demonstrate the setup and use of following AKS platform capabilities:
|
||||
|
||||
- The [Kubernetes Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/), adding and removing pods to the existing set of virtual machines as load changes.
|
||||
- The [Cluster Autoscaler of an AKS cluster](https://docs.microsoft.com/en-us/azure/aks/cluster-autoscaler), adding and removing virtual machines to scale up the scale set and providing more CPU and memory capacity.
|
||||
- The [Cluster Autoscaler of an AKS cluster](https://learn.microsoft.com/azure/aks/cluster-autoscaler), adding and removing virtual machines to scale up the scale set and providing more CPU and memory capacity.
|
||||
|
||||
To emulate user load, this tutorial uses [Azure Load Testing](https://docs.microsoft.com/en-us/azure/load-testing/overview-what-is-azure-load-testing).
|
||||
To emulate user load, this tutorial uses [Azure Load Testing](https://learn.microsoft.com/azure/load-testing/overview-what-is-azure-load-testing).
|
||||
|
||||
## Walthrough Overview
|
||||
In this walkthrough, you will...
|
||||
|
|
@ -52,12 +52,20 @@ In this walkthrough, you will...
|
|||
|
||||
|
||||
```bash
|
||||
$ az acr list -g "az-k8s-khim-rg" -o table
|
||||
az acr list -g "az-k8s-khim-rg" -o table
|
||||
```
|
||||
|
||||
```output
|
||||
NAME RESOURCE GROUP LOCATION SKU LOGIN SERVER CREATION DATE ADMIN ENABLED
|
||||
------------------------ ---------------- ---------- ----- ----------------------------------- -------------------- ---------------
|
||||
crazk8skhimqwzol4vktwxre az-k8s-khim-rg westeurope Basic crazk8skhimqwzol4vktwxre.azurecr.io 2022-09-08T11:03:35Z False
|
||||
|
||||
$ az aks list -g "az-k8s-khim-rg" -o table
|
||||
```
|
||||
|
||||
```bash
|
||||
az aks list -g "az-k8s-khim-rg" -o table
|
||||
```
|
||||
|
||||
```output
|
||||
Name Location ResourceGroup KubernetesVersion CurrentKubernetesVersion ProvisioningState Fqdn
|
||||
--------------- ---------- --------------- ------------------- -------------------------- ------------------- -------------------------------------------------
|
||||
aks-az-k8s-khim westeurope az-k8s-khim-rg 1.23.8 1.23.8 Succeeded az-k8s-khim-dns-318a3497.hcp. westeurope.azmk8s.io
|
||||
|
|
@ -66,32 +74,35 @@ In this walkthrough, you will...
|
|||
1. Check that you have the [`kubectl`](https://kubernetes.io/docs/tasks/tools/#kubectl) available on your machine.
|
||||
|
||||
```bash
|
||||
$ kubectl version
|
||||
kubectl version
|
||||
```
|
||||
|
||||
```output
|
||||
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.0", GitCommit:"a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2", GitTreeState:"clean", BuildDate:"2022-08-23T17:44:59Z", GoVersion:"go1.19", Compiler:"gc", Platform:"linux/amd64"}
|
||||
Kustomize Version: v4.5.7
|
||||
...
|
||||
```
|
||||
|
||||
If `kubectl` is not yet available, install it using Azure CLI:
|
||||
|
||||
```bash
|
||||
$ az aks install-cli
|
||||
az aks install-cli
|
||||
```
|
||||
|
||||
1. Log in to your AKS cluster using [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-get-credentials).
|
||||
1. Log in to your AKS cluster using [Azure CLI](https://learn.microsoft.com/cli/azure/aks?view=azure-cli-latest#az-aks-get-credentials).
|
||||
|
||||
```bash
|
||||
$ az aks get-credentials \
|
||||
az aks get-credentials \
|
||||
--resource-group az-k8s-khim-rg \
|
||||
--name aks-az-k8s-khim
|
||||
Merged "aks-az-k8s-khim" as current context
|
||||
```
|
||||
|
||||
1. Get the list of nodes of your AKS cluster to check connectivity to your AKS cluster:
|
||||
|
||||
```bash
|
||||
$ kubectl get nodes
|
||||
kubectl get nodes
|
||||
```
|
||||
|
||||
```output
|
||||
NAME STATUS ROLES AGE VERSION
|
||||
aks-npsystem-40226941-vmss000000 Ready agent 23m v1.23.8
|
||||
aks-npuser01-40226941-vmss000000 Ready agent 23m v1.23.8
|
||||
|
|
@ -103,20 +114,23 @@ In this walkthrough, you will...
|
|||
This application provides a very simple API to generate some load on your worker nodes.
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/Azure/AKS-Landing-Zone-Accelerator.git
|
||||
git clone https://github.com/Azure/AKS-Landing-Zone-Accelerator.git
|
||||
```
|
||||
|
||||
|
||||
1. Change into directory `Scenarios/Testing-Scalability/dotnet` that hosts the demo application to be used in this walkthrough.
|
||||
|
||||
```bash
|
||||
$ cd AKS-Landing-Zone-Accelerator/Scenarios/Testing-Scalability/dotnet
|
||||
cd AKS-Landing-Zone-Accelerator/Scenarios/Testing-Scalability/dotnet
|
||||
```
|
||||
|
||||
1. (optional) If you want to test it, run `dotnet run`...
|
||||
```bash
|
||||
$ dotnet run
|
||||
|
||||
```bash
|
||||
dotnet run
|
||||
```
|
||||
|
||||
```output
|
||||
Welcome to .NET 6.0!
|
||||
---------------------
|
||||
SDK Version: 6.0.202
|
||||
|
|
@ -131,7 +145,10 @@ In this walkthrough, you will...
|
|||
and browse to endpoint `https://localhost:7230/RandomNumbers`:
|
||||
|
||||
```bash
|
||||
$ curl -k https://localhost:7230/RandomNumbers
|
||||
curl -k https://localhost:7230/RandomNumbers
|
||||
```
|
||||
|
||||
```output
|
||||
{"numbersGenerated":10000000,"timeUsed":45.7634}
|
||||
```
|
||||
|
||||
|
|
@ -191,7 +208,7 @@ In this walkthrough, you will...
|
|||
|
||||
### Run your first Load Test with Azure Load Testing
|
||||
|
||||
1. (For your information only) We will use [Azure Load Testing](https://docs.microsoft.com/en-us/azure/load-testing/overview-what-is-azure-load-testing) in the following steps. This takes a [Apache JMeter](https://jmeter.apache.org/) test plan as input to simulate load on workloads running on the Azure platform. You can use the JMeter GUI (see screenshot) to define a testplan; in this tutorial, we will use a predefined test plan.
|
||||
1. (For your information only) We will use [Azure Load Testing](https://learn.microsoft.com/azure/load-testing/overview-what-is-azure-load-testing) in the following steps. This takes a [Apache JMeter](https://jmeter.apache.org/) test plan as input to simulate load on workloads running on the Azure platform. You can use the JMeter GUI (see screenshot) to define a testplan; in this tutorial, we will use a predefined test plan.
|
||||
|
||||
|
||||
|
||||
|
|
@ -306,10 +323,15 @@ In this walkthrough, you will...
|
|||
1. Increase `maxReplicas` in `randomnumbers-hpa.yaml` with an editor of your choice to 50 and redeploy the pod autoscaler.
|
||||
|
||||
```bash
|
||||
$ grep maxReplicas ./randomnumbers-hpa.yaml
|
||||
maxReplicas: 50
|
||||
grep maxReplicas ./randomnumbers-hpa.yaml
|
||||
```
|
||||
|
||||
$ kubectl apply -f randomnumbers-hpa.yaml
|
||||
```output
|
||||
maxReplicas: 50
|
||||
```
|
||||
|
||||
```bash
|
||||
kubectl apply -f randomnumbers-hpa.yaml
|
||||
```
|
||||
|
||||
1. Create another test in Azure Load Testing with "50 threads, 150 loops" and run it.
|
||||
|
|
@ -331,7 +353,7 @@ In this walkthrough, you will...
|
|||
|
||||
The query is defined as follows...
|
||||
|
||||
```
|
||||
```kql
|
||||
KubeEvents
|
||||
| where TimeGenerated > ago(7d)
|
||||
| where not(isempty(Namespace))
|
||||
|
|
@ -340,19 +362,19 @@ In this walkthrough, you will...
|
|||
|
||||
...and will reveal messages like:
|
||||
|
||||
```
|
||||
```output
|
||||
0/2 nodes are available: 1 Insufficient cpu, 1 node(s) had taint {CriticalAddonsOnly: true}, that the pod didn't tolerate.
|
||||
```
|
||||
|
||||
The interesting part is `1 Insufficient cpu`, preventing pods from being assigned to the nodes of our `npuser01` node pool. Let us take a closer look at one of these nodes using:
|
||||
|
||||
```bash
|
||||
$ kubectl describe node aks-npuser01-37699233-vmss000000
|
||||
kubectl describe node aks-npuser01-37699233-vmss000000
|
||||
```
|
||||
|
||||
The output reveals that 1870 of 1900 available mili cores have already been allocated.
|
||||
|
||||
```
|
||||
```output
|
||||
Allocatable:
|
||||
cpu: 1900m
|
||||
...
|
||||
|
|
@ -372,9 +394,9 @@ In this walkthrough, you will...
|
|||
|
||||
### Enable the cluster autoscaler to add scale up the Virtual Machine Scale Set and add further VMs on demand
|
||||
|
||||
1. In the next step, you will enable the [cluster autoscaler](https://learn.microsoft.com/en-us/azure/aks/cluster-autoscaler) for your AKS cluster. It will add nodes to your node pool when pods cannot be scheduled due to resource constraints and will remove nodes from node pools when consolidation of pods allows.
|
||||
1. In the next step, you will enable the [cluster autoscaler](https://learn.microsoft.com/azure/aks/cluster-autoscaler) for your AKS cluster. It will add nodes to your node pool when pods cannot be scheduled due to resource constraints and will remove nodes from node pools when consolidation of pods allows.
|
||||
|
||||
1. Run the following command to change the default cluster autoscaler profile (default values can be found in the [AKS Cluster REST API documentation](https://learn.microsoft.com/en-us/rest/api/aks/managed-clusters/create-or-update?tabs=HTTP#autoscalerprofile)). These parameters enable a rather aggressive scale-down to avoid longer waiting times in this tutorial. Please be mindful when setting these values in your own cluster.
|
||||
1. Run the following command to change the default cluster autoscaler profile (default values can be found in the [AKS Cluster REST API documentation](https://learn.microsoft.com/rest/api/aks/managed-clusters/create-or-update?tabs=HTTP#autoscalerprofile)). These parameters enable a rather aggressive scale-down to avoid longer waiting times in this tutorial. Please be mindful when setting these values in your own cluster.
|
||||
|
||||
```bash
|
||||
az aks update \
|
||||
|
|
@ -438,7 +460,8 @@ In this walkthrough, you will...
|
|||
|
||||
|
||||
1. After some time, the number of pods and nodes will decrease again. The AKS logs reveal some further information:
|
||||
```
|
||||
|
||||
```kql
|
||||
AzureDiagnostics
|
||||
| where Category == "cluster-autoscaler"
|
||||
| project TimeGenerated, attrs_s, log_s, pod_s
|
||||
|
|
@ -446,7 +469,8 @@ In this walkthrough, you will...
|
|||
```
|
||||
|
||||
...there will be messages like:
|
||||
```
|
||||
|
||||
```output
|
||||
I0504 09:10:17.929359 1 azure_scale_set.go:755] Calling virtualMachineScaleSetsClient.DeleteInstancesAsync(&[6]) for aks-npuser01-61737176-vmss
|
||||
I0504 09:10:17.929205 1 azure_scale_set.go:705] Deleting vmss instances [azure:///subscriptions/ce9d064e-10a7-4b7c-8e8e-561fb2e718dd/resourceGroups/leho-rg-leho-aks-rio6zecikhluy-nodepools/providers/Microsoft.Compute/virtualMachineScaleSets/aks-npuser01-61737176-vmss/virtualMachines/6]
|
||||
I0504 09:10:17.928963 1 scale_down.go:1478] All pods removed from aks-npuser01-61737176-vmss000006
|
||||
|
|
@ -465,23 +489,23 @@ In this walkthrough, you will...
|
|||
# Resources
|
||||
|
||||
## AKS Scaling and Monitoring
|
||||
[Automatically scale a cluster to meet application demands on Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/cluster-autoscaler)
|
||||
[Automatically scale a cluster to meet application demands on Azure Kubernetes Service (AKS)](https://learn.microsoft.com/azure/aks/cluster-autoscaler)
|
||||
|
||||
|
||||
[Examine the node and pod health](https://docs.microsoft.com/en-us/azure/architecture/operator-guides/aks/aks-triage-node-health)
|
||||
[Examine the node and pod health](https://learn.microsoft.com/azure/architecture/operator-guides/aks/aks-triage-node-health)
|
||||
|
||||
[AKS troubleshooting](https://docs.microsoft.com/en-us/azure/aks/troubleshooting)
|
||||
[AKS troubleshooting](https://learn.microsoft.com/azure/aks/troubleshooting)
|
||||
|
||||
_Resource logs_ in [Monitoring AKS data reference](https://docs.microsoft.com/en-us/azure/aks/monitor-aks-reference#resource-logs)
|
||||
_Resource logs_ in [Monitoring AKS data reference](https://learn.microsoft.com/azure/aks/monitor-aks-reference#resource-logs)
|
||||
|
||||
[Monitoring Azure Kubernetes Service (AKS) with Azure Monitor](https://docs.microsoft.com/en-us/azure/aks/monitor-aks)
|
||||
[Monitoring Azure Kubernetes Service (AKS) with Azure Monitor](https://learn.microsoft.com/azure/aks/monitor-aks)
|
||||
[Horizontal Pod Autoscaling](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/)
|
||||
|
||||
## Kubernetes Scaling
|
||||
|
||||
[Horizontal Pod Autoscaler Walkthrough](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/)
|
||||
|
||||
[How to query logs from Container insights](https://docs.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-log-query#resource-logs)
|
||||
[How to query logs from Container insights](https://learn.microsoft.com/azure/azure-monitor/containers/container-insights-log-query#resource-logs)
|
||||
|
||||
_Frequently Asked Questions_ (`autoscaler/cluster-autoscaler/FAQ.md`) in [kubernetes/autoscaler](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md)
|
||||
|
||||
|
|
@ -491,7 +515,7 @@ _Frequently Asked Questions_ (`autoscaler/cluster-autoscaler/FAQ.md`) in [kubern
|
|||
|
||||
## Azure Load Testing
|
||||
|
||||
[Quickstart: Create and run a load test with Azure Load Testing Preview](https://docs.microsoft.com/en-us/azure/load-testing/quickstart-create-and-run-load-test)
|
||||
[Quickstart: Create and run a load test with Azure Load Testing Preview](https://learn.microsoft.com/azure/load-testing/quickstart-create-and-run-load-test)
|
||||
|
||||
|
||||
# :construction: Todos
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче