RodrigTech

API Server VNET Integration

—

by

rodrigtech
in

Connectivity in AKS

If you’re running AKS in production you’ll likely encounter the private link scope and integration of leverage private DNS zones for putting the API server behind private IP’s rather than accessible on port 6443 or you should be doing this. But what about other options? Perhaps you’re spinning up a dev/test cluster for 10-20 minutes and still want a hybrid option. Few do exist in terms of access and document below on how you can operate AKS Creation and API Server Access.

Access Methods

Broadly the access methods currently can be broken down into a few options.

  • API IP Authorized Address Ranges
  • Private AKS Clusters (No API Server Exposed)
  • API Server VNET Integration (Relatively new)

For the purposes of the discussion in this scope well cover the last one but to be aware of your options they are expanding.

API Server VNet Integration

When you deploy Azure Kubernetes Service this opens up the path to a delegated subnet in the VNet where AKS is deployed. API Server. This path opens up an entrance point to indirectly access API Server from a separate network not the main virtual network defined but rather subnet. You can also add or remove public access after a cluster is provisioned this is a huge advantage from the perspective of the administration if you typically deploy in a private cluster. The API server will sit behind an internal load balancer this obfuscates the direct access to the cluster.

Before this iteration non-virtual network integrated clusters, always communicate directly with the private IP address of the API server internal load balancer (ILB) IP without DNS. Important to note that in a private cluster, the control plane or API server has an internal IP defined by RFC1918 (this is the most secure use).

Getting Started

You have a couple options but notably I’m going the CLI route for this you need the following a pre-requisites before leveraging this in your subscription.

az extension add --name aks-preview
az extension update --name aks-preview
az feature register --namespace "Microsoft.ContainerService" --name "EnableAPIServerVnetIntegrationPreview"

After this installs locally you have to validate the registration by the next command.

az feature show --namespace "Microsoft.ContainerService" --name "EnableAPIServerVnetIntegrationPreview"

# Then we install the following

az provider register –namespace Microsoft.ContainerService

This should show the following visual above as Registered.

#!/bin/bash

# Create a resource group
az group create -l westus -n aks-private-west

# deploy a aks cluster
az aks create -n private-west-aks \
   -g aks-private-west \
   -l westus \
   --network-plugin azure \
   --enable-private-cluster \
   --enable-apiserver-vnet-integration \
   --generate-ssh-keys

Once this is deployed we should see some of the results in the CLI output and now we have to address a few items if we want to access the cluster we have a few options we can run a bastion on the virtual network to the cluster, or we can create a bastion host in a separate v-net that can be peered with our virtual network to separate the access.

Notice this is the output from the command represented back in JSON this falls under the apiServerAccessProfile I’ve also validated that this translates into terraform in passing this through the AzureRM provider.

Now if we go into our cluster we can see the following items of the API Server V-Net Integration.

You can use the portal Run command to not directly connect to the cluster and reveal services and other commands.

Ensure you tear down the cluster you will incur cost for running this so to group the deletion you can use the following.

az group delete --resource-group aks-private-west

Summary

Azure is expanding the access to the API Server in a number of ways that give options to access the cluster what I see the expansion is those who are previously the use authorized IP ranges changing to this as another approach. Private clusters are still consider the most secure in nature and should be the first option from the security perspective due to the nature of importance in the API Server controlling operations across the cluster. This is still in preview and should not be pushed to GA but I’d definitely considered getting familiar with the use of this because I can see this increasingly being another option on securely accessing your cluster.