Azure Kubernetes Fleet Manager (First Look)

Azure Kubernetes Fleet Manager is a product that makes it easier to manage and scale a fleet of Azure Kubernetes Service (AKS) clusters. It allows users to deploy and manage multiple AKS clusters across various environments and regions, all from a single pane of glass. This simplifies the process of managing and maintaining large-scale container-based applications in the cloud.

One of the key features of Azure Kubernetes Fleet Manager is its ability to automate the provisioning and scaling of AKS clusters. It can automatically create new clusters based on predefined rules and policies, ensuring that you have the right amount of resources available to meet the demands of your applications. It can also automatically scale clusters up or down based on workload and resource utilization, helping you to optimize your costs and maintain high levels of performance.

In addition to these automation capabilities, Azure Kubernetes Fleet Manager also provides a range of tools and features for monitoring and managing your AKS clusters. These include real-time monitoring and alerts, in-depth performance analysis, and a range of integration options with other Azure services. With Azure Kubernetes Fleet Manager, you can easily keep track of the health and performance of your AKS clusters, and take proactive steps to ensure that your applications are always running smoothly.

Open the CLI and run the command to register the feature in your subscription.

Next we will install the fleet extension

az extension add --name fleet

Run the following commands with your values for your subscription and resource along with fleet.

export SUBSCRIPTION_ID=<subscription_id>
export GROUP=<your_resource_group_name>
export FLEET=<your_fleet_name>

Now if you don’t have a resource group yet no worries these commands will get you going

az account set -s ${SUBSCRIPTION_ID}
az group create --name ${GROUP} --location eastus

Next we will create the fleet resource

az fleet create --resource-group ${GROUP} --name ${FLEET} --location eastus

export FIRST_VNET=first-vnet #you can have this as any value you choose.
export SECOND_VNET=second-vnet
export MEMBER_1_SUBNET=member-1
export MEMBER_2_SUBNET=member-2
export MEMBER_3_SUBNET=member-3

#creating the vnet1-3
az network vnet create \
    --name ${FIRST_VNET} \
    --resource-group ${GROUP} \
    --location eastus \
    --address-prefixes 10.0.0.0/8

az network vnet create \
    --name ${SECOND_VNET} \
    --resource-group ${GROUP} \
    --location westcentralus \
    --address-prefixes 10.0.0.0/8

az network vnet subnet create \
    --vnet-name ${FIRST_VNET} \
    --name ${MEMBER_1_SUBNET} \
    --resource-group ${GROUP} \
    --address-prefixes 10.1.0.0/16

#provisioning 1/3 of subnets
az network vnet subnet create \
    --vnet-name ${FIRST_VNET} \
    --name ${MEMBER_2_SUBNET} \
    --resource-group ${GROUP} \
    --address-prefixes 10.2.0.0/16

az network vnet subnet create \
    --vnet-name ${SECOND_VNET} \
    --name ${MEMBER_3_SUBNET} \
    --resource-group ${GROUP} \
    --address-prefixes 10.1.0.0/16

Then we will create a AKS Cluster

export MEMBER_CLUSTER_1=aks-member-1

az aks create \
    --resource-group ${GROUP} \
    --location eastus \
    --name ${MEMBER_CLUSTER_1} \
    --node-count 1 \
    --network-plugin azure \
    --vnet-subnet-id "/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${GROUP}/providers/Microsoft.Network/virtualNetworks/${FIRST_VNET}/subnets/${MEMBER_1_SUBNET}" --generate-ssh-key

The documentation was a little tricky for the above command but I reran this and it worked today just took some more time for provisioning.

export MEMBER_CLUSTER_ID_1=/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${GROUP}/providers/Microsoft.ContainerService/managedClusters/${MEMBER_CLUSTER_1}
export MEMBER_NAME_1=aks-member-1

az fleet member create \
    --resource-group ${GROUP} \
    --fleet-name ${FLEET} \
    --name ${MEMBER_NAME_1} \
    --member-cluster-id ${MEMBER_CLUSTER_ID_1}

Now we will go to Fleet Manager in the portal and the UI will show “Joining”

az fleet member list --resource-group ${GROUP} --fleet-name ${FLEET} -o table

Then we can get the credentials from the hub (the manager)

export FLEET_ID=/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${GROUP}/providers/Microsoft.ContainerService/fleets/${FLEET}

Now lets see how we can use cluster object propagation from the fleet to our joined cluster

az fleet get-credentials --resource-group ${GROUP} --name ${FLEET} --file fleet

az aks get-credentials --resource-group ${GROUP} --name ${MEMBER_CLUSTER_1} --file aks-member-1

KUBECONFIG=fleet kubectl create namespace aks-fleet

Next use a text editor to put this YAML file to the cluster

apiVersion: fleet.azure.com/v1alpha1
kind: ClusterResourcePlacement
metadata:
  name: aks-fleet
spec:
  resourceSelectors:
    - group: ""
      version: v1
      kind: Namespace
      name: aks-fleet
  policy:
    affinity:
      clusterAffinity:
        clusterSelectorTerms:
          - labelSelector:
              matchLabels:
                fleet.azure.com/location: eastus

Then run

KUBECONFIG=fleet kubectl apply -f run.yaml #ensure you have this

KUBECONFIG=fleet kubectl get clusterresourceplacements

Now let’s see if we can communicate this to our aks-member-1 provisioned earlier

KUBECONFIG=aks-member-1 kubectl get namespace aks-fleet

So what does this all mean well if we reveal on the previous yaml file we chose to push out the Resource Object to clusters that matched a specific label

This is to show if you have multiple amounts of clusters with resource objects you’d like to uniformly push out this is one specific use case on how Azure Fleet Manager enables cluster management at scale.

Reference

https://github.com/Azure/AKS/tree/master/examples/fleet/helloworld