KubeArmor Explored

KubeArmor is a cloud-native runtime security enforcement system that works with restricting behavior (this resides with execution, file access, and network operations) of pods, containers, and nodes (VM’s) at the system level. The way this tool works is by using Linux Security Modules which to no surprise are enamored in the Certified Kubernetes Security Specialist exam so you should be familiar with the use of AppArmor, SELinux, or BPF-LSM. Little at least in the cloud-native runtime space leverage this as a predominant offering outside of Tetragon by Cilium or Isovalent that is using eBPF for enforcement and visibility. Given that this tool sits at the kernel level the visibility is really as close to the processes executed broadly across your cluster and makes this type of tool in a enforcement capacity real efficient.

Getting Started with KubeArmor

Requirements to follow along with this will be the following.

• Kubernetes Cluster (I’m using 1.29)
• Helm Installed
• KubeArmor CLI (karmor)

First we start in our terminal running the follow from helm.

helm repo add kubearmor https://kubearmor.github.io/charts
helm repo update kubearmor
helm upgrade --install kubearmor-operator kubearmor/kubearmor-operator -n kubearmor --create-namespace
kubectl apply -f https://raw.githubusercontent.com/kubearmor/KubeArmor/main/pkg/KubeArmorOperator/config/samples/sample-config.yml

If you want a visual of the raw file I’ve added it here to assist on what is occurring.

Notice we are applying to the kubearmor operator a few items in our spec we reference pulling the kubearmor/kubearmor image and others to assist the tool for configuration, notice also we can enable stdout, stdout alerts and more if those are true but we start in audit.

After this installed you’ll know you are working with kubearmor after running the karmor tool from the CLI. After this is installed using this script notice we are using “sudo”.

curl -sfL http://get.kubearmor.io/ | sudo sh -s -- -b /usr/local/bin
# sudo access is needed to install it in /usr/local/bin directory. But, if you prefer not to use sudo, you can install it in a different directory which is in your PATH.

Notice our options include quite a lot out of the box lets explore the logs first by running the following command. karmor logs

This starts or initiates the gRPC client on the local host that is using the kubearmor-relay additionally KubeArmor allows for OpenTelemetry with a otel-adapter should you want to send logs from your cluster to a SIEM tool such as Microsoft Sentinel, I’m currently building out a implementation architecture on using this as a proof-of-concept (more details in the future).

Following a example deployment of WordPress on the cluster I’ve opened up two terminals to monitor the logs in real time to show you the view here is a snap shot of the deployment and the processes that are picked up in our logs.

To simulate a violation lets run the following commands to open a shell in our container and see what it triggers.

POD_NAME=$(kubectl get pods -n wordpress-mysql -l app=wordpress -o jsonpath='{.items[0].metadata.name}') && kubectl -n wordpress-mysql exec -it $POD_NAME -- bash
# apt update

The way a policy works as we can see in this example we annotate what are selectors are telling the system specifically apply this to app: wordpress. We then define what process is off limits in the path area such as /usr/bin/apt and /usr/bin/apt-get.

While this demo is a little dated in the github repo this still gets the intended output when we run the following to see the visibility. To validate visibility we run the same command just change the pipe to annotated “kubearmor-visibility” to let the end user know what is being watched such as process, file, network, capabilities.

Since we’ve defined our policy we should have this log since this will violate our policy.

This should log in our next terminal on the violation, I ran into issues with that local cluster therefore this visual isn’t shown but as a way to show off further the KubeArmor capabilities by switching gears to visibility on posture.

If we run the following command this will run against our cluster and output recommended profile of various operations on our cluster.

karmor recommend

The output is a table format with the following parameters on the table categories such as Block, Audit along with reference to the exact control along with a description and specific YAML.

We change the terminal command to the following.

karmor recommend --out-dir ./policy --report report.txt # this defaults to report.txt

This is how it looks visually from that folder it appears it created.

If we expand to the calico-system-calico-kube-controllers this will house the policies from the report as shown in the terminal.

We can see the defined YAML will go even further from the recommendation based on the action of Audit you can alternate this to Block as well.

Below that was truncated from the image is shown to have the reference.

The report.txt that is from the karmor recommend command is a summary of all the findings along with the actions annotated once I’ve changed the font size this changed the output visual.

Now we can explore how our network is looking using KubeArmor profile feature.

karmor profile

We can see what is communicating across the network and what is the the process running along with the count aggregated.

Another view you’d like that is summarizing operations in the cluster is using the following feature known as probe this essentially checks on the entire cluster with the installation of KubeArmor.

This will also show the pods that are armored up across the cluster along with policies that are applied on workloads and will state the posture along with visibility.

Summary

KubeArmor is a great open-source CNCF Sandbox project that has real potential for cluster security, along with the use of native Linux Security Modules and also having visibility of the logs and processes around the cluster. In the background I’m working on integrating this tool into OpenTelemetry into Microsoft Azure platform for logs fed into Sentinel to leverage KubeArmor further. The security landscape is constantly evolving and the use of Linux Security Modules are still a large player in the ecosystem given that moving to the kernel level of operations this is able to lock down operations. If you are attempting Certified Kubernetes Security Specialist (CKS) I could see this working along side Falco for Threat Detection in the ecosystem as well.