Container Scanning in Kubernetes

While the escapades of Certified Kubernetes Security specialist renewal are continued and ongoing I’ve felt it was best to craft another scenario to leverage for the container security portion of the exam. Particularly you’re required to remediate many items in your cluster and ideally these items should be caught in your CI/CD pipeline before deployment. However, when that isn’t the case what is the fastest way to identify a subset of vulnerabilities? Given the mass amount of volume most practitioners face such as an uncovered CVE is our cluster affected?

Scenario

The first script deploys a few pods in various namespaces specifically one named production and another infra inside these reside a few deployments we will search for vulnerabilities for.

git clone https://github.com/sn0rlaxlife/cks-scenarios.git

cd scenario_2

Two scripts will exist in here for our scenario we will use them sequentially as we progress first we deploy our resources with the scenario.sh

chmod +x scenario.sh validate.sh
./scenario.sh

We can gather our cluster details on the following commands to get the images associated with the first namespace.

kubectl get pods -n production -o yaml | grep image:

This return Nginx:1.26.3 we can save this input first then move to our other namespace infra.

kubectl get pods -n infra -o yaml | grep image:

This returns busy box:1.36 as the image used throughout the infra namespace.

Now we were told in our scenario that CVE-2023-52425 and CVE-2024-10041.

We’re aware time is of the essence in the sense that when we are hunting throughout our environment to find affected images fairly quickly so what is a quick command for this?

trivy image <image>:<tag> | grep -E '<cve>|<cve>'

So we now ran the scan on our busy box which is associated with our namespace infra and it appears unscathed we move on to our production running the same syntax against the image running in the pods.

Now that we know that our deployment in production has the CVE’s actively running we scale down the deployments.

kubectl get deployments -n production
# our deployment is named deploy-metrics
kubectl scale deployment deploy-metrics -n production --replicas=0

Validation

Now that we’ve scaled down our identified deployment this is where the validation script comes in. We run the validation script against our cluster to validate this meets the criteria of the scenario challenge.

./validate.sh

Now to clean up the cluster we can delete our namespaces to ensure we get rid of the pods associated with infra.

k delete ns production
k delete ns infra

Summary

This scenario demonstrates the use of trivy for container scanning in our cluster which is a component of the Certified Kubernetes Security curriculum. While trivy is not limited to the scanning container images it has expanded to SBOM and Kubernetes Bill of Materials use covered in other blogs such as this one. The use of open-source tools for security is a large portion of understanding the core constructs of minimizing our attack surface with a minimal base image that is also ideally with 0 CVE’s before moving to production. While the work of this is ongoing and nothing is black and white in terms of how you meet these types of best practices its important to identify these items effectively and given time constraints for these exams any shortcut that can help you find the CVE will assist you.