RodrigTech

Kyverno Policy-as-Code in Prometheus

—

by

rodrigtech
in

Policy-as-code is a declarative nature that you can use for safe guards in your kubernetes cluster notably this relies on the Open Policy Agent. A project known as Kyverno, I’ve covered in a previous post uses this and expands the usage in a short form YAML. For this demo today I’m running Kubernetes on a KinD cluster locally and going to deploy kyverno and utilize policies report to summarize the policies we deploy along with violations.

First let’s take a look at what is needed if you’re following along

  • Cluster (hosted locally/cloud)
  • Kyverno
  • Helm 3 Installed

We start with the installation using the following syntax

kubectl create -f https://github.com/kyverno/kyverno/releases/download/v1.10.0/install.yaml

Let’s now apply some policies to our cluster ensuring we following along we will use the label policy listed below

kubectl create -f- << EOF
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-labels
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-team
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      message: "label 'team' is required"
      pattern:
        metadata:
          labels:
            team: "?*"
EOF

Additionally, we will need to use prometheus installed on our cluster so we can take advantage of the visual of our policy report so let’s use helm for this.

kubectl get policyreport

Additionally if we need to get a inventory of our policies.

kubectl get clusterpolicy

If we do a describe we get our recent violation annotated in the logs.

kubectl describe clusterpolicy require-labels

So let’s go with installing Prometheus since we will use it as our visual

https://artifacthub.io/packages/helm/prometheus-community/prometheus?modal=install

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install my-prometheus prometheus-community/prometheus --version 23.3.0

If you need to specify a namespace you could also include the –namespace flag in the above command if you’d like to segment where prometheus is deployed.

Now let’s port-forward our prometheus server with the following

kubectl port-forward my-prometheus1-server-6498d454cf-8w82h 9090

You can add the following if you need assistance and this should access our dashboard.

We are going to navigate to our +Dashboards -> Add Dashboard

curl https://raw.githubusercontent.com/kyverno/grafana-dashboard/master/grafana/dashboard.json -o kyverno-dashboard.json

Once the dashboard is up let’s generate some noise on violations by doing the same YAML manifest from earlier and keep running the kubectl run command with a image defined and no label.

I’ve navigated back to Dashboards > Kubernetes/Compute Resources/Namespace (Pods)

Back to the Kyverno Dashboard

Summary

The use of policy as code along with adding monitoring to grafana with prometheus will route to your Security Operations to be early indicators of audit trails to follow if anything should occur from a users request to the api-server. Metrics and visualizations that are able to be served to operations team gives a health metric to quickly identify who, what, when and where. Of course you could opt for native capabilities such as Azure Monitor, BigQuery in GCP. This is including the trusted prometheus and grafana stack in kubernetes.