Kubernetes 1.29.0 Security Enhancements

Introduction

Kubernetes 1.29.0 is now released this marks a significant milestone in the evolution of container orchestration, introducing robust security enhancements that cater to the ever-evolving demands of modern infrastructure management. This latest release encapsulates Kubernetes’ commitment to providing a secure, scalable, and highly efficient platform for deploying and managing containerized applications. With a focus on strengthening security, Kubernetes 1.29.0 introduces new features and improvements that address critical vulnerabilities and enhance overall system resilience.

Enhancements

I’m running Kubernetes 1.29.0 on a kind cluster using killercoda for quick installation I will also upload the configuration file that you can leverage if you’d want to follow along.

Changes include a few details primarily its built with Go 1.21.2 and the kube-apiserver exposes four new metrics about errors on the clusterIP and nodePort allocation logic.

Security Findings

Notably kubeadm (if self-hosted) as a cluster will have a separate file “super-admin.conf”. The User in “admin.conf” is now going to be bound to kubeadm:cluster-admins that will have cluster-admin, ClusterRole. Moving forward the break-glass account in “super-admin.conf” will have the highest privileges bound to system:masters.

Adding ImageMaximumGCAge this field being added to the kubelet configuration will allow the user to set maximujm age of an image being unused, I could seeing this being highly encouraged on security posture management systems to have a control that does this natively. The big news at least from the security lens is the expansion use userNamespacesPodSecurityStandards feature gate to enable user namespace support for the Pod Security Standards. From the change log its noted this enablement modifies all Pod Security standards from the following hierarchy spec[.*].securityContext.[runAsNonRoot,runAsUser]. But also highlights to the audience this should only be enabled if all nodes in the cluster support the user namespace feature and have it enabled.

kubectl label --overwrite ns my-existing-namespace \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/enforce-version=v1.29

Provided the code above from documentation that shows overwriting a namespace policy to have enforce=restricted.

Another improvement is the continued supports of ValidatingAdmissionPolicy this expands the use of Common Expression Language, provided the snippet below to show the broad usage of the policy mind you this is relatively new since 1.28.

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicy
metadata:
  name: "demo-policy.example.com"
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
      - apiGroups:   ["*"]
        apiVersions: ["*"]
        operations:  ["CREATE", "UPDATE"]
        resources:   ["*"]
  matchConditions:
    - name: 'exclude-leases' # Each match condition must have a unique name
      expression: '!(request.resource.group == "coordination.k8s.io" && request.resource.resource == "leases")' # Match non-lease resources.
    - name: 'exclude-kubelet-requests'
      expression: '!("system:nodes" in request.userInfo.groups)' # Match requests made by non-node users.
    - name: 'rbac' # Skip RBAC requests.
      expression: 'request.resource.group != "rbac.authorization.k8s.io"'
  validations:
    - expression: "!object.metadata.name.contains('demo') || object.metadata.namespace == 'demo'"

The above example is a little more complex then a common introduction of a concept however, this is used to expression with match Conditions. These conditions are fairly straight-forward but would require some tailoring/customization for your environment but the enhancement of using these in a granular fashion is a enhancement overall for visibility and control. These can also be used to emit audit events that are created for actions you are trying to detect or would like to detect such as high request to the API server for resources (aka….costs!). Like anything I’d recommend taking a alerting approach and fine tune with also using open source tools such as Cast.AI, or Kubecost.

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicy
metadata:
  name: "demo-policy.example.com"
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
    - apiGroups:   ["apps"]
      apiVersions: ["v1"]
      operations:  ["CREATE", "UPDATE"]
      resources:   ["deployments"]
  validations:
    - key: "high-replica-count"
      expression: "object.spec.replicas > 50"
      messageExpression: "'Deployment spec.replicas set to ' + string(object.spec.replicas)"

The enhancement can also be tailored to provide more of a detailed response to the user such as the following example invoking a ReplicaLimit.

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicy
metadata:
  name: "deploy-replica-policy.example.com"
spec:
  paramKind:
    apiVersion: rules.example.com/v1
    kind: ReplicaLimit
  matchConstraints:
    resourceRules:
    - apiGroups:   ["apps"]
      apiVersions: ["v1"]
      operations:  ["CREATE", "UPDATE"]
      resources:   ["deployments"]
  validations:
  - expression: "object.spec.replicas <= params.maxReplicas"
    messageExpression: "'object.spec.replicas must be no greater than ' + string(params.maxReplicas)"
    reason: Invalid

This will produce the following when invoked.
 kubectl create deploy --image=nginx nginx --replicas=5
error: failed to create deployment: deployments.apps "nginx" is forbidden: ValidatingAdmissionPolicy 'deploy-replica-policy.example.com' with binding 'demo-binding-test.example.com' denied request: object.spec.replicas must be no greater than 3

Summary

In the context of cloud-native security, ValidatingAdmissionPolicy plays a pivotal role. It allows for in-process, declarative policy enforcement, which is more efficient and less resource-intensive than previous methods like admission control webhooks. The improvements in this update specifically in the context of security is definitely going to enhance what is also offered by the CSPs as they build the version of 1.29 to distribute/support to end users. I could see some enhancements that are traditionally full blown CRD’s becoming more accepted as being a part of the native use of kubernetes as the ease-of-use and expansion of popularity in organizations leveraging orchestration. This blog doesn’t cover all aspects of the 1.29.0 update but providing a view of security in the space this focused on key areas I’m looking into.