Image Signing Intro
Image signing is a native feature of Oracle Cloud Infrastructure and this is a feature that can ensure your cluster doesn’t deploy images that aren’t signed. For instance your developers design a container image and want to push upstream to your cluster, but prior to allowing this the image has to be signed to verify authenticity. This stops a number of headaches in kubernetes such as deploying vulnerable images inadvertently or unsanctioned images. Most cloud service providers have native features such as Azure Gate Keeper which extends Open Policy Agent that can restrict images to specific repositories (such as private).
How does this break down? Architecturally a cluster administrator can set policies that are followed by the cluster operators and verifying images authenticity ensures no unapproved images are deployed into your cluster. In this tutorial we will walk through the feature along with the “break-glass” YAML if needed to use to bypass authorization.
What will we need for this?
- Container Registry (can use Oracle Container Registry – Private)
- Oracle Kubernetes Engine (I’ve chosen 2 nodes) you can select one.
- Oracle Key Vault (KMS) Service we will use this to sign the image with SHA-256
For creation of the registry we can navigate to to Developer Services -> Container Registry -> Create Repository
For scope if you’d like to restrict this ensure you have this in a compartment (container) that you’re able to delete after we are completed.
If you are following along ensure you create a auth-token to access programmatically the private repository (important to note – deletion of this after this tutorial).
Select My Profile -> Auth Tokens –> Generate Token – remember this token as it will only be shown once
Open the Cloud Shell we will now build a container and push it to our private registry
To connect to the cluster via the CLI we will grab our credentials so we can populate our ./kubeconfig so after you’ve created your Oracle Kubernetes Cluster select Access Cluster -> copy the text presented into your shell to gain access (We are doing this ahead of time to have this ready when we are done with our registry)
Image provided above to show the user interface if following along where the “access cluster” is located.
Now to access our private registry we will use the following
docker login sjc.ocir.io
Username: <tenancy-namespace>/<repository>
Password <auth-token>
This should be the login from the shell, you’ll have to put SJC as the annotation if you have the region that you’ve created your repository such as Phoenix this will be phx.
Let’s pull a image from docker registry run the following commands
docker pull httpd
docker images
So now we have the container image id now we are going to tag this and push it to our registry
Now we will use the container image ID and run the following to push into our registry.
docker tag <image-id> <region>.ocir.io/<tenancy-namespace>/<repo-name>:<tag> - tag can be latest/1.0/2.0 etc..
docker push <region>.ocir.io/<tenancy-namespace>/<repo-name>:<tag>
Go back to your console in the Container Registry you’ll see the following
Notice now under our registry we’ve added “latest” – this represents the httpd image we’ve pulled.
To turn on the native Image Scanning of our registry you can do this in the following screen navigate to virtualwest this will be part of our configuration of the directory.
You can move through the Remove Scanner to remove the detection recipe/configuration, however I’ve enabled it to show the vulnerabilities if any do exist.
We can see a listing grouping and hyperlinks of the image scanned and CVE’s rated by risk level.
Next we will navigate back to the image and grab the OCID ocid1.containerimage.oc1.<ocid……>
It will be long so ensure you keep this somewhere safe, we will now move to creating a vault. Identity & Security -> Vault -> Create Vault
We will need a Master Key created for this to use in the signature ensure this is RSA as AES for image signing is not supported.
The command we will use next to sign/upload the image we will need a few items
oci artifacts container image-signature sign-upload --compartment-id <compartment-ocid> --kms-key-id <key-ocid> --kms-key-version-id <key-version-ocid> --signing-algorithm <signing-algorithm> --image-id <image-ocid> For assistance on key-version and key-id see image below to assist after creation of our Master Key we then select the key that we’ve named image this will have a version ocid and ocid key.
To see the images signed you’ll go back to your repository image and the signature will now appear along with the date. So the way Oracle Kubernetes Engine works is you can deploy a “basic cluster” or a “add-on enabled” cluster think of them as two different tiers this also plays into pricing, so essentially one allows native managed add-ons that can be enhanced in the cluster such as the following image below
So now we will navigate to the Developer Services -> Oracle Kubernetes Engine -> Clusters then we will select our cluster and navigate to the Image Verification.
You’ll see that we have to select the key from our vault that we’ve created to use as the source of trust. Once this is finished it will show Enabled on our page as shown below.
Congrats if you’ve made it this far on demonstration of using various native services to Oracle Cloud.
Summary
Image signing is a pillar of guard rails you should either have from creation or enablement however, depending on your cluster set up this is one way to prevent images from being pushed that aren’t signed. Additionally ensure that the use of images are a registry that has proper access control and auditing and logging enabled. I couldn’t tell you how many times I’ve seen audit and logging overlooked in lieu of not adding costs and have to severely have issues later on because of this decision.