Secure VMs for Kubernetes: Hardening Kata Containers

Yesterday I used Astra to delete ~60% of Kata Containers code while maintaining full support for using it to run x86_64 Kubernetes workloads in the Firecracker VM. My goal was to reduce the attack surface as much as possible in agent-runtime Kata code, and make such code more easily auditable.

https://github.com/srcreigh/kata-containers

This post is completely hand-written. All of the changes to the repo were via Codex CLI.

Following Astra’s edits, the resulting codebase has 13.5k SLOC for the host runtime, and 8.1k SLOC for the agent. The changes are not purely deletion; there are quite a few fixes for issues surfaced by deleting code.

The changes were verified against my homelab cluster where I run a handful of untrusted Kubernetes workloads in Firecracker via Kata. This caught many issues.

Use at your own risk. I am not a security professional. I have not audited the code myself. I am not committed to keeping the fork updated with Kata upstream. I offer no guarantees. There are no setup instructions. It has not been tested in a real production setting.

The Kata project is actively maintained by some genuinely kind folks. I have had great experiences contributing to the project in the past. I intend to upstream as much as possible, but since I have very high standard for code I expect others to review, it won’t happen soon or ever.

Why?

AFAIK, there is no project available with the express goal of providing minimal attack surface Kubernetes-Firecracker integration. Other more serious projects will certainly arise to meet this need later. It’s part of the Firecracker roadmap for example.

For the time being, now there’s something.

Background

We have seen evidence lately that VMs are not, in general, a security boundary. Artem Dinaburg published experiments on Aug 26, 2026 in which GPT-5.6 Cyber broke out of qemu to Debian host 3 different times.

Firecracker is one of the only secure Linux VMs. However it is a mistake to assume that a workload running inside Firecracker is by default just as secure.

Any tool which can run Kubernetes pods inside Firecracker needs to open up new communication channels with the guest VM. For example, Kubernetes lets you copy files into a running pod. How does an isolated VM receive those files? In Kata, there is an agent which runs inside the VM and receives RPCs from the host. The host is therefore now exposed in ways which base Firecracker host would not be. The guest can now send, for example, malformed RPC responses.

The Firecracker project does not currently offer any way to run container-based workloads in it. They are working on firecracker-containerd, but without support for Kubernetes at this time.

My process

I have a 20x OpenAI subscription and wanted to use a weeks worth of credits on this yesterday.

The repo has many Astra-generated docs linked to from the README. Reports for each pass, and summary of all additive fixes since upstream.

Initial passes

At first, I set Astra Med/High on deleting Kata code which is not needed for my workloads. It had access to my homelab GitOps repo and could see which Kube features I needed and had the ability to deploy its Kata changes to my cluster and validate on real workloads.

These passes removed a lot of unneeded things, but also removed some useful Kubernetes stuff which I later decided to add back.

Some things removed in these passes: alternate VM support, the entire legacy Go runtime, some Kata features such as Kata agent runtime kernel module loading.

Removing runtime parsing code

After a few passes to remove already some 40% of the code, I did more targeted passes.

I tried to have Astra High fuzz runtime host behaviour from guest-produced malformed data. This worked for a while but I eventually ran into safety limitations, so I scrapped the whole thing.

Rather than have Astra fuzz the code, I asked Astra to make an inventory of any Kata Runtime code which parses or works with kata Agent-produced data, and then to analyze this code removing as much as possible without affecting functionality.

“Parse” here is interpreted very liberally. Much of the “parsing” was as simple as unwrapping a string from a protobuf into a Rust struct.

Here there were cases found where an agent RPC result returned data which got stored into a Kata runtime object, but then not used. Easy remove. Later passes would have the agent stop sending such data as well.

There were some cases found here where malformed agent could cause a panic in the runtime shim.

At a certain point here, Astra began trying to repro bugs, I had to ask it to stop and just delete code, not wanting to hit the safety checks again.

Re-adding functionality

At some point I decided that I was removing too much code by limiting to what my cluster needs at this time. I decided to have Astra add back some of the code which was removed. An example of code added back after being removed is Kubernetes volumeDevices which Kata can use to mount a host block device into Firecracker.

Ultra - line by line auditing

One of my last passes was with Astra Ultra where I gave it a broad goal: audit every line of code in the runtime / agent, analyzing whether its needed to run x86_64 Kubernetes workloads in Firecracker, removing anything unnecessary.

It ran for I believe 55 minutes, removing a lot more code.

One of the interesting issues it found was a networking init deadlock. There was a synchronous thread yield from an async context, after removing some init code, the thread now was being launched prior to some of its needed state. Astra’s explanation here.