srcreigh.ca

GPT-6 Astra won't try hard unless you ask

I am sometimes disappointed by GPT-6 Astra. The model doesn’t try hard enough, even on High. To fix this, I prompt the model to loop over small details (eg conditionals in a codebase, sentences in its plan draft), analyzing them for relevant considerations. These prompts result in 20-60m turns and much higher quality output.


Last night I was using Codex CLI to design blue-green deployments for my homelab.

I gave GPT-6 Astra a list of design considerations and kubectl access. Disappointingly, even at High reasoning level, Astra failed to produce a detailed plan which integrates well with my existing infrastructure, minimizes complexity, and thoughtfully addresses my design considerations.

Even worse, if I express a concern with an aspect of its sloppy output, Astra often responds with an equal amount of more slop as a fix.

It’s hard for me to not be frustrated. I pay a lot of money for access to frontier models. Why do they make such poor and careless decisions? Are the cloud AI companies screwing me over?

In any case, the slop was unsustainable. The original plan would have added a lot of unnecessary code. I don’t like fixing bugs caused by code that doesn’t need to exist in the first place.

Why does this happen?

The model is capable of trying hard. But there is no end to the depth and breadth of effort that could be applied to a coding task. The model must therefore infer where spend its effort, and how much, based on its context.

The baseline effort seems to be pretty low, and I think this is a good thing. It takes a lot of time for the model to try harder, and I usually only want this after some interactive back-and-forth to ensure that it’s worth the effort.

How do I fix it?

For my homelab blue-green deployment planning, we weren’t really getting anywhere until I sent something like this to GPT-6 Astra High:

can you please loop on this plan, check every aspect, ask if it makes sense for the codebase, ask if anything can be removed or is not well specified, ask if it doesn’t meet my criteria, spent at least 10 minutes optimizing the plan.

This produced a better result but still not good enough, so I prompted again.

Please loop more on this plan, think through the consequences of it, add more details to ensure that everything you mention is actually fully addressed. Check every single aspect of your plan, ensure that you explicitly write down what might break as a result of such change, determine a fix for those issues, and then minimize.

When I was trying to get Astra to minimize the attack surface in Kata Containers, I did something similar (using Ultra, not High).

could you please look through the runtime and agent code, check every branch and determine if it’s needed for Kubernetes Firecracker workloads. Carefully read through the code, check every conditional if it’s needed for Kube/FC, remove any which are ineffective. Thank you