The model is not enough. It needs a reliable system around it.

Strong in a demonstration, fragile in production

An agent can succeed in a demonstration and fail in your operations. The harness organizes what it knows, what it can do and what must be checked.

The system around the model

Instructions define the work. Context and memory provide useful information. Tools and permissions bound actions. Checks and supervision control decisions. Cost tracking helps monitor operations.

Two types of intervention

Build a new harness: start from tasks, define access and write checks before connecting tools.

Audit and correct an existing system: reproduce its failures, prioritize corrections and test them on the same tasks.

Evaluation protocol

We use the same tasks before and after corrections. Criteria are written before testing and results are recorded, including failures.

Human supervision remains defined for actions that commit the company.

Deliverables

An audit, a prioritized correction list, a rebuilt harness, a test suite and an operations guide, depending on the agreed scope.

Limits

A successful test does not prove reliability in every context. Untested cases still need evaluation. No performance measurements are published here.

Journal

1. Securing an AI coding agent starts with the shell

   Our first guard on a coding agent watched the file editor. Eight weeks of logs later, 97 percent of what the second version refused had come through shell commands. A rule written in a prompt is a request; a hook that runs before every tool call is a permission.

2. The claim a green pipeline never makes

   A pull request with no code in it. A dashboard as quiet as a slow Sunday. A test suite that stayed green because the thing it was testing never ran. Three of our own systems reported success this month, and none of them had earned the word.

3. Agents that fabricate facts: what a 1,355-row cleanup taught us

   The real risk of production AI is not a chatbot hallucinating in front of a user. It is a process silently writing plausible, false records into a database everything else trusts.

4. AI agent security in the enterprise: the rules we apply

   Deploying an AI agent means giving it access to your data. The 8 security rules we apply to every deployment, and the questions to ask any provider.