Directing AI Coding Agents as an Engineering Workforce: Specification, Testing and Review

Much of the public discussion about AI coding agents concerns what they can write. Luis Lacerda, an AI systems architect and founder working in LegalTech, has organised his practice around a different question: what has to be true before anything they write is allowed to ship. His answer is a discipline of specification, testing and review, and it has turned a group of AI agents into something that behaves like an engineering workforce rather than a source of plausible code.

This essay describes that discipline at the level of method. It concerns how the work is governed rather than any particular codebase.

The premise: generation is cheap, correctness is not

A modern coding agent will produce a complete module in minutes. The output is fluent, well formatted and often correct. The difficulty lies in "often". In software that routes legal enquiries, records consent and determines which firm receives which case, an error that slips through is a business error and occasionally a compliance error. Fluency does not lower that risk. If anything, it raises it, because confident code invites less scrutiny.

Lacerda's working premise follows from this. The cost of producing code has fallen sharply; the cost of being wrong has not. The scarce resource in an agent-driven practice is therefore judgement: the ability to say precisely what should exist, to establish whether it does, and to decline what does not meet the standard. He treats that judgement as his own role. The agents write; he specifies, verifies and approves.

Specification as the primary engineering artefact

In a conventional team, a specification is often a starting point that the engineers refine in conversation. With agents, the written specification carries more weight, because it is the only channel through which intent reaches the worker. An ambiguity that a human colleague would resolve over coffee becomes, for an agent, a silent decision made on the user's behalf.

Lacerda therefore writes specifications as the primary artefact of the work. A good specification in his practice states the behaviour required, the inputs and outputs, the boundaries the change may not cross and, above all, the conditions under which the result will be judged correct. Interfaces are described formally where possible; his lead-distribution software, for example, is documented through an OpenAPI specification, so that the contract between components is explicit before any implementation is written against it.

The effect is that the agent receives a bounded problem with a stated definition of done, and the reviewer later has a fixed reference against which to measure the result. The specification serves both the agent that writes the change and the person who reviews it.

Testing as an independent act

The second element is verification that does not depend on the author's account of its own work. An agent will readily report that a task is complete. Lacerda treats that report as a claim to be checked rather than a fact.

The checks are concrete. His lead-distribution platform is held to a regression suite of more than 1,000 automated API tests, which run against the specified behaviour rather than against the implementation's opinion of itself. When an agent changes the system, the suite establishes whether previously correct behaviour remains correct. When new behaviour is specified, the tests that define it are part of the work, not an afterthought.

This mirrors a principle stated elsewhere on this site: evidence before completion. Work is not finished because its author says so. It is finished when independent checks agree.

Review and approval through a pull-request workflow

The third element is human review. Every change produced by an agent arrives as a pull request. Lacerda reads it against the specification, examines what the tests do and do not cover, and either approves it, returns it with direction, or rejects it. Nothing reaches production by any other route.

His background makes this step more than a formality. Much of his recent work concerns the evaluation of language models: assembling instruction data and judging model output for correctness, including an ongoing parameter-efficient fine-tuning project on a 14-billion-parameter open model. Reviewing agent-written code is, in his view, a form of the same task. The reviewer is evaluating the output of a model, and the habits that make evaluation reliable apply directly: a fixed rubric, attention to failure modes rather than to fluency, and willingness to reject output that reads well but does not meet the standard.

Running agents as a team, not a tool

At the scale of a single change, the method is specification, test and review. At the scale of a business, Lacerda operates a fleet of more than ten coordinated AI agents with distinct responsibilities, including coding, code audit, recruiting, outreach, monitoring of team communications, search indexing and media production. Two design choices make that fleet manageable.

The first is separation of duties. An agent that audits code is not the agent that wrote it, for the same reason that an organisation separates authorship from review. The audit is more credible because it is independent.

The second is operational supervision. Scheduled routines assign recurring work, and watchdogs detect work that has stalled so that it can be recovered rather than silently abandoned. Agents also share a versioned memory: a common record of facts, decisions and lessons that each agent reads from and contributes to. Shared memory reduces repeated mistakes and keeps the fleet's behaviour consistent across sessions, much as documentation and institutional knowledge do in a human organisation.

What the method produces

The measure of the approach is what it has allowed one founder to build and operate. Under this discipline Lacerda has launched about twelve legal lead-generation sites, each with a multi-step qualifying form and Lighthouse performance scores around 99. He has built a lead-distribution platform with per-buyer validation, criteria-gated delivery and consent capture, and a vendor portal with tracked referral links and live lead status. He has also built dedicated lead validators, including one for Social Security Disability enquiries.

None of these results depends on an agent being right on the first attempt. They depend on a process in which agent output is specified in advance, verified independently and approved by a person who answers for the outcome.

Lessons for senior engineering leaders

Several observations from Lacerda's practice generalise beyond his domain.

• Write the definition of done before the work begins. Agents optimise for the request they receive. A request without acceptance criteria invites output that satisfies the letter of the task and misses its purpose.
• Make verification independent of authorship. Tests, audits and reviews should not rely on the author's description of its own work, whether the author is a person or a model.
• Treat review as evaluation. A reviewer of agent-written code is evaluating model output. Rubrics, attention to edge cases and a readiness to reject are more valuable than speed.
• Separate duties across agents. Distinct agents for writing and auditing produce more trustworthy results than one agent asked to check itself.
• Supervise the system, not only the output. Routines, watchdogs and shared memory turn a collection of capable tools into a dependable operation.

Conclusion

The arrival of capable AI agents has not removed the need for engineering judgement; it has concentrated it. In Luis Lacerda's practice the agents supply volume and speed, while specification, testing and review supply correctness. The arrangement places the human where the human is irreplaceable, at the points where intent is defined and quality is decided. It allows a small operation to build and run systems whose standards are set by people and enforced by evidence.