How a Complex Agent Improved Effective Delivery Efficiency by 96.3×

A prototyping Agent is a representative complex-task system in generative AI. Its job is to turn a natural-language request into a complete, interactive, runnable product prototype.

That journey spans requirement interpretation, solution design, page decomposition, parallel implementation, integrated validation, and iterative repair. Together, these steps form a dynamic task graph. As prototype complexity grows, dependencies between nodes multiply: a mismatched component, a visual inconsistency, or a broken interaction on one page can trigger several rounds of cascading repair. A wrong global design decision can invalidate the entire graph.

Optimizing such a system requires three architectural boundaries:

1. Outcome boundary: what result counts as genuinely complete;
2. Reasoning boundary: which state transitions deserve nondeterministic model reasoning;
3. Control boundary: how deterministic computation and rule-based workflows should execute efficiently.

This article traces several months of architectural evolution. The system moved from open-ended ReAct exploration toward a more efficient design built around contracts, deterministic workflows, and an optimized context topology.

1. The failure mode: when post-hoc review becomes a cost and latency sink

The early architecture used ReAct to drive the entire generation lifecycle. The Agent reasoned about the next action from the current state, called tools to write or validate code, and used the feedback to decide whether to continue, repair, or stop.

To protect the quality floor, the system accumulated several review layers inside that loop. Every generated page entered a mandatory review step. A detected defect triggered a repair, followed by another review.

graph LR
    A[Implement page] --> B[Review]
    B -- Issue found --> C[Repair]
    C --> B
    B -- Pass --> D[Deliver]

This architecture intercepted low-quality output before delivery, but its dependence on post-hoc review created significant system-level side effects:

• An expanding compute chain: every review reloaded context, invoked the model, and interacted with tools. A repair could also rewrite a correct module and create another review cycle.
• Severe tail latency: on complex tasks, model cost frequently exceeded $20 while end-to-end time passed 60 minutes.
• Lower predictability: even a simple task could enter repeated reviews because of evaluation variance, making latency commitments unreliable.

In a representative sample with a consistent complexity distribution, the review-heavy version developed a pronounced long tail:

The numbers reveal a counterintuitive result. More review did raise the quality floor, but it did not improve effective delivery capacity. Instead, the system spent progressively more money for progressively less predictable completion times.

The optimization target therefore shifted from maximizing success rate in isolation to delivering engineering-grade results under controlled cost and latency constraints.

2. A better metric: effective delivery efficiency

The system introduced effective delivery efficiency as its primary engineering metric.

Traditional throughput counts processed requests. Effective throughput counts requests that satisfy a service-quality target. For an Agent, one user request can contain dozens of model calls and several internal repair attempts, so the correct unit of evaluation is the complete delivered result.

The metric is defined as:

In practical terms, it asks: how many qualified prototypes can the system deliver within its promised time for every $100 of model compute?

Both sides of the ratio are constrained:

• Numerator: only tasks that meet the product-completeness standard and finish inside the end-to-end SLA;
• Denominator: all compute expenditure, including tokens, tool execution, intermediate reviews, retries, and resources consumed by failed or cancelled tasks.

The metric also sets the order of optimization. Quality must first reach the delivery threshold. Latency must then fit inside a reasonable service window. Only after those conditions hold should the system minimize compute cost.

For comparison across stages, the following figures normalize against the same task structure. Each stage uses 40 representative tasks—200 samples in total—covering two to six pages, with an eight-minute delivery SLA. Cost includes failures, pre-cancellation consumption, and retries. Historical versions are estimated from observed behavior and replays on the same task set; the current version is calibrated against staging traces. The figures express an architectural trend, not audited production finance.

The turning point was not a cheaper model. The system first removed unnecessary reasoning, then ran the remaining reasoning with greater parallelism and a higher cache hit rate. May became the efficiency low point despite its stronger review floor, proving that raising quality in isolation does not make the overall system more effective.

3. Architecture stage one: move planning forward and concentrate uncertainty

The new metric pushed the system to compress uncertainty before implementation began.

A dedicated planning node was introduced ahead of execution. It established three system-level contracts:

1. Task decomposition contract: the page tree and boundaries of the product;
2. Shared constraint contract: global visual rules, shared components, and data-flow patterns;
3. Acceptance contract: objective conditions for declaring the task complete.

graph TD
    subgraph OLD[Old architecture]
        O1[Page A interprets the full request] --> O2[Implement] --> O3[Global review finds drift] --> O4[Rebuild]
    end

    subgraph NEW[New architecture]
        N1[Planning node creates global contract] --> N2[Implement page A within boundary]
        N1 --> N3[Implement page B within boundary]
        N1 --> N4[Implement page C within boundary]
        N2 --> N5[Deterministic acceptance gate]
        N3 --> N5
        N4 --> N5
    end

Previously, every page-generation node independently interpreted the entire product. That repeated decisions and encouraged drift. The planning node moved global design decisions forward, allowing downstream tasks to focus on local implementation inside explicit boundaries.

Post-hoc review was then replaced by deterministic gates and localized repair. The system checked output against the delivery contract defined in advance instead of launching an undifferentiated chain of model reviews.

4. Architecture stage two: safe parallelism and context topology

Once the global contract was stable, the execution graph could safely exploit page-level parallelism and shared-context caching.

4.1 Contract-backed parallel execution

Page tasks share the same product facts after planning, so they can proceed independently in parallel.

• Serial time: \( T_{\text{serial}} \approx T_{\text{plan}} + \sum T_{\text{page}} + T_{\text{gate}} \)
• Parallel time: \( T_{\text{parallel}} \approx T_{\text{plan}} + \max(T_{\text{page}}) + T_{\text{gate}} \)

End-to-end time no longer grows with the sum of page execution times. It is governed by the slowest path.

The benefit becomes clear when results are grouped by page count:

The old architecture’s time grew almost linearly with page count. In the new architecture, additional pages mostly increase concurrency width rather than critical-path length. Complete-delivery rate does not fall, showing that planning has moved the quality floor formerly enforced by final review to the start of execution.

4.2 Tree-shaped context and prefix caching

Parallel page branches share a large amount of context: global requirements, product structure, and visual constraints.

The new architecture separates that input into a global static prefix and a local dynamic task.

flowchart TB
    P["Global static prefix<br/>requirements · structure · visual rules · conventions"]
    P -->|Shared prefix cache| A["Page A<br/>local task"]
    P -->|Shared prefix cache| B["Page B<br/>local task"]
    P -->|Shared prefix cache| C["Page C<br/>local task"]

The inference engine stores the global prefix as a KV cache that every parallel branch can reuse.

The more complex the prototype and the more pages it contains, the larger the relative benefit. Once the first-write cost is amortized across several branches, the cost of retrieving repeated context falls by more than 80% on complex prototypes.

The “more than 80%” figure describes repeated-context retrieval on complex tasks, not the total task cost. Each page still requires unique implementation output, and tool execution and gates do not disappear. Even so, context topology alone reduces full-task cost by 56% for five- to six-page prototypes.

5. Architecture stage three: from open ReAct to a bounded workflow

After separating task boundaries and reducing context cost, the system addressed the unpredictable execution length of the ReAct loop.

ReAct is useful for exploring unknown space. Once the objective and acceptance criteria are explicit, however, allowing the model to decide whether review should continue adds execution entropy.

The new architecture removes deterministic execution from ReAct and places it in a state-machine workflow.

flowchart LR
    P["Plan<br/>Agent exploration"] --> E["Parallel branches<br/>workflow control"]
    E --> G["Contract gate<br/>rule validation"]
    G -->|Pass| D[Deliver]
    G -->|Fail| R["Exception repair<br/>local Agent"]
    R --> G

In this control flow:

• the workflow owns concurrency, timeouts, and retry budgets;
• the gate produces objective validation signals;
• the Agent concentrates on open decisions during planning and bounded local repair after a failed gate.

When the gate passes, the workflow stops immediately and prevents needless optimization loops. When it fails, repair context is limited to the failing module instead of triggering a global rerun.

The result is a much tighter latency tail. In the current standardized sample of successful deliveries, minimum end-to-end time is 2.1 minutes, P50 is 4.1 minutes, P95 is 7.6 minutes, and the maximum is 8.0 minutes. Simple tasks finish in roughly two minutes, while complex tasks remain inside the eight-minute SLA.

The cost tail contracts with the control flow. A simple prototype can now cost as little as $0.60. Cost still rises with page count and interaction complexity, but unbounded review can no longer amplify it indefinitely.

More importantly, eight minutes is not merely a good experimental observation. It is a designed boundary produced by branch concurrency, retry budgets, localized repair scope, and explicit termination conditions.

6. Architectural reflection: optimize the task graph itself

Several prominent approaches to Agent efficiency focus on model allocation inside a given task graph:

• Anthropic recommends starting with the simplest possible architecture and adding complexity only when needed;
• OpenAI recommends establishing a quality baseline before trying smaller models;
• research such as EvoRoute uses dynamic routing to match models to individual steps;
• AI21 Maestro searches the configuration space of collaborating Agents.

This system’s evolution focuses instead on restructuring the task graph itself.

flowchart LR
    subgraph ROUTE["Model-routing optimization"]
        R1["Given task graph"] --> R2["Select models"] --> R3["Optimize routing"]
    end
    subgraph SYSTEM["This system's optimization"]
        S1["Restructure task graph"] --> S2["Plan first"] --> S3["Contract decomposition"] --> S4["Bound the control flow"]
    end

Model routing decides which engine performs the work. Planning constraints, deterministic workflows, acceptance gates, and a prefix-cache topology reduce how much unnecessary reasoning occurs in the first place.

7. Conclusion

The architecture evolved through three stages:

1. Open exploration: rely on review to protect the quality floor;
2. Structural decomposition: concentrate uncertainty in planning, then use contracts, parallelism, and prefix caching to optimize the compute topology;
3. Control convergence: replace an open ReAct loop with a deterministic workflow that enforces explicit cost and latency bounds.

Using the stable May version as the baseline, and holding quality and SLA requirements constant, qualified prototypes delivered per $100 rose from roughly 1.03 to 99.43—an improvement of about 96.3×. More concretely, tasks that frequently cost more than $20 under the review-heavy framework can now cost about $0.60 in simple scenarios.

Complex Agent optimization follows three principles:

The core responsibility of the architecture is to decide, before invoking a model, whether that reasoning call needs to happen at all.