When Agents Write the Code, the Tooling Trade-Offs Flip

TL;DR: Go packages in our repository doubled in 12 weeks, and the checks an engineer waits on before merging still finish in about three minutes. Here is what Bazel fixed, what our automated rules fixed, and what still takes time.

Most of the code in the repository behind Anchorage Digital Agentic Banking is written by AI agents. The repository is dedicated to that product and sits apart from the rest of Anchorage Digital. Over 12 weeks, the number of Go packages grew from 188 to 366. The full set of automated checks an engineer waits on before merging still finishes in about three minutes.

Bazel helped, but it was not the whole answer. We also kept generated files out of the repository, prevented long chains of dependencies between packages, kept most tests away from external services, and made the build reject recurring problems we had previously caught in review. A few years ago, maintaining all of that would have consumed too much of a small team's time. Now agents handle much of the routine work: updating the files that tell Bazel how to build each package, connecting code generators to the build, and keeping automated checks current.

On most pull requests, the slowest part is not the time it takes to run the tests themselves because very few tests run on pull requests, only the necessary ones. Every run of our automated build-and-test system, or CI, gets a newly started virtual machine, a clean computer in the cloud. Before any test can run, Bazel examines every backend package and works out what might need rebuilding. Waiting for the machine and completing that preparation now takes longer than most of the tests.

Why Bazel Became Worth It

In 2022, rejecting Bazel was the right call for us. Its map of how packages depend on one another becomes more useful as a repository grows, but someone has to keep that map current, fix code that assumes it can reach any file or network service on the machine, and pin compilers and other tools to exact versions. Previously, our small team had a small codebase, and in that codebase, the native go toolchain and scripts worked well enough. Now a small team can have a large codebase generated by agents, so the native toolchain frequently proves insufficient. 

Once agents began writing most of the code, the old trade-off changed. The repository grew faster, but agents also took over much of Bazel's routine upkeep. We revisited tools we had rejected for their maintenance burden and asked one question: could this help an agent take a feature from first edit to merge without relying on a human to catch a bug?

At that point, Bazel was worth it. It gave us one reliable way to reuse results across builds, tests, and code checks. Plain go build plus scripts did not. We also considered Nx, Pants, and Buck2, but none supported Go as well as Bazel.

Repository size now carries a direct cost. Before Bazel runs a test, it reads the description of every backend package and decides what work might be required. Bazel calls this preparation "analysis," and CI repeats it on every new machine. Bazel can reuse completed build and test results, but not this preparation, so a larger repository slows down even a tiny change. In response, we delete dead code, keep generated files out of the repository, limit the amount of JavaScript sent to the browser, and prevent long chains of package dependencies.

Put the Rules in the Build

We use Bazel for more than compiling code. It also decides when a test result can be safely reused, keeps generated files out of Git, and rejects changes that break rules we used to enforce in review.

Run every test on every push

Every CI run asks Bazel to consider the whole backend test suite. We do not first guess which tests might be affected by the files that changed. Bazel reuses a previous test result unless the code, configuration, or tools that produced it have changed.

On a typical push, only two of roughly 400 test packages actually run again.

Make it safe to reuse old results

This works only if an old result could not have been influenced by something Bazel did not know about. Each build or test step runs in a restricted environment with no network access by default. We use exact, verified versions of compilers and other tools, rerun a test when its environment changes, and fail the build if the files that record exact dependency versions, called lockfiles, are out of date. Skipping a step is safe because that step could only ever see the files and tools Bazel recorded.

The main constraint is that everything builds in pure Go, with cgo turned off. cgo lets Go call C code. Turning it on would make builds depend on the C compiler and system libraries installed on whichever machine happened to run them.

The same setup keeps generated code out of the repository. Today that’s the HTML templates rendered by the server. Bazel creates them during the build and reuses the output until an input changes, so the generated files exist only in a temporary build directory. Committing them would bloat the repository and its history, slowing Git clones, fetches, and checkouts without helping the build.

Keep the frontend and backend in one build

The stack uses two languages: Go for the backend, and TypeScript for infrastructure and the frontend. Every additional language brings another set of tools to lock down, more code maintained outside our team, and another set of conventions for an agent to follow.

Go was our first choice. Agents produce Go that the team can review quickly, and the compiler catches many mistakes before the program runs. For this backend, TypeScript would allow more ways to express the same idea and catch fewer mistakes before startup, while Rust would make each cycle of editing, building, and testing slower.

Pulumi lets us define infrastructure in TypeScript. Terraform uses HCL, a language designed for configuration. We suspect agents are better at ordinary TypeScript than at HCL, but we have not measured it.

The server returns finished HTML, and HTMX adds small interactions in the browser. We do not run a React-style application there. The templates become Go code during the build, so the frontend and backend share the same build, the same saved test results, and the same automated checks. Component tests render each component in Go and inspect the HTML. A React application would require the frontend and backend to agree on the shape of every JSON message they exchange, and it would introduce errors the Go compiler cannot see.

HTMX handles most browser interactions, but we still use some JavaScript. We send about four kilobytes of our own JavaScript to the browser, and an automated check fails if it grows beyond that limit. Small interaction libraries ship alongside it. The benefit is having one build instead of two, not eliminating JavaScript.

NPM packages remain a risk because they bring code maintained outside our team into the build. Package installs on developer workstations route through Socket instead of going straight to NPM, and CI downloads dependencies through an internal cache instead of going straight to npmjs.com.

Turn repeated review comments into automated checks

We do not want the same problem to depend on a reviewer noticing it twice. If we write the same review comment a second time, we try to write an automated check that finds the problem in code.

nogo lets Bazel run Go code checks while it compiles each package. Local builds and CI therefore run the same checks. Running the same checks as a separate step would spend roughly 72 seconds rebuilding information about the code that Bazel already has.

nogo does not include everything we need by default, so we added roughly 150 checks that inspect code without running it. It also has real limitations. We cannot add a comment that disables one rule for one line, so exceptions apply to whole files or folders, and it reports failures one package at a time rather than showing all of them together.

We also wrote 14 checks of our own. They replaced goarchlint, the tool we had used to enforce which parts of the code could depend on which others. It was too slow to run after every edit and could no longer see generated code once we stopped committing those files. Some of the new checks enforce dependency boundaries that Bazel's built-in rules cannot express, especially where production and test code live in the same package. Others forbid an old pattern once we introduce a safer replacement.

A few of these checks came from failures that did not look like code-style problems at first:

We apply the same idea to tests that keep implementations and configuration in agreement. They verify that the browser-test environment wires up its services the same way production does, with stand-ins for secrets, external services, and the compliance checks; that feature flags match the Kubernetes deployment configuration; and that the public API contains only approved operations.

Automated checks need maintenance too. We found that our JavaScript size check was using an outdated list of files, so it was no longer measuring all the JavaScript we meant to limit. The check saved reviewers from doing that work by hand, but it still had to be updated as the code changed.

Make Slow Tests Hard to Write

Unit tests are fast by default. The more important goal is to stop them from quietly becoming slow. Tests use working in-memory replacements for roughly 17 data stores instead of a live database. The compiler verifies that each replacement provides the same methods as the real store, and none of them makes network calls. Tests also use Go's synctest package to simulate the passage of time, so a test can model seven seconds of waiting without actually waiting seven seconds.

We also make slow dependencies hard to introduce. The Spanner emulator package, a local stand-in for the database, is available only to a short list of approved packages. If an unrelated test tries to depend on it, Bazel stops the build during compilation.

An in-memory repository can behave differently from its Spanner-backed implementation. To catch that, each repository has one shared set of test cases that runs against both implementations, and both must produce the same results. Tests that only make sense for one implementation stay in its own package. That shared set is what lets the rest of the test suite use the faster in-memory implementations with confidence.

On a developer's machine, these comparison tests run only when a Spanner emulator is already running, so a normal bazel test stays fast. In CI, skipping them is an error, so they cannot silently stop running. We use two database stand-ins: the standard emulator, which takes about two minutes, and a stricter test database that enforces more of Spanner's SQL rules. A pull request can turn on the stricter version with a label.

Browser-level tests use the same in-memory stores. About 95 browser test files written with Playwright run in 16 processes against one backend that is already running and is never reset. The tests do not wait for a database or spend time resetting the system before and after each case.

That speed has a cost. Each test must avoid using the same data as another test running at the same time. We use unique IDs, separate keys for shared resources, and assertions that look at what a test changed rather than assuming the database started empty. Nothing currently prevents a new test from accidentally colliding with another test's data.

Where the Three Minutes Go

Because Bazel reuses most previous results and we keep dependency chains short, the tests themselves no longer take most of the time. The roughly three minutes break down like this:

• Getting a machine ready: more than a minute, about half the total, goes to assigning a virtual machine and downloading the software image that contains the test environment.
• Bazel's preparation step: most of the remaining build-and-test time. Bazel examines the full package map even when almost nothing needs to run. On a 16-core workstation, this preparation took about 60 percent of one complete Bazel run.
• Work that actually runs again: roughly one backend test package in 300, plus about half a minute of browser tests.

The preparation time grows with the repository, no matter how many previous results Bazel can reuse. That is why we keep the repository small and dependency chains short. Many independent packages can be built at the same time, while a long chain forces each package to wait for the one before it. During the 12 weeks when the package count grew from 188 to 366, our measure of the longest dependency chain moved from 47 to 46. Our rules help preserve that shape, but they cannot guarantee it. A new long chain would still slow the build.

Each CI run gets a fresh virtual machine, with no Bazel process or in-memory information left over from the previous run. We save some time by running the build, tests, and code checks in one Bazel command, reusing stored build and test results in both CI and local development, and downloading the software image ahead of time. More than a minute is still spent assigning the machine and getting it ready, none of it caused by the code being tested.

We looked at the obvious fixes:

• First calculate which packages a change could affect. We measured this, and it was slower. Bazel's saved results already select the work more precisely, so the expected saving came to about 1.6 seconds per pull request.
• Send build and test work to a pool of other machines. We have not measured this one. It would help only with work that actually runs, which is not where most of our time goes, and letting Bazel run as many as 600 tasks at once instead of 32 changed nothing. That argues against the approach, but it is not a test of it. It gets its first real measurement when we evaluate the next CI system.

Our next experiment is to reuse CI machines between runs instead of creating a new one every time. That would remove most of the wait for a machine and the repeated software download, and it may let Bazel keep some of its preparation in memory. We think this could bring the checks down to about a minute, but we have not tried it yet.

The Part Worth Copying

Most teams should not copy this exact stack. The part worth copying is simple: revisit tools you rejected because they took too much work to maintain. Agents can change that old trade-off.

They do not make the maintenance disappear. Dependency chains still need to stay short, automated checks can fall out of date, and fresh CI machines still spend more than a minute getting ready to do useful work.

We designed these rules for a repository 10 times this size. We expect the checks to still finish in about three minutes at that scale, but we have not run at that scale yet. CI is not our bottleneck today, so we are not rushing to optimize it further. Reviewing agent-written code now takes longer than CI does, and that is where we will focus next.

‍