Anurag’s Read

AI coding has made CI a bottleneck, so we reworked ours to keep up.

The post lists many CI fixes with before and after times, from lighter checkouts to shared test state. A product engineer can copy the habit of measuring the critical path before changing anything.
Score
8.6
Verdict
Must read
Blog and date
Linear,
Original length and level
10 min, intermediate
Topics
ci, developer productivity, testing, cost optimization
Read the original

Context

An issue from the CTO said CI costs were high and CI was slow. AI coding raised test volume, and the test suites nearly quadrupled in 2026. The team tracked PR wait time and runner time.

Architecture

A PR starts gating jobs that run change detection and cache checks. Eight API test shards wait for those jobs. Each shard sets up a base image, installs packages, prepares the database, and runs Vitest files.

Cache markers are written after the shards, in a job that gates nothing. Third party runners with faster CPUs and storage run all jobs.

Mechanisms

M1

Lint and type checks

The team moved to the native compiler tsgo, and weekly median tsc time fell 73%. It rewrote type-aware ESLint rules as syntax only AST analysis. API lint time fell 68%.

M2

Gating jobs

Change detection capped fetch depth, and the slowest gate fell from 94 to 20 seconds. A custom checkout action retries with backoff and aborts stalled connections after about 30 seconds.

M3

Setup cost

A shared base image preinstalls Node and the Postgres client. A filtered pnpm install cut install time from 44-73 to 16-18 seconds. With an unchanged schema, containers load a schema snapshot, and database setup fell from 12 to 2 seconds.

M4

Test sharing

Vitest splits work by file, so the team split large files. An opt-in Vitest project with isolate false lets safe files share a module registry. Files with fake timers stay isolated.

Design decisions

No.ChoseInstead ofWhy
D1Cut setup cost before adding shardsFour shards with 110 to 140 seconds of setup eachDoubling shards doubles setup, but with setup near 40 seconds eight shards use less setup than four did.
D2Remove the node_modules cacheA cache keyed on the lockfileA hit took 28 seconds, while a filtered install took about 7.5.

Key numbers

  • PR wait fell from over 6 minutes to just over 5.
  • Runner time per test roughly halved.
  • Batching seven checks into two jobs saved about 87,000 runner minutes per month.
  • Shared module state cut the slowest shard from 300-379 to about 195 seconds.

Failures and limits

  • The link from the third party runners to GitHub degraded at times, and checkouts hung.
  • Shared module state is the riskiest change, and some files cannot convert.

Lessons you can use

  • Measure setup cost per shard before you add shards.
  • Move work off the critical path when it gates nothing.

Glossary

ShardOne slice of a test suite that runs on its own runner.