Introducing AAT!

AAT is open source today. It is a Go CLI that models an API as a graph once and then runs long, multi-step test plans against it: the data wired between steps, one plan multiplied into a matrix, every exchange recorded, and the same graph handed to your AI coding tools over MCP. v0.2.0 is the launch release.

That is seven plans crossed with two layer groups. Sixty-three runs; thirty-six of them would have sent requests identical to another run's, so they are skipped; twenty-seven execute, four at a time. One command, and nothing copied.

Sixty seconds

No signup, no account, no network:

brew install gburgyan/tap/aat      # or a release archive, Docker, or go install
aat-sandbox init shop && cd shop   # the offline demo project
aat-sandbox serve &                # shop API on :8765, payments API on :8766
aat run plan full-lifecycle        # one order through every state

  [ 1/15] listProducts         200  0ms
  [ 2/15] checkInventory       200  609ms  retried 1x: response_error
  [ 3/15] createCart           201  0ms
  [ 4/15] addProduct (addItem) 201  0ms
  ...
  [ 8/15] checkout             201  0ms
          Order: ord_0001
          Total: $130.66
  [ 9/15] paymentCharge        201  351ms
  [10/15] shipOrder            201  601ms
  [11/15] getShipment          200  1.3s  retried 2x: transient
  ...
  [15/15] verify_getOrder      200  0ms

  cleanup:
    deleteOrder            204  0ms
    deleteCart             204  0ms

PASSED (15/15 steps, 2.9s)

Fifteen calls, two steps that retry by error category, a verification pass over the result, and the order and the cart deleted at the end. Nobody wired the data by hand: the graph says where every input comes from, and the plan only lists steps. aat web view latest opens the same run as a timeline you can click through.

Three files

Everything you know about an API splits three ways, and AAT keeps the three apart.

API knowledge is a graph of operations and request templates, written once. A node says what an operation takes, what it returns, where each input's value comes from, what must run before it, and what undoes it:

checkoutCart:
  adapter: checkoutCart
  inputs:
    - name: cartId
      default: {from: createCart.cartId}    # wired from an earlier step
    - name: shippingTier
      type: enum[standard, express, overnight]
      default: standard
  outputs:
    - name: orderId
  cleanup: deleteOrder                      # undone after the run
  requires: [cartPopulated]
  satisfies: [orderCreated]

Test intent is a plan that lists steps, not wiring. This is a whole test:

kind: recipe
selection:
  workflow: Checkout
  choices: {customer: Registered, payment: PayPal}
  addons: [Apply Coupon, Return After Delivery]

Variation is layers and environments. A layer is a named set of input values, each --layer-group adds a dimension, and a permutation that would repeat another run's requests is skipped instead of run twice.

Small files, reviewed like code, diffed in a pull request, each one small enough to read in a sitting. The point is not that the files are tidy. It is that they run.

What you get today

• Long chains that clean up after themselves. Values flow between steps, retries follow error categories, verification runs after the flow, and cleanup unwinds what was created — in reverse, with when guards and release tracking.
• A matrix from one plan. --layer-group crosses layers into permutations, duplicates are skipped, and --parallel runs the rest at once.
• Archives worth sending. Every request, response, resolved value, retry, and assertion, with secrets redacted, exported as a single file the other team opens in the same viewer. Better evidence than a screenshot of one pane.
• An MCP server. aat mcp serve hands an AI coding tool the same graph the engine runs: each operation's exact request, the order calls go in, what each needs from the calls before it, the composed flows, and sample responses from real runs. With that much detail a working client has taken a single prompt; the shop's version of that test is reproducible, with the exact prompt and the results of a Python run and a Go run.
• Environments that share a base. extends and vars keep regions honest, a single operation can route to another host with its own credentials, and --override points one call at the build on your laptop.
• Answers before a request is sent. Unknown keys are errors that name the line and the nearest valid key. aat validate --strict catches broken wiring, a plan that names an output nothing produces, and a template that disagrees with the spec; --oas-validate then holds every request and response to the OpenAPI document as the run goes.
• CI-native. Exit codes 0/1/2/130, --json, --quiet, JUnit XML, and a Docker image. --stop-after and --dump-state hand a live, half-finished run to whatever comes next in your pipeline.
• A web UI in the binary. Runs as a Gantt timeline, steps with Copy as cURL, batches as a matrix.

The proof is that it runs

Three complete projects run AAT against real, public APIs in their test modes. Every number in their READMEs is something a run recorded:

Project	Scale	What it shows
aat-duffel	66 operations, 47 plans, 14 layers; 47/47 in ~3½ min	Flight search and booking against an API with no official OpenAPI spec
aat-stripe	82 operations, 53 plans, 14 layers; 53/53 in ~5 min	Card and bank payments, with every exchange checked against Stripe's 205,000-line spec
aat-shippo	46 of 70 operations, 28 plans, 9 layers; 28/28 in ~2½ min	Rating, buying, refunding, and tracking shipments; layers as the headline; real labels in the web UI

Clone one, export a free test-mode key, and it runs against your own account. Each is also a worked integration: the graph and its domain file are a reference whose every claim has a run behind it, and a .mcp.json hands the whole thing to a coding assistant.

They also settle the question of whether small files are enough. Stripe's own OpenAPI document is 205,445 lines. The project that runs 53 plans against it is about 6,300 lines of graph and templates, and the median request template is 28 lines. An assistant — or a reviewer — opens the one operation the task needs.

Before those three were built, a different question got asked: can an agent build such a project from an API's public documentation alone, with no help? It can. In clean rooms, from one prompt each, coding agents built a Duffel project twice and a Stripe project three times; the three Stripe builds took about half an hour each. Every attempt validated clean and passed every plan it wrote; the Duffel runs handled 13 of the 14 flows asked for unaided, the Stripe runs 13 of 13.

And running against a spec finds what reading it does not. Stripe's spec does not describe one of the responses its own API sends; Shippo's marks nothing as nullable, and the API sends null constantly. In both projects the finding is pinned by a plan, reported rather than fatal, and counted on every run — with the archive to attach when you report it upstream.

Where it came from

AAT started as a pile of Postman collections.

They worked, at first. Then the team grew. Everyone had their own copy with their own tweaks, and none of them were reliable. Nothing was in source control, so there was no diff, no review, and no way to tell whose version was right. Every new test case meant editing a collection in place, so the case it replaced was gone. The chaining lived in pre-request scripts, which put the interesting part of a flow — what depends on what — inside JavaScript instead of in front of you. And none of it was legible to an AI coding tool: the export was one file too large to read, in a shape nothing else consumes.

The underlying problem is that real integrations are not one call. Buying something means browse, cart, checkout, pay, ship, and maybe return and refund: 8 to 20 calls, each needing IDs from the calls before it, leaving state behind that someone has to clean up. Postman is good at what it is for — exploring an API by hand, one request at a time — but it is a poor place to keep the knowledge of how an API works. That knowledge ends up in a format only Postman reads, in a workspace rather than your repository, and it scales by copying. So the knowledge moved into the repository, and AAT is what it moved into. AAT was built and proven against a large private API in daily use, and then against three public ones. Why AAT exists tells that story the long way.

The other half is the loop. An agent is only as good as its feedback: give it a validator that names the line, a real run against a sandbox, and a precise error, and it edits one small file and tries again. Without that, it guesses. Everything above is built so the feedback arrives fast and says something specific — which is also why the clean rooms worked.

What it isn't

LLMs are optional and authoring-time only: aat prompt can draft a plan, and the MCP server teaches AI tools your API. Execution never calls an LLM. The loop needs an API it can actually call, so a sandbox or a test mode. There is no Postman importer; the on-ramps are aat generate --oas from a spec, or an assistant reading a collection through a reader and closing the loop with a validator and a run. It is pre-1.0 with one maintainer, and the formats may still change.

Get it

brew install gburgyan/tap/aat        # or a release archive, Docker, go install
aat-sandbox init shop && cd shop     # the offline demo project
aat-sandbox serve &
aat run plan full-lifecycle          # one order through every state, verified and cleaned up
aat web view latest

The docs are at gburgyan.github.io/aat, the code at github.com/gburgyan/aat, and the three projects are linked above.

I made this tool so working with APIs sucks less.