The Command Line: Metering and the Transcript

The loop from the last chapter runs, but it runs blind. It calls a model, spends tokens, edits files, and stops — and when it is done, you have no idea what it cost or exactly what it did. A toy can run blind. A tool people rely on cannot: it has to report what it spent, and it has to leave a record of what happened, both because a run that goes wrong needs to be inspectable and because the next chapter’s trick — replaying a run without spending a token — depends on that record existing. This chapter is the command line that wraps the loop and holds it to account.

3.1 Running the loop

Here is how the command line drives one run, from Hoist’s cli.ts.

  const meter = new Meter();
  const name = flag("session", String(Date.now()));
  const session = await SessionLog.load(join("sessions", `${name}.jsonl`));

  let provider: Provider | undefined;
  if (flag("provider", "openai") === "anthropic") {
    const apiKey = process.env["ANTHROPIC_API_KEY"];
    if (apiKey === undefined) {
      console.error("--provider anthropic needs ANTHROPIC_API_KEY");
      process.exit(2);
    }
    provider = new AnthropicProvider(
      flag("base-url", "https://api.anthropic.com"),
      flag("model", "claude-sonnet-5"),
      apiKey,
    );
  }

  const autoApprove = process.argv.includes("--yes");
  const messages = await runLoop(task, {
    baseUrl: flag("base-url", "http://127.0.0.1:8400"),
    model: flag("model", "forge"),
    cwd: flag("cwd", process.cwd()),
    ...(provider !== undefined ? { provider } : {}),
    maxTurns: Number(flag("max-turns", "24")),
    maxTokens: Number(flag("max-tokens", "1024")),
    sandbox: parseSandbox(flag("sandbox", "workspace-write")),
    session,
    approve: autoApprove ? () => Promise.resolve(true) : terminalApprove,
    onEvent: (event) => {
      if (event.kind === "usage") meter.add(event.usage);
      const line = describe(event);
      if (line !== null) console.log(line);
    },
  });

  console.log(`\n${meter.summary()}`);
  console.log(`${messages.length} messages, ${session.events.length} events in ${session.path}`);

Listing 3.1 · the run: a meter, a session log, the loop, and a summary. Every event the loop emits is fanned two ways — usage into the meter, a human line to the terminal.

The shape is small. A Meter and a SessionLog are created before the loop starts; runLoop is handed the task and its options; and when it returns, two lines print — what the run spent and where its record lives. The options carry the run’s two bounds, both defaulted: --max-turns is the loop backstop from the last chapter — the outer for that stops a runaway after twenty-four turns — and --max-tokens is the ceiling on each single response the model may generate. Neither is a running budget that halts on total spend; a coding run’s cost is bounded by its turns, not metered against a wallet. What the command line adds is not another limit but accountability, and it lives in the onEvent callback: each event the loop emits is inspected once, its usage folded into the meter, and a readable line printed. The meter and the log are fed from that one stream of events.

3.2 Metering the spend

Every call to the model comes back with a usage report — how many tokens went in as prompt, how many came out as completion. The meter’s whole job is to add those up.

export class Meter {
  requests = 0;
  promptTokens = 0;
  completionTokens = 0;

  add(usage: Usage): void {
    this.requests += 1;
    this.promptTokens += usage.promptTokens;
    this.completionTokens += usage.completionTokens;
  }

  costUsd(promptPerMillion: number, completionPerMillion: number): number {
    return (
      (this.promptTokens * promptPerMillion + this.completionTokens * completionPerMillion) /
      1_000_000
    );
  }

  summary(): string {
    return (
      `${this.requests} requests, ` +
      `${this.promptTokens} prompt tokens, ` +
      `${this.completionTokens} completion tokens`
    );
  }
}

Listing 3.2 · the meter. It sums usage across every turn and, given a price, turns the totals into dollars.

add is called once per model response, from the onEvent fan-out above. It bumps the request count and adds the turn’s prompt and completion tokens onto the running totals. Prompt and completion are kept apart on purpose: they are priced differently — a completion token typically costs several times a prompt token — so costUsd needs them separate to multiply each by its own rate. The one honest limit of this meter is worth naming: it counts the usage the provider reports. It measures the bill the model hands back, not some independent ground truth; if a provider under-reported, the meter would believe it. For accounting a run against a real API, the reported usage is exactly the right number — it is what you are charged for.

3.3 The transcript

The meter tells you what a run cost. The transcript tells you what it did — and it is the more consequential of the two, because it is the record the next chapter replays. Every event flows into a SessionLog, one line at a time.

  append<T extends SessionEventType>(type: T, data: SessionEventData[T]): SessionEvent {
    const event: SessionEvent = {
      seq: this.events.length + 1,
      time: Date.now(),
      type,
      data,
      ...(IGNORABLE_TYPES.has(type) ? { ignorable: true as const } : {}),
    };
    this.events.push(event);
    this.writer = this.writer
      .then(() => appendFile(this.path, `${JSON.stringify(event)}\n`))
      .catch((error: unknown) => {
        this.writeError = error;
      });
    return event;
  }

Listing 3.3 · appending one event. It is stamped with the next sequence number, kept in memory, and written as a single JSON line to the log file.

append is the only way an event enters the log, and it does three things at once. It stamps the event with seq — one plus the number already held, so the sequence numbers are dense and strictly increasing, a property the replay chapter leans on to prove the log is append-only. It pushes the event onto the in-memory list. And it queues the event to be written as a single JSON line to the file on disk, so the record survives the process. A session/start, every user and assistant message, every tool result, every usage report — each becomes one line, in the order it happened. The run in cli.ts prints how many events landed and the path they landed at; that file is the whole run, in order, on disk.

3.4 Correct: the numbers add up, the record reloads

Two things have to hold for the run to be accountable: the meter’s totals must be the true sum of what was spent, and the transcript must be a complete, reloadable record. The companion driver checks both without a real model. It runs Hoist’s actual loop with a scripted stand-in that reports known usage — 100/20 tokens on the first turn, 130/8 on the second — feeds a real Meter and a real SessionLog, and then inspects them.

The meter reports exactly 230 prompt and 28 completion tokens across 2 requests, and costUsd(3, 15) returns $0.001110 — the arithmetic of (230 × 3 + 28 × 15) / 1{,}000{,}000 to the cent. The transcript holds nine events, and reloading it from disk yields the same nine, in order, with the sequence check passing — the record is durable and append-only. Any drift in the meter, or a log that fails to reload, breaks one of the four assertions. The checkpoint breaks one on purpose.

The run is now accountable: you can see what it cost and read back everything it did. But the transcript is doing more than sitting on disk as a record — because it captures every event in order, the whole run can be reconstructed from it, and even re-executed without calling the model at all. The next chapter builds the streaming path the transcript records, and then the one after replays it. First, streaming: how the loop shows its work token by token, and handles a tool call that arrives mid-sentence.