Reading Pi Coding Agent: From Multi-Provider Compatibility to Structured Tool Results

Recently I started reading Mario Zechner's article What I learned building an opinionated and minimal coding agent and studying the source code of earendil-works/pi to learn about coding agent harness.

On my first read, I haven't yet reached the complete agent loop, but I've already encountered several questions that are unavoidable when building a harness: how to unify APIs from different providers, how to hand off context between models, how to properly abort streaming requests, and why tool results need to serve both LLM and UI. This article records my current understanding.

Starting with Pi's Four Components

The author's direct motivation for developing Pi was to have a simple, stable, and transparent coding agent: you should be able to inspect what's in the context, the session format should be easy to post-process, and the agent core should not be coupled to any particular UI.

Pi is therefore split into four main parts:

• pi-ai: Unifies different LLM providers, handles streaming, tool calling, reasoning, and usage.
• pi-agent-core: Runs the agent loop, executes tools, and manages state and events.
• pi-tui: Provides terminal UI and differential rendering.
• pi-coding-agent: Assembles the three above into an actually usable coding agent CLI.

This layering is important. Provider API differences belong to pi-ai; the "model—tool—model" loop belongs to agent core; how messages are displayed in the terminal belongs to TUI. When learning harness, if you don't first clarify these boundaries, it's easy to conflate model protocols, orchestration logic, and UI behavior.

Unifying APIs Doesn't Mean Assuming All Providers Are Identical

The article mentions that covering most models only requires understanding four mainstream APIs: OpenAI Completions, OpenAI Responses, Anthropic Messages, and Google Generative AI. The difficulty lies not in the number of APIs, but in many services claiming OpenAI compatibility only supporting part of it.

For example, different providers may have these differences:

• Whether they accept the store field;
• System prompt uses developer or system role;
• Output length field is called max_tokens or max_completion_tokens;
• Whether they accept reasoning_effort;
• Reasoning content goes in reasoning or reasoning_content;
• Whether streaming usage is returned via stream_options.include_usage.

Pi's approach is not to find a "greatest common divisor" that all providers strictly follow, but to retain a compatibility layer beneath a unified Model and Context. The current source code first auto-detects based on provider and baseUrl, then allows model.compat to explicitly override the detection result.

This interface, excerpted from the current project source code, transforms differences into declarable capabilities rather than scattered judgments in business logic:

// packages/ai/src/types.ts
export interface OpenAICompletionsCompat {
  supportsStore?: boolean;
  supportsDeveloperRole?: boolean;
  supportsReasoningEffort?: boolean;
  supportsUsageInStreaming?: boolean;
  maxTokensField?: "max_completion_tokens" | "max_tokens";
  // ...other Provider compatibility options
}

When constructing requests, the adapter chooses fields based on these capabilities:

// packages/ai/src/api/openai-completions.ts
if (compat.supportsUsageInStreaming !== false) {
  (params as any).stream_options = { include_usage: true };
}

if (compat.supportsStore) {
  params.store = false;
}

if (options?.maxTokens) {
  if (compat.maxTokensField === "max_tokens") {
    (params as any).max_tokens = options.maxTokens;
  } else {
    params.max_completion_tokens = options.maxTokens;
  }
}

My first takeaway from this is: a good unification layer doesn't eliminate differences; it concentrates, names, and constrains them. Abstraction still leaks, but the leak points are visible.

Why Token Accounting Can Only Be Best Effort

Providers don't report usage uniformly. Some report input tokens at the start of SSE, others only return complete usage in the final chunk. If the user aborts early, the final chunk never arrives, and the client can't know the exact consumption.

Pi requests and parses provider-reported data as much as possible. In OpenAI-compatible streaming, it prioritizes reading chunk.usage, while also accommodating services that place usage in choice.usage. It then normalizes to:

input + output + cacheRead + cacheWrite + reasoning + cost

With Anthropic, mid-stream accounting is more appropriate: message_start may already contain input and cache tokens, and subsequent message_delta events update output tokens. So even if aborted mid-stream, Pi may still retain some reliable accounting.

But "retaining data already received" doesn't mean "estimating data the provider didn't report". If a provider only reports usage at the end, Pi won't magically get exact billing. This is what the author calls best effort: suitable for personal cost tracking, but not sufficient for accurate customer-facing billing systems.

Browser Integration and CORS

pi-ai can run in the browser, but that doesn't mean the browser can request all providers directly. Only services with appropriate CORS policies allow direct connections, and putting API keys in the browser is usually unsafe anyway.

A more universal structure is:

Browser Agent
    ↓ Pi's unified streaming protocol
Own Backend /api/stream
    ↓ Provider native API
OpenAI / Anthropic / Other Provider

The browser requests its own backend via streamProxy(), while provider credentials stay server-side. The backend calls the actual model and forwards the unified event stream to the client. This simultaneously solves CORS, key protection, and server-side authentication issues.

Context Handoff: Moving Semantics, Not Guaranteeing Lossless Native State

Switching between providers isn't just resending historical text. Different providers embed signatures or encrypted opaque blobs in reasoning and tool call events. These blobs are not the SSE chunks themselves, but data carried by certain chunks. Clients typically can't interpret them, yet need to return them verbatim in subsequent requests.

When continuing conversation between the same or compatible providers, Pi preserves native thinking and signatures as much as possible. When switching to another provider that can't verify the signature, only lossy conversion is possible. For example, Anthropic reasoning can be downgraded to plain assistant content:

<thinking>
reasoning content generated by the original model
</thinking>

The target model can read this history, but won't treat it as native reasoning it generated and signed. So seamless context handoff's more accurate meaning is "the conversation can continue", not "all provider-private state migrates losslessly".

There's also an easy-to-overlook constraint here: switching from a 1M context window model to a 256K model, format conversion can't solve the capacity issue. The harness still needs to compact, summarize, or truncate history. pi-ai handles protocol compatibility; context window management belongs to the upper layer harness.

Custom Models and End-to-End Abort

The model directory can provide type hints, capabilities, context window, and pricing, but it can never be timely enough to include all models. Pi therefore allows directly constructing a Model that describes a local Ollama model or an as-yet-unregistered OpenAI-compatible endpoint.

This doesn't write the model back to a global directory; it's providing a complete model description for the current call. It still must use an API style Pi already supports; the Model interface won't automatically invent a new transport protocol.

This section subsequently discusses abort, primarily criticizing unified AI SDKs rather than just the low-level provider. Many providers' HTTP/SSE requests could originally be canceled, but the upper-layer SDK might not expose AbortSignal, might not pass the signal down, or might discard already-received content after cancellation.

Pi emphasizes end-to-end cancellation:

AbortController
  → Pi stream API
  → Provider SDK / fetch
  → SSE connection

Already-transmitted text_delta, thinking, or partial tool call parameters are accumulated, so even after aborting, you can get a partial response with stopReason: "aborted". It bears repeating: partial response can be retained, which doesn't mean the token usage reported only at the end can also be retained.

Structured Split Tool Results

Tool results face two consumers: LLM and UI. They don't need the same information.

A weather tool can return a short sentence to the model while returning structured data to the UI:

return {
  output: `Temperature in ${args.city}: ${temp}°C`,
  details: { temp }
};

The model needs to know what the temperature is, while the UI might need { temp: 24 } to draw a card or dashboard. If there's only a text string, the UI would have to re-parse the model-readable string; if all UI data is handed to the model, it wastes context tokens.

Therefore, you can understand the result as:

ToolResult
├─ output/content → enter model context, can also be displayed
└─ details        → for UI and application logic use, need not enter model context

The tool's content can also contain images. Images are not only UI attachments; they can also be converted to the Provider's native image input to participate in the next round of reasoning when the target model supports vision. details and image attachments are therefore not the same concept: the former is application data, the latter belongs to model-consumable content blocks.

At the time of writing, Pi already supported models generating tool call parameters via SSE streaming and performing incremental parsing on incomplete JSON. This allowed the UI to preview forming calls ahead of time. However, streaming of tool execution results themselves was not yet supported, such as real-time display of bash continuously producing ANSI output.

So the focus of this design is not "obtaining tool results via SSE," but rather three more explicit capabilities:

• Incremental parsing of tool call parameters streamed by the model;
• Separating model-visible content from UI-specific data;
• Allowing tool results to carry unified content blocks such as text and images.

Summary of the Current Stage

After reading the first half of the article, my understanding of harness began to shift from "the loop of calling the model" to "the system of managing boundaries":

• Provider abstraction must unify calling conventions, but must acknowledge compatibility differences;
• usage statistics are constrained by Provider event protocols and can only take responsibility for reported data;
• context handoff resolves format and semantic handover, without guaranteeing lossless migration of private state and ultra-long contexts;
• abort must run through the entire network and event chain so that partial responses are not lost;
• tool results are boundary objects between model context and UI state.

References

• What I learned building an opinionated and minimal coding agent
• earendil-works/pi