The Ideal AI SDK
What a great AI SDK would do: streaming, structured outputs, tool calling, evals, and observability—without the pain of today's options.
Most AI SDKs give you a function that takes a prompt and returns text. That's not enough to build production apps. Here's what the ideal AI SDK would do.
Streaming by default
Users need to see progress. The SDK should stream tokens by default, with a simple escape hatch for "give me the whole thing." Same API for chat and completion; the caller decides whether to consume the stream or await the full string.
Structured outputs
Raw text is a liability. The SDK should support typed outputs: JSON that conforms to a schema, or structures produced by a DSL (e.g. BAML). Validation and parsing should be built in, so you get runtime safety and clear errors when the model drifts.
Tool calling as a first-class feature
Agents and tools are core. The SDK should make it easy to declare tools, pass them to the model, and handle the loop: model returns tool calls → execute → inject results → continue. No hand-rolled parsing of "I'll call get_weather now."
Evals and regression checks
You should be able to attach evals to a call or a batch: run this scorer, this judge, or this script on the output. The SDK (or a companion harness) should support "run these N cases and fail if something regresses." That's how you ship with confidence.
Observability out of the box
Every call should be traceable: prompt, response, tokens, latency, model, and (if you opt in) a link to your observability backend. No "add our SDK on top of the AI SDK." One instrumentation story. OpenTelemetry Gen AI semantics so traces are portable.
One abstraction across providers
You shouldn't rewrite your app when you switch from OpenAI to Anthropic to OpenRouter. The ideal SDK abstracts the transport and model choice; you get the same streaming, structured output, and tool-calling API regardless of backend. Provider-specific options can be passed through, but the core contract is stable.
We're not there yet. The ecosystem is converging: better streaming, more structured output support, and observability are showing up in multiple places. The SDK that nails all of the above—and stays simple—will win a lot of builders.