Why we built a unified deployment surface
Production AI is not a model behind a URL. It is a model, an API in front of it, workers around it and a control plane nobody wanted to write. Here is why we collapsed them into one.
Ask a team what they deploy and they will say “the model”. Look at their infrastructure and you will find the model, a gateway that authenticates and rate-limits, a queue that smooths bursts, three workers that pre- and post-process, a cache, and a dashboard stitched together from four vendors. The model is the smallest piece.
The sprawl is structural
None of that sprawl is anyone’s fault. Inference platforms were built to serve models, so they serve models well and stop at the endpoint. Everything around the endpoint gets built on general-purpose infrastructure by the same two engineers who were supposed to be improving the model.
The cost is not the tooling. It is the seams: the API deploys on Tuesday, the model on Wednesday, and the version mismatch is discovered by a customer on Thursday.
One surface, one rollout
Terrane’s deployment surface treats an inference endpoint, an HTTP service and a scheduled worker as the same kind of object. They are declared in one file, deployed in one command and rolled out together. If the canary fails on any of them, all of them stop.
This sounds obvious. It was not easy. Inference workloads want accelerators and tolerate cold starts; HTTP services want CPUs and do not. Workers want queues. Making one scheduler reason about all three, and one router send traffic to the right one, took most of our first year.
What we refused to build
We do not host training. We do not sell a vector database. We do not have an agent framework. Every one of those would have been easier to sell than a better rollout, and every one would have pulled the runtime away from the thing it has to do perfectly: keep production traffic flowing when hardware fails, when a region goes dark, when a deploy goes wrong.
A runtime that does one thing reliably is worth more to an operator than a platform that does ten things adequately.
Where this goes
The unified surface is the foundation for everything on the roadmap: cross-service tracing without instrumentation, rollouts that understand dependencies between services, capacity planning across a whole application instead of one endpoint at a time. Runtime 2.0 shipped the foundation. The rest of the year is about building on it.