Features

One runtime. Four responsibilities.

Terrane replaces the stack you would otherwise assemble — router, scheduler, deployer, dashboards — with one runtime you configure in one file. Below: what each module does, what it needs permission for, and how it fails safely.

Start building Read the docs
The four modules

What it does, and where it stops.

Open a module to see the workflow plus the five things an operator actually asks about: who can change it, what waits for approval, what is audited, what happens on failure, and how you roll back.

01 Module 01 · Global edge runtime Route every request to the healthiest nearby runtime.

Edge nodes in 24 regions score each runtime on live round-trip time, queue depth, hardware status and recent error rate. The lowest score wins. Failover is a ramp, not a switch: a degrading region sheds traffic proportionally before it is ever marked unhealthy.

Regions
24
Health interval
250 ms
Failover
proportional ramp
permissions
Routing policies are edited by runtime admins; viewers see them read-only.
approval
Region additions apply immediately; region removals wait for in-flight requests to drain.
audit
Every routing decision is sampled to the request log with its score breakdown.
exception
A region below its health floor is shed proportionally, then marked unhealthy after three failed probes.
rollback
Pinning traffic back to a region is one command and takes effect within one health interval.
02 Module 02 · Model orchestration Versions, hardware and rollouts from one deployment layer.

Declare the model, the accelerator tier and the rollout policy. Terrane builds the image, reserves capacity in the right order and moves traffic in canary steps with automatic pause on regression.

Hardware tiers
H200 · Gaudi 3 · L40S · CPU
Rollout
canary 5 → 25 → 100
Rollback
< 60 s
permissions
Deploys require the deployer role; production rollouts can require a second approver.
approval
Each canary step waits for its error-rate and latency gates, or for a human when you ask for one.
audit
Deployment timeline with who, what, when and the diff of the runtime file.
exception
A failed gate pauses the rollout and keeps the previous version serving all traffic.
rollback
terrane rollback returns to the last healthy version in under a minute.
03 Module 03 · Dynamic batching & autoscaling Set a latency budget. The scheduler does the rest.

Each runtime measures arrival rate and execution cost live and forms the largest batch it can execute inside your latency budget. Autoscaling reads the same signals, so capacity follows demand without a separate rules engine.

Control input
one latency budget
Typical utilisation
78–86 %
Scale to zero
yes, with warm pools
permissions
Budgets and capacity ceilings are set per service by runtime admins.
approval
Scale-up above a configured ceiling requires an explicit raise of the ceiling — never silent.
audit
Capacity changes are logged with the signal that triggered them.
exception
When capacity is unavailable in a region, requests are routed to the next healthy region instead of queuing.
rollback
Ceilings and budgets are versioned with the runtime file; reverting the file reverts the behaviour.
04 Module 04 · Observability Four questions, one view: healthy, where, slow, changed.

Regional availability, live request flow, latency by model, region and batch size, and a deployment timeline overlaid on every chart. Every panel links to logs for the same window, and alerts are configured from the panel you are looking at.

Percentiles
p50 · p95 · p99
Log retention
30 days
Alerts
from any panel
permissions
Dashboards are readable by every member; alert routing is managed by admins.
approval
Alert changes are applied immediately and recorded.
audit
Log queries and exports are themselves logged.
exception
If metrics delivery is delayed, panels show the staleness explicitly instead of flat lines.
rollback
Alert configuration is versioned and restorable to any previous state.
Who runs on it

Three shapes of workload.

The same runtime file, three very different sets of settings. Each tab shows the handful of lines that matter for that workload, and a team that runs it in production.

Serve models behind an endpoint that never changes.

Embeddings, rerankers, chat and search models with a latency budget in the tens of milliseconds. The runtime batches inside that budget, rolls new versions out as canaries and keeps the endpoint stable while the hardware underneath moves.

Modules involved Global edge runtimeModel orchestrationDynamic batching & autoscaling

terrane.yaml · the lines that matter 
services:
  - name: rerank
    kind: inference
    model: bge-reranker-v2
    hardware: l40s
    latency_budget_ms: 40
rollout:
  strategy: canary
  steps: [5, 25, 100]
E-commerce Marketplace search team p99 latency 340 ms98 ms Read the case study
Regions

24 regions, one endpoint.

Deploy to a list of regions or to all. Your endpoint URL never changes; the router decides where each request lands. 3 hubs anchor the control plane; every region runs the same four hardware tiers.

Hardware tiers

Same tiers in every region

A deployment file is portable because the hardware underneath it is. Every region runs the same four tiers, so moving a workload from sfo to sin is a one-word change.

TierAcceleratorsMemoryBest for
h2008× NVIDIA H200141 GB HBM3e / GPULarge-model inference, sustained traffic, lowest latency
gaudi34× Intel Gaudi 3128 GB HBM2e / cardBalanced inference loads, cost-sensitive throughput
l40s2× NVIDIA L40S48 GB GDDR6 / GPUEmbeddings, rerankers, small and medium models
cpuCPU-onlyup to 64 vCPU / 256 GBHTTP services, workers, pre- and post-processing

Per-tier usage rates are on the pricing estimator.

How it works

Describe. Deploy. Observe.

  1. 01

    Describe

    One file declares services, regions, hardware and how rollouts should behave. Commit it next to your code.

    terrane.yaml 
    runtime: edge-inference
regions: [iad, fra, sin]
autoscale: enabled
batching: dynamic

services:
  - name: chat
    kind: inference
    model: llama-3-70b
    hardware: h200
  2. 02

    Deploy

    The dry run prints the plan. The deploy streams build and rollout logs, then hands you an endpoint that never changes.

    terminal 
    terrane deploy --dry-run
terrane deploy
terrane endpoint chat
  3. 03

    Observe

    Health, request flow, latency and the deployment timeline in one view. Alerts from any panel; logs one click away.

    terminal 
    terrane status --watch
terrane logs chat --region fra --since 15m
Questions

Before you deploy

Pricing questions live on the pricing page. Anything else: ask an engineer.

01 Do I need Kubernetes?

No. Terrane is the runtime. You describe services in one file and the platform schedules them on managed capacity. There is no cluster to create, upgrade or right-size.

02 Can I bring my own container?

Yes. Use image: instead of model: for any service. The managed vLLM and TensorRT images are conveniences, not requirements.

03 What happens during a region outage?

The router sheds traffic from the degraded region proportionally as its health score falls, then marks it unhealthy. Neighbouring regions absorb the load. You get a notification, not a page, in most incidents.

04 Can I migrate from another inference provider?

Usually in an afternoon. terrane migrate reads common deployment formats and prints an equivalent Terrane file for review. Enterprise plans include hands-on migration help.

Next step

See it run.

The fastest way to evaluate a runtime is to deploy something on it. The free tier includes two regions and enough capacity for a real endpoint.

Start building Talk to an engineer