Terrane runtime v2.0

Run inference where your users are.

Terrane is a unified runtime for inference workloads, APIs and backend services. Region-aware routing, dynamic batching, elastic capacity and live observability, from a single deployment file.

Start building Read the documentation
runtime-status.log
Runtime live 24 regions / global edge

Control plane

Global deployment topology

Synced
P99 latency
18ms
Global traffic
14.2kreq/s
Regions online
24/24
Failover
none
Runtime health Stable

99.98%

Regional availability
Router events Live

    Failover scenario replays every 14 s

    Platform capabilities

    Infrastructure that ships with the model

    Deploy high-performance inference, orchestrate workloads automatically, route traffic globally and run on the frameworks your team already uses. The control plane underneath is ours to manage.

    Runtime v2.4 · 24 regions · 4 modules

    Inference 01 / 04
    Tier
    8× H200
    Throughput
    41.2k tok/s
    Utilisation
    94 %
    p50 latency
    38 ms
    Live view of the Terrane runtime: inference throughput, autoscaling, global edge latency and framework support.
    1. 8× H200 · 141 GB HBM3e · 3.2 TB/s

      Accelerated inference

      Run LLMs, embeddings and multimodal workloads on compute tiers sized for sustained production traffic, not demos.

    2. 0 → 105 GPUs · scale-to-zero · 40 s cold start

      Serverless orchestration

      Scale from zero to multi-region capacity automatically, without tuning clusters, queues or hardware pools by hand.

    3. 24 regions · anycast routing · health-aware failover

      Global edge inference

      Route every request to the healthiest nearby runtime so latency stays low and availability stays high, everywhere.

    4. PyTorch · vLLM · TensorRT · ONNX · JAX · OCI

      Any framework, any model

      PyTorch, TensorFlow, TensorRT, vLLM or your own container. Bring the stack you have, keep the deployment workflow.

    Core infrastructure

    Four modules, one runtime.

    Terrane brings deployment, routing, batching and model orchestration into one unified runtime so technical teams can ship production AI systems without stitching together fragmented infra.

    runtime v2.4 · 24 regions · synced 12s ago

    1. Deploy inference workloads close to demand with region-aware routing, resilient failover and low-latency execution across the distributed runtime.

      Regions
      24 online
      Median latency
      21 ms

    2. Manage model versions, hardware allocation, image builds and rollout policies from a single deployment layer designed for production inference.

      Live version
      v2.4 · 75 %
      Canary
      v2.5 · 25 %

    3. Increase throughput automatically with request batching, queue-aware scheduling and runtime-level optimisations that improve GPU utilisation under live traffic.

      Batch size
      32 avg
      GPU utilisation
      94 %

    4. Ship APIs, inference endpoints and backend services through one deployment workflow with built-in routing, autoscaling, observability and infrastructure-aware execution.

      Surfaces
      API · endpoint · service
      Last deploy
      synced 12 s ago
    Workflow

    How Terrane works

    From runtime setup to live observability: one command, six steps, no fragmented infrastructure to stitch together. Watch the run.

    $ terrane deploy llama-3 --regions iad,fra,sin
    terrane.run · llama-3 · 3 regions
    00:00.0 / 00:41.2 idle

    1. Bring models, APIs, queues and backend services into one deployment-ready control surface.

    2. Configure compute, regions, scaling rules and rollout policies before traffic ever goes live.

    3. Roll workloads out across regions through one workflow instead of managing isolated surfaces.

    4. Send requests to the healthiest and closest runtime path using latency- and region-aware routing.

    5. Expand active capacity under live inference demand without manually tuning queues or pools.

    6. Track runtime health, request flow, regional status and system activity through one operational view.

    1. — waiting to start

    • 0s

      to live

      Define deployment behaviour once and let Terrane manage execution across regions and hardware tiers.

    • 0

      regions, one workflow

      Stay informed with real-time metrics for latency, health, throughput and regional runtime status.

    • 0

      manual steps

      Control compute classes, scaling thresholds, routing rules and rollout behaviour without rebuilding workflows.

    Proof / readiness

    Trusted for production workloads

    Terrane is built for teams running live inference, distributed APIs and regional runtime infrastructure, with reliability, observability and operational control designed into the platform.

    Operator feedback Platform team / applied AI
    “Terrane removed the infrastructure sprawl from our inference stack. We can deploy globally, monitor runtime health and scale traffic without building a custom control plane around it.”
    • Global routing
    • Autoscaling
    • Full observability
    Lead Platform Engineer Production inference team

    Performance visibility

    Monitor latency, throughput, queue health and regional runtime behaviour through one operational surface.

    Built for SRE teams · platform operators

    Runtime flexibility

    Deploy across regions, compute classes and model-serving stacks without rebuilding your delivery workflow.

    Supports vLLM · APIs · backend services

    Operational reliability

    Regional failover, health-aware routing and runtime safeguards keep production traffic stable under live demand.

    Used for Inference APIs · global runtimes

    Performance visibility

    Monitor latency, throughput, queue health and regional runtime behaviour through one operational surface.

    Built for SRE teams · platform operators

    Runtime flexibility

    Deploy across regions, compute classes and model-serving stacks without rebuilding your delivery workflow.

    Supports vLLM · APIs · backend services

    Operational reliability

    Regional failover, health-aware routing and runtime safeguards keep production traffic stable under live demand.

    Used for Inference APIs · global runtimes
    Availability
    0%

    Runtime health across active regions under sustained production traffic.

    Median latency
    0ms

    Region-aware request routing keeps inference performance close to end users.

    Active regions
    0

    Distributed runtime presence for global AI services and backend workloads.

    Scale profile
    0Burst

    Elastic runtime expansion aligned to live load, queue pressure and inference demand.

    Journal

    Notes from the runtime team

    Latency histograms, rollout post-mortems and the occasional opinion on how production AI should be run.

    All articles
    1. 01 Product updates Terrane Runtime 2.0: unified deployment surface, 24 regions, live observability The second major release of the runtime brings one deployment file for inference, APIs and backend services, six new regions and a rebuilt observability layer. DR Daniel Reyes
    2. 02 Case studies Case study: transcribing 900 hours of audio a night on capacity that goes to zero by morning How a media group moved its overnight transcription batch onto Terrane, cut the cost per audio hour by more than half and stopped babysitting failed jobs. DR Daniel Reyes
    3. 03 Engineering Cutting p99 latency in half with region-aware routing How the Terrane router picks a runtime path using live health, queue depth and network distance, and what it did to our tail latency across 24 regions. MO Maya Okonkwo
    Start deploying

    Ship to production this afternoon.

    Move from prototype to production with a unified runtime for inference, APIs and backend services, built for global routing, elastic scale and real operational visibility.

    Start building Explore the features
    npx terrane deploy
    1. --:-- Ashburn IAD 8× H200 Live
    2. --:-- San Francisco SFO 4× Gaudi 3 Live
    3. --:-- Chicago ORD 2× L40S Live
    4. --:-- Dallas DFW 4× Gaudi 3 Live
    5. --:-- Toronto YYZ 2× L40S Live
    6. --:-- Seattle SEA 4× Gaudi 3 Live
    7. --:-- Frankfurt FRA 8× H200 Live
    8. --:-- Amsterdam AMS 4× Gaudi 3 Live
    9. --:-- London LHR 2× L40S Live
    10. --:-- Paris CDG 4× Gaudi 3 Live
    24 / 24 regions live Global runtime Elastic scaling Built-in observability No cluster management required