Platform

The runtime, layer by layer.

Terrane is one runtime with four responsibilities. Each one is a product surface you can configure; none of them is a separate system you have to operate.

Start building Documentation
  1. 01

    Layer 01 · Global edge runtime

    Route every request to the healthiest nearby runtime.

    Edge nodes in 24 regions score each runtime on live round-trip time, queue depth, hardware status and recent error rate. The lowest score wins. Failover is a ramp, not a switch: a degrading region sheds traffic proportionally before it is ever marked unhealthy.

    Regions
    24
    Health interval
    250 ms
    Failover
    proportional ramp
  2. 02

    Layer 02 · Model orchestration

    Versions, hardware and rollouts from one deployment layer.

    Declare the model, the accelerator tier and the rollout policy. Terrane builds the image, reserves capacity in the right order and moves traffic in canary steps with automatic pause on regression. Rollbacks are a single command and take under a minute.

    Hardware tiers
    H200 · Gaudi 3 · L40S
    Rollout
    canary 5 → 25 → 100
    Rollback
    < 60 s
  3. 03

    Layer 03 · Dynamic batching & autoscaling

    Set a latency budget. The scheduler does the rest.

    Instead of static batch sizes and timers, each runtime measures arrival rate and execution cost live and forms the largest batch it can execute inside your latency budget. Autoscaling reads the same signals, so capacity follows demand without a separate rules engine.

    Control input
    one latency budget
    Typical utilisation
    78–86 %
    Scale to zero
    yes, with warm pools
  4. 04

    Layer 04 · Observability

    Four questions, one view: healthy, where, slow, changed.

    Regional availability, live request flow, latency by model, region and batch size, and a deployment timeline overlaid on every chart. Every panel links to logs for the same window, and alerts are configured from the panel you are looking at.

    Percentiles
    p50 · p95 · p99
    Log retention
    30 days
    Alerts
    from any panel
Hardware tiers

Same tiers in every region

A deployment file is portable because the hardware underneath it is. Every region runs the same four tiers, so moving a workload from sfo to sin is a one-word change.

TierAcceleratorsMemoryBest for
h200 8× NVIDIA H200 141 GB HBM3e / GPU Large-model inference, sustained traffic, lowest latency
gaudi3 4× Intel Gaudi 3 128 GB HBM2e / card Balanced inference loads, cost-sensitive throughput
l40s 2× NVIDIA L40S 48 GB GDDR6 / GPU Embeddings, rerankers, small and medium models
cpu CPU-only up to 64 vCPU / 256 GB HTTP services, workers, pre- and post-processing
Regions

24 regions, one endpoint.

Deploy to a list of regions or to all. Your endpoint URL never changes; the router decides where each request lands.

North America

  • iad
  • sfo
  • ord
  • dfw
  • yyz
  • sea

Europe

  • fra
  • ams
  • lhr
  • cdg
  • arn
  • mad

Asia-Pacific

  • sin
  • nrt
  • hkg
  • syd
  • bom
  • icn

South America, Africa, Middle East

  • gru
  • scl
  • jnb
  • dxb
  • bah
  • cpt
Next step

See it run.

The fastest way to evaluate a runtime is to deploy something on it. The free tier includes two regions and enough capacity for a real endpoint.

Start building Talk to an engineer