Deployments & scaling

A deployment is one JSON spec: where requests for it come from (routes), what answers them (exactly one backend kind), and how the pool behaves (scaling, health).

The three backend kinds

A valid spec sets exactly one of these; app-lb refuses a spec with two.

Kind	Spec field	What answers the request	Can scale?
managed	`vm`	An autoscaled pool of Firecracker/KVM microVMs, booted and health-checked by app-lb.	yes — this is the whole point
static	`upstreams`	A fixed list of `host:port` addresses, least-in-flight with failover and periodic health re-probing.	no — the addresses are yours
site	`site`	Files in a directory on the app-lb host, served by app-lb itself: index, 404 page, SPA fallback, ETags and range requests.	nothing to scale

Routing

Each route rule may set host (exact, case-insensitive), host_suffix (label-boundary suffix, so sb.example.com matches a.sb.example.com but not notsb.example.com), and path_prefix. The most specific matching rule wins; an exact host always outranks a suffix. A managed deployment may declare no routes at all — an agent sandbox reached only through exec/shell.

"routes": [
  { "host": "web.example.com" },
  { "host_suffix": "preview.example.com", "path_prefix": "/api" }
]

The scaling policy

Field	Default	Meaning
`min_replicas`	0	Never fewer ready VMs than this. 0 enables scale-to-zero.
`max_replicas`	5	Hard ceiling, boots included.
`warm_pool`	0	VMs kept ready regardless of traffic.
`target_concurrency`	10	In-flight requests per VM the autoscaler aims for.
`scale_to_zero_after_secs`	300	Idle time before the pool drains to zero. Only applies when `min_replicas` and `warm_pool` are both 0.
`cold_start_timeout_secs`	120	How long a request is held waiting for a VM before a 503.
`boot_timeout_secs`	300	How long a booting VM gets before it is killed and retried.
`drain_timeout_secs`	30	How long a draining VM may finish its in-flight requests.
`idle_action`	`destroy`	`retain` stops idle VMs instead of destroying them; the next scale-up resumes the stopped VM, keeping its `/workspace` disk.

What a cold start looks like

A request routes to a deployment whose pool has no available backend.
The proxy registers the request as demand and nudges the autoscaler.
The autoscaler resumes a suspended VM if one exists (its disk is the deployment's state), otherwise creates a fresh one.
The VM passes its health check, is promoted, and the held request completes.

The dashboard's cold_start_waits / hits / timeouts counters trace this pipeline — if waits is not moving, requests are being answered before the wait (a 404 route miss, a 401 from the auth gate, a 403 from a block rule).

Health checks

"health": { "path": "/healthz", "timeout_secs": 2 }

Managed VMs are probed before promotion and while serving; static upstreams are re-probed every tick so a recovered upstream rejoins on its own. No path means a TCP connect check.

Getting an image into a managed deployment

Two mutually exclusive spec blocks, each with its own trigger; neither disturbs the pool until a job finishes and rewrites vm.image:

build — a git repository and Dockerfile, built on the app-lb host. Run with serverctl build <id>.
artifact — a rootfs somebody already built, in an artifact store. Run with serverctl pull <id>.

Static deployments use update instead: commands run in a working directory on the host (git pull && cargo build…), then app-lb verifies the upstreams came back. Sites accept update or artifact — the two ways their files get replaced.

A fuller example

{
  "id": "agent",
  "namespace": "team-a",
  "routes": [ { "host": "agent.example.com" } ],
  "vm": {
    "driver": "firecracker",
    "image": "agent-base",
    "port": 8080,
    "size_class": "medium",
    "disk_size_gb": 20,
    "env_vars": { "RUST_LOG": "info" },
    "ttl_seconds": 86400
  },
  "scaling": {
    "min_replicas": 0,
    "max_replicas": 2,
    "target_concurrency": 8,
    "idle_action": "retain"
  },
  "health": { "path": "/healthz", "timeout_secs": 2 },
  "build": { "repo": "https://github.com/example/agent", "ref": "main" },
  "feed": { "announce": true, "issues": true }
}

Registration order is deliberate: the routing table is swapped before the old pool is torn down, and specs persist one file per deployment under the state directory, so a restart reloads the fleet exactly as registered.