Concurrency and connections

wasmCloud gives you two families of controls that decide how much work a component does at once and how much of the network it may hold open while doing it:

• Instance limits set how many calls a single warm instance (a component the host keeps ready between calls) serves concurrently, and how many warm instances the host keeps. They live on the component, in the Workload (or WorkloadDeployment) custom resource.
• Connection quotas set how many outbound and inbound connections a workload may hold. They live on the host, set through Helm values or host flags, and apply to every workload the host runs.

Raising instance concurrency raises how many connections the workload needs, so they are worth tuning as a pair. This page explains how they fit together and how to size them.

For the full field reference, see component runtime fields and workload security.

Scoping

Three settings decide a component's total in-flight work, each at a different scope:

Scope	Field	What it sets
Instance	maxConcurrency	calls one warm instance serves at the same time
Component	poolSize	warm instances the host keeps for the component
Deployment	replicas	copies of the workload across the host group

• maxConcurrency and poolSize are set on the component entry.
• replicas is set on the WorkloadDeployment.
• A single component's warm capacity is poolSize × maxConcurrency in-flight calls. Multiply by replicas for the deployment's total. Each of those calls can open its own outbound connections, so the connections a workload needs grow with the same product.

That is the interaction to keep in mind: instance concurrency sets the demand, connection quotas set the ceiling.

Instance limits (per component)

These three fields sit directly on a component entry, under spec.template.spec.components[*] in a WorkloadDeployment or spec.components[*] in a Workload:

Field	Default when unset	What it does
poolSize	none kept (every call gets a fresh instance)	how many warm instances the host keeps between calls
maxConcurrency	1	how many calls one warm instance serves at the same time
maxInvocations	unlimited reuse	retires a warm instance after it admits this many calls, then replaces it

yaml

spec:
  template:
    spec:
      components:
        - name: api
          image: ghcr.io/example/api:1.0.0
          poolSize: 10
          maxConcurrency: 8
          maxInvocations: 1000

• maxConcurrency above 1 lets one instance overlap calls while it waits on I/O, which is what makes pooling pay off for network-bound work. It is only safe for a guest (the component's own code) that yields while it waits rather than blocking the thread.

• maxInvocations bounds how long a warm instance lives, which is useful when a component accumulates state you would rather reset periodically. Both are only meaningful alongside poolSize, since an unpooled component starts fresh on every call.

The component runtime fields reference covers the pooling semantics in full, including how linked components share a store and what a guest trap does to calls in flight.

Connection quotas (per workload)

Every workload gets one connection quota across three surfaces. You set these on the host, either as Helm values in a host group's networking block (runtime.hostGroups[].networking) or as host flags, and they apply to each workload the host runs:

Helm value	Host flag	Default	Bounds
maxOutboundHttpConnectionsPerWorkload	--max-outbound-http-connections-per-workload	128	pooled outbound HTTP and gRPC connections a workload may hold (idle keep-alive connections count)
maxOutboundSocketConnectionsPerWorkload	--max-outbound-socket-connections-per-workload	256	raw wasi:sockets outbound connections a workload may hold
maxInboundSocketConnectionsPerWorkload	--max-inbound-socket-connections-per-workload	256	inbound published-port connections a workload serves at once
maxConnections	--max-connections	derived from the process file-descriptor limit	host-wide ceiling across every workload

The three surfaces behave differently when a workload reaches the limit:

• Outbound HTTP waits. A request over the quota waits for a slot, up to --http-connection-wait (default 5s), then fails. This is the surface most sensitive to instance concurrency, because the outbound pool is what warm, overlapping calls draw from.
• Raw sockets and inbound connections are refused immediately. They do not wait, which avoids a guest deadlocking against its own quota.

All three roll up into the host-wide maxConnections ceiling. The workload security page covers these quotas alongside the raw-socket egress policy they ship with.

Sizing them together

Take the component above: poolSize: 10 and maxConcurrency: 8 is up to 80 calls in flight on one replica. If each call makes one outbound HTTP request, the workload wants up to 80 outbound connections at peak, comfortably under the default 128. Raise maxConcurrency to 16, or have each call fan out to several backends, and the demand crosses the default. Past that point, outbound requests wait on --http-connection-wait and then fail, which shows up as latency spikes and timeouts under load rather than an obvious error.

Two separate things are at work here, and only one of them is a value you set:

• Keep-alive pool sizing is automatic. For reuse, the host keeps some idle connections warm to each destination, and it sizes that from the component's declared poolSize × maxConcurrency, so raising concurrency keeps proportionally more connections ready without a second setting to keep in sync. Idle connections above real demand close on their own after a timeout. You do not tune this.
• The connection ceiling is the value you set. maxOutboundHttpConnectionsPerWorkload (default 128) caps how many outbound HTTP connections the workload may hold at once, across every destination. This is the limit a request waits behind, and it is what you raise when peak demand outgrows the default.

So the tuning workflow is:

1. Set poolSize and maxConcurrency for the throughput you want.
2. Estimate peak outbound connections as poolSize × maxConcurrency × (outbound requests per call).
3. If that estimate approaches maxOutboundHttpConnectionsPerWorkload (default 128), raise the quota, and confirm the host-wide maxConnections still leaves room for every other workload.

note

These quotas are per host process. A workload scaled to several replicas gets the quota once per replica, not divided across them, so scaling out multiplies the workload's total connection budget the same way it multiplies throughput.

Replicas

replicas is the third scaling axis: copies of the whole workload spread across the host group (the pool of hosts the operator manages). It scales throughput past what one host's instance pool can serve, and it is the axis a HorizontalPodAutoscaler or KEDA drives through the /scale subresource. See Autoscaling for how to scale replicas automatically, and why the host group is a precondition for it.

Takeaways

• A component's warm capacity is poolSize × maxConcurrency calls in flight; replicas multiplies that across hosts.
• Instance concurrency sets connection demand; the per-workload quota sets the ceiling. Size them as a pair.
• The keep-alive pool is sized automatically from declared concurrency, so you tune concurrency and the quota, not the pool directly.
• Outbound HTTP over quota waits up to --http-connection-wait (default 5s) then fails; raw sockets and inbound connections are refused immediately.
• The connection quota applies once per replica, so scaling out multiplies the total connection budget.