Scale and Distribute

In the previous steps, we started a developer loop, added two interfaces (wasi:logging and wasi:keyvalue), and extended our application with pluggable capabilities.

Now we'll learn how to:

• Scale our application to handle multiple parallel requests
• Distribute an application globally
• Use the wasmCloud dashboard

Prerequisites

This tutorial assumes you're following directly from the previous steps. Make sure to complete Quickstart, Add Features, and Extend and Deploy first.

Scaling up 📈

So far, our application can only handle a single request at a time. This is because only a single instance is defined for it in the application manifest (wadm.yaml), as a property of the spreadscaler:

yaml

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: hello-world
  annotations:
    description: 'HTTP hello world demo'
spec:
  components:
    - name: http-component
      type: component
      properties:
        image: file://./build/http_hello_world_s.wasm
      traits:
        - type: spreadscaler
          properties:
            instances: 1
...

Following the specification in the manifest, wasmCloud instructs the host's WebAssembly runtime (wasmtime) to instantiate only a single instance of our component at any time to process incoming requests. As a result, requests received at the same time are processed sequentially, one after the other.

Let's try adding a simple sleep to the handler to simulate a longer processing time:

Go

go

//go:generate go run go.bytecodealliance.org/cmd/wit-bindgen-go generate --world hello --out gen ./wit
package main

import (
	"fmt"
	"net/http"
	"time"

	atomics "github.com/wasmcloud/wasmcloud/examples/golang/components/http-hello-world/gen/wasi/keyvalue/atomics"
	store "github.com/wasmcloud/wasmcloud/examples/golang/components/http-hello-world/gen/wasi/keyvalue/store"
	"go.wasmcloud.dev/component/log/wasilog"
	"go.wasmcloud.dev/component/net/wasihttp"
)

func init() {
	// Register the handleRequest function as the handler for all incoming requests.
	wasihttp.HandleFunc(handleRequest)
}

func handleRequest(w http.ResponseWriter, r *http.Request) {
	logger := wasilog.ContextLogger("handleRequest")

	name := "World"
	if len(r.FormValue("name")) > 0 {
		name = r.FormValue("name")
	}
	logger.Info("Greeting", "name", name)

	sleep := 2 * time.Second 
	logger.Info(fmt.Sprintf("Sleep for %v to simulate longer processing time", sleep))
	time.Sleep(sleep)

	kvStore := store.Open("default")
	if err := kvStore.Err(); err != nil {
		w.Write([]byte("Error: " + err.String()))
		return
	}
	value := atomics.Increment(*kvStore.OK(), name, 1)
	if err := value.Err(); err != nil {
		w.Write([]byte("Error: " + err.String()))
		return
	}

	logger.Info(fmt.Sprintf("Replying greeting 'Hello x%d, %s!'", *value.OK(), name)) 

	fmt.Fprintf(w, "Hello x%d, %s!\n", *value.OK(), name)
}

// Since we don't run this program like a CLI, the `main` function is empty. Instead,
// we call the `handleRequest` function when an HTTP request is received.
func main() {}

Rust

rust

use wasmcloud_component::http::ErrorCode;
use wasmcloud_component::wasi::keyvalue::*;
use wasmcloud_component::{http, info};
use std::{thread, time}; 

struct Component;

http::export!(Component);

impl http::Server for Component {
    fn handle(
        request: http::IncomingRequest,
    ) -> http::Result<http::Response<impl http::OutgoingBody>> {
        let (parts, _body) = request.into_parts();
        let query = parts
            .uri
            .query()
            .map(ToString::to_string)
            .unwrap_or_default();
        let name = match query.split("=").collect::<Vec<&str>>()[..] {
            ["name", name] => name,
            _ => "World",
        };

        info!("Greeting {name}");

        let sleep = time::Duration::from_secs(2); 
        info!("Sleep for {} to simulate longer processing time", sleep.as_secs());
        thread::sleep(sleep);

        let bucket = store::open("default").map_err(|e| {
            ErrorCode::InternalError(Some(format!("failed to open KV bucket: {e:?}")))
        })?;
        let count = atomics::increment(&bucket, &name, 1).map_err(|e| {
            ErrorCode::InternalError(Some(format!("failed to increment counter: {e:?}")))
        })?;

        info!("Replying greeting 'Hello x{count}, {name}!'"); 

        Ok(http::Response::new(format!("Hello x{count}, {name}!\n")))
    }
}

TypeScript

typescript

function handle(req: IncomingRequest, resp: ResponseOutparam) { 
async function handle(req: IncomingRequest, resp: ResponseOutparam) { 
// Start building an outgoing response
const outgoingResponse = new OutgoingResponse(new Fields());

// Access the outgoing response body
let outgoingBody = outgoingResponse.body();
{
  // Create a stream for the response body
  let outputStream = outgoingBody.write();

  // Write to the response stream
  const name = getNameFromPath(req.pathWithQuery() || '');

  log('info', '', `Greeting ${name}`);

  const sleep = 2000; 
  log('info', '', `Sleep for ${sleep} to simulate longer processing time`);
  await new Promise(resolve => setTimeout(resolve, sleep));

  // Increment the bucket's count
  const bucket = open('default');
  const count = increment(bucket, name, 1);

  log('info', '', `Replying greeting - Hello x{count}, {name}!`); 

  {
    // Create a stream for the response body
    let outputStream = outgoingBody.write();
    // Write hello world to the response stream
    outputStream.blockingWriteAndFlush(
      new Uint8Array(new TextEncoder().encode(`Hello x${count}, ${name}!\n`)),
    );
    // @ts-ignore: This is required in order to dispose the stream before we return
    outputStream[Symbol.dispose]();
  }
...

Why are we adding a sleep period?

The response time of our hello handler is very low. To show that requests are not processed in parallel, we need to ensure a longer response time, which we can exploit in our tests.

Because we've made changes, run wash build again to compile the updated Wasm component.

bash

wash build

If you stopped your local wasmCloud environment and Redis server in the previous tutorial, start them up again:

shell

wash up -d

Local Redis Server

shell

redis-server &

Docker

shell

docker run -d --name redis -p 6379:6379 redis

Deploy the latest version of our application:

shell

wash app deploy wadm.yaml

Now we'll try to send multiple requests in parallel.

Linux/macOS

shell

seq 1 10 | xargs -P0 -I {} curl --max-time 3 "localhost:8000?name=Alice"

text

Hello x1, Alice!
curl: (28) Operation timed out after 3002 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3006 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3006 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3005 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3005 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3003 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3005 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3001 milliseconds with 0 bytes received

Windows

powershell

1..10 | ForEach-Object -Parallel { curl --max-time 3 'localhost:8000?name=Alice' }

text

Hello x1, Alice!
curl: (28) Operation timed out after 3002 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3006 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3005 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3005 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3003 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3005 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3001 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3005 milliseconds with 0 bytes received
curl: (28) Operation timed out after 3001 milliseconds with 0 bytes received

As you can see, only the first curl command receives the expected response in time, while all the others run into a timeout. However, if you check the logs of the wasmCloud host, you will see that multiple requests have been received and forwarded to our component one after the other.

text

2024-10-20T19:29:30.897232Z  INFO log: wasmcloud_host::wasmbus::handler: Greeting Bob component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:30.897253Z  INFO log: wasmcloud_host::wasmbus::handler: Sleep for 2 to simulate longer processing time component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:32.905355Z  INFO log: wasmcloud_host::wasmbus::handler: Replying greeting 'Hello x1, Bob!' component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:32.906138Z  INFO log: wasmcloud_host::wasmbus::handler: Greeting Bob component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:32.906258Z  INFO log: wasmcloud_host::wasmbus::handler: Sleep for 2 to simulate longer processing time component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:34.914152Z  INFO log: wasmcloud_host::wasmbus::handler: Replying greeting 'Hello x2, Bob!' component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:34.914992Z  INFO log: wasmcloud_host::wasmbus::handler: Greeting Bob component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:34.915023Z  INFO log: wasmcloud_host::wasmbus::handler: Sleep for 2 to simulate longer processing time component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:36.923568Z  INFO log: wasmcloud_host::wasmbus::handler: Replying greeting 'Hello x3, Bob!' component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:36.924326Z  INFO log: wasmcloud_host::wasmbus::handler: Greeting Bob component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:36.924351Z  INFO log: wasmcloud_host::wasmbus::handler: Sleep for 2 to simulate longer processing time component_id="rust_hello_world-http_component" level=Level::Info context=""
2024-10-20T19:29:38.933227Z  INFO log: wasmcloud_host::wasmbus::handler: Replying greeting 'Hello x4, Bob!' component_id="rust_hello_world-http_component" level=Level::Info context=""

Checking the DEBUG or TRACE logs of the wasmCloud host

You can also check the wasmCloud host's DEBUG or TRACE logs for more detailed information (e.g. using wash up --log-level=debug). In these logs, you can clearly see that for received requests, our hello component is instantiated sequentially.

If you wish, you can also use wash spy to check which messages the capability providers and the hello component have received and sent.

bash

wash spy --experimental hello_world-http_component

Requests are not forwarded to our component anymore

After multiple requests were received but timed out, it is no longer possible to send further requests to our hello application. The reason for this is that the httpserver capability provider is no longer able to invoke the hello component via NATS. To continue, we must first delete and redeploy our application.

To receive multiple requests in parallel, we need to instruct wasmCloud to scale our component according to the incoming load. WebAssembly components can be easily scaled due to the small size and portability of the binaries as well as wasmtime's ability to efficiently instantiate multiple instances of a given component.

We leverage these aspects to make it simple to scale your applications with wasmCloud. Components only use resources when they're actively processing requests, so you can specify the number of instances you want to run and wasmCloud will automatically scale up and down to meet demand.

Let's allow our application to scale up to 100 instances simultaneously by editing the application manifest in wadm.yaml:

yaml

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: hello-world
  annotations:
    description: 'HTTP hello world demo'
spec:
  components:
    - name: http-component
      type: component
      properties:
        image: file://./build/http_hello_world_s.wasm
      traits:
        - type: spreadscaler
          properties:
            instances: 1
            # Update the scale to 100
            instances: 100
...

Our application is ready to be deployed again with up to 100 instances, meaning it can handle up to 100 simultaneous incoming HTTP requests.

Deploy the application again:

shell

wash app deploy wadm.yaml

Now wasmCloud will be able to automatically scale the component according to the incoming load. Let's try sending multiple requests in parallel again.

Linux/macOS

shell

seq 1 10 | xargs -P0 -I {} curl --max-time 3 "localhost:8000?name=Bob"

text

Hello x1, Bob!
Hello x2, Bob!
Hello x3, Bob!
Hello x5, Bob!
Hello x4, Bob!
Hello x6, Bob!
Hello x8, Bob!
Hello x7, Bob!
Hello x9, Bob!
Hello x10, Bob!

Windows

powershell

> 1..10 | ForEach-Object -Parallel { curl --max-time 3 'localhost:8000?name=Bob' }

text

Hello x1, Bob!
Hello x2, Bob!
Hello x3, Bob!
Hello x5, Bob!
Hello x4, Bob!
Hello x6, Bob!
Hello x8, Bob!
Hello x7, Bob!
Hello x9, Bob!
Hello x10, Bob!

Utilization planning is important

As we've seen, if a component receives too many requests in parallel, it may break down and wasmCloud will not be able to forward further requests. Therefore, it is important to plan and manage the specified maximum number of concurrent instances for Spreadscaler components according to the expected load.

When you're done with the application, delete it from the wasmCloud environment:

shell

wash app delete hello-world

Shut down the environment:

shell

wash down

Stop your Redis server:

Local Redis Server

Enter CTRL+C.

shell

redis-cli shutdown

Docker

shell

docker stop redis

Distribute Globally 🌍

No matter where your components and capability providers run, they can seamlessly communicate on the lattice. This means you can deploy your components to any cloud provider, edge location, or even on-premises and they will be able to communicate with each other. This is possible because wasmCloud uses NATS as its messaging layer, which is a lightweight, high-performance, and secure messaging system that can be deployed anywhere.

Distributing your application just requires updating your manifest to include spread information and making sure there are available wasmCloud hosts in the desired locations. We can update our hello application to run in multiple locations by editing wadm.yaml:

yaml

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: hello-world
  annotations:
    description: 'HTTP hello world demo'
spec:
  components:
    - name: http-component
      type: component
      properties:
        image: file://./build/http_hello_world_s.wasm
      traits:
        - type: spreadscaler
          properties:
            instances: 100
            # Ensure the component is spread across multiple zones
            # Of the 100 replicas, place 80% in us-east-1 and 20% in us-west-1
            spread:
              - name: eastcoast
                weight: 80
                requirements:
                  zone: us-east-1
              - name: westcoast
                weight: 20
                requirements:
                  zone: us-west-1

These requirements will ensure that 80% of the replicas are placed in us-east-1 and 20% in us-west-1. You can specify any number of zones and weights to distribute your components across multiple locations. Each requirement matches directly with a label that must be present on a host, and your currently launched host doesn't have any specified labels. Let's launch two more hosts with the required labels, and then deploy our new application.

info

If you already deployed your application, it will be placed in the Failed state because the requirements are not met. As soon as we launch the hosts with the required labels, the application will be automatically rescheduled.

bash

wash up --multi-local --label zone=us-east-1 -d
wash up --multi-local --label zone=us-west-1 -d

Just run wash app deploy wadm.yaml again, wasmCloud will be configured to automatically distribute your component across multiple locations based on the spread requirements. You can see all of your hosts and the components running on them by running wash get inventory.

View the wasmCloud dashboard

You can start the wasmCloud dashboard using the wash.

bash

wash ui

If you would like to change the port that the websocket is enabled on, you can pass the --nats-websocket-port option to wash up. Note that you will need to stop NATS if it is already running or the new config will not apply. You can do this with wash down --all to stop everything.

bash

wash up --nats-websocket-port 4444 # defaults to 4223

What will happen to my application?

Treat wasmCloud hosts as commodity. Stopping your host will stop all of the components running on it, and wadm will look for other available hosts to reschedule your application on. As soon as you launch your new host with the --nats-websocket-port option, your application will be rescheduled on it and work just as it did before.

Then, you can launch the wasmCloud dashboard using wash ui, which will launch the dashboard on http://localhost:3030. This is a great way to visualize what is running on your host, even multiple hosts that are connected to the same NATS server.

Summary

Now you've learned how to scale your applications with wasmCloud and distribute them across multiple locations. You can use this same approach to scale and distribute any wasmCloud application based on load and location requirements. Read on to the next page to learn more about the benefits of wasmCloud and how to continue to develop after this quickstart.