Transcript: wasmCloud Q4 Roadmap: Python, Workload Identity, Wasm Components
Transcript
Bailey Hayes 0:09
Hello and welcome to wasmCloud Wednesday for September 30th. So this is our last community call before spooky October, and also the end of our quarter three, and so we're wrapping up a few remaining things that are on our project board for Q3, and then next week we're going to do our Q4 planning. That's a collaborative session. We're going to do it all together, and I'm going to dive more into that.
But first, I just want to call out that we're getting ready to cut 2.10.2. So this is another patch release on top of 2.10. Got a couple little bug fixes in there that I'm going to go ahead and get out before we drop some bigger features. So get those patches out. I'm expecting to have that cut later today. And if you haven't been building on the latest 2.10 — heads up that last week I took you through our very hard to read release notes. Eric has created a much easier to read release blog, and that's now up on wasmCloud's release blog. So if you want to see the differences between 2.9 and 2.10, that is the place to look. Let me actually bring that up really quick.
So we went through this in detail in the last call, so I don't plan on doing that again. But, hey, we have a huge new feature called same-host routing. That's pretty nice performance if you choose to opt into it. We also have host outbound identity, and then a whole bunch of OTel stuff. So all good things in here, including an upgrade to Wasmtime 48. One thing to highlight what's coming is that Wasmtime 48 is one of the first releases of Wasmtime where I can enable cooperative threads that match what I produce with WASI SDK 34. So, if you're doing experiments in that, heads up, you can with wasmCloud. And if you find stuff, please report it.
So yeah, that is basically our 2.10 release summary. If anybody has any questions on that, happy to dive through any of these features line by line. But I think Eric did a great job summarizing. So I'm going to move on to the next agenda item. If you go to the wasmCloud repo and then go to Discussions, here we have our Q4 roadmap.
This is what we've done for the past several roadmap sessions. We do quarterly planning, and we do most of it together live on the call. I like to basically do an Excalidraw that I share with everybody, and then we can kind of point at things. People can drop ideas. We can talk more about each individual thing together collaboratively. But I actually did my pre-work this time, and I went ahead and came up with what I think are the big things that we should definitely hit this quarter further down in this doc. But the main thing to call out is it's next week, same time and place as the call that we're on right now, and I really, really welcome feedback and thoughts. I have these three little prompts that I've been using for the past couple. Nobody answers them, so I might remove them. But if you wanted to make me happy and answer them, that would be very much appreciated.
I did have this little summary for what we shipped in the last quarter, and oh my goodness, it's a lot, and it's a lot of really awesome, important features. Host component plugins make us, in my opinion, the best WebAssembly runtime out there. Nobody else can extend functionality and capabilities of their host using components, and it is definitely the best way to do it. So, pretty happy with that.
We've got a really nice pluggable framework for having different types of ways to trigger services. We've leaned in huge on P3 with async messaging. We also have a P3-native NATS plugin and interface. We support being able to have multiple components in the same workload, all with the same API that they import or export, and be able to multiplex between all of them. That's freaking huge, and then the ability to also multiplex across different capabilities.
Bailey Hayes 4:45
So I can have, let's say, an import for my users database to Postgres, and I can also have my accounts database be separate, and those have totally different, separate configurations and credentials, but all against the same interface. We did a ton of work in performance, including having concurrency, elastic instance pools, all kinds of good stuff. And not only that, but we spent two full sprints really just hardening and improving reliability and security, and vastly expanded basically all the different knobs for policy settings that you can set in wasmCloud. So feeling pretty awesome about this Q3.
So the main reason why I added this is we will do things that aren't explicitly on the roadmap as needs arise. Obviously, this wasn't all completely scoped for, but the big rocks definitely are up there at the top that we did call out, and so I want to highlight those for what I think should be in the Q4 roadmap. I always like to try to do a couple of bucketed themes, because I think folks like to work in one of these layers, and so I just want to highlight: there's a chunk of things that might be fun for you, and they all kind of go together.
Starting with just the component ecosystem — I think this is one of those places that's really great for new contributors. It's definitely user-facing, right? You get immediate value anytime you improve this. You get people using it right away. I wanted to highlight that basically every quarter we always have a component ecosystem theme. There's always more work to do. This past quarter we focused on Go, and I think we got to a really great place with it. We have our own Go SDK in wasmCloud. It builds now on top of componentize-go, which means people get easy access to WASI P3 through that SDK. And so now I'm kind of wondering if for this next quarter we do a Python SDK, because that's probably by far the number one language that I'm getting asked about, because of this whole AI thing. Everybody wants to use it, so I think now could be a great time.
And remember what I said about our update to Wasmtime 48 and how you can flip on cooperative threading in the background? That means we can start working towards this Python epic. So yes, you'll have to flip on flags. In the South, we have a saying: "hold your mouth right." So it's going to be a little brittle. You're going to do silly things to make it work. You might not be holding it right to get it all the way through. But I think by December, which is what we're trying to scope for in Q4 planning, we could have something really nice that's awesome and stable.
Another thing that is totally on the table is the C++ SDK. Again, with the cooperative threading work, we basically have full pthread compatibility. We know this because — check this out — we have 100% compatibility with this POSIX test suite for pthreads, so I know that we can make something really good there. So I think a C/C++ SDK is also on the table. If people are really into it — I don't think I'm going to commit my time on it, but happy to help people drive that through. So if that's something that you need for your company, then that's obviously something we could definitely scope for.
I also am totally expecting in the next couple weeks to refresh everything with the latest wstd, which is going to have WASI P3 support. So do a rev through the Rust ecosystem and continue to make that even better. Oh, other thing I wanted to highlight, just in that same vein: there's a million and one of these kinds of tasks that need to be done.
Bailey Hayes 8:55
I'm not going to scope them as part of wasmCloud's roadmap, but I do think that we should call them out here, because there's a ton of different places where we can add value to the whole ecosystem. In this case, there's a really popular library that everybody uses basically for TLS, and you can compile this to WASI now, and it's just really tricky to do it, and now it's approved. So I'm just waiting on this last thing to merge, and other people will get this.
The trick basically is: how do you configure compiling with WASI SDK for a Rust project that actually uses C stuff underneath? So it wasn't the case that I made it compile so much as I've now provided a recipe for anybody to be able to use, perhaps with an LLM or agent doing their thing. Now they can look at this and be like, "Oh, cool! Yeah, WASI's supported. I have to set these knobs to be able to compile it, and now it'll just work." And so that's the combination of having enhancements in our WASI SDK and wasi-libc. And then also now WASI P3 is a target in nightly here for Rust, and very soon Rust 1.100 will be coming out, and it'll be a stable target that you can use directly. So lots of things in the component ecosystem umbrella. There's work, actually, Aditya, you've been doing. You're really great at going upstream and getting X, Y, and Z to compile to WASI, right?
Aditya Salunkhe 10:28
Yep. Yeah. Where do I start? I've tried doing a lot of stuff with Lapin, which is the AMQP driver. I got that to compile. I've also made Sentry compile. Also got, I think it was SQLx at some point, which was WASI P3. But it was later revised by Bailey herself, which was a really simple revision using the Tokio update for the WASI P2 support for Tokio, which completely blew it out of proportion. But yeah, that's also something we could prioritize.
Bailey Hayes 11:15
The short answer for that SQLx one is it's just flags, and same thing with this AWS change. It was just flags. For so much of the ecosystem now — before, it was like, okay, I've got to change 40 dependencies, and I've got to change what the code does and how it does it in all these different ways. And now it's like, oh no, I just flip on or off a feature, and so long as that's documented in that repo, my downstream LLM is going to know exactly what to do and be able to get it working. And so, just tons of opportunity here and a ton of value. The work that Aditya has done to get things like Sentry, AMQP, all that kind of stuff compiling to Wasm — now everybody gets to just use it, right? And we all get to build on top of it. So work here is a huge uplift, and it's super appreciated. And anytime you do work in that space, give it a shout in our community call, because once people find out, "Oh, I can do that now" — awesome. So let's make that a point. Anytime you do another ecosystem change, Aditya, let me know. Holler. I want to give it a shout.
Aditya Salunkhe 12:27
Yeah, because earlier it was quite difficult, because you needed to change every layer of the socket, the internal net libraries, the async support and tooling. But yeah, now with the additional ecosystem support, it's like Bailey said. It's just a simple flag switch, so it should be much easier.
Bailey Hayes 12:47
Yeah. So okay, that's that half of the component ecosystem side, and all of that is what we envision for an "Are we componentized yet?" initiative, so people can see: how am I supposed to do AMQP in Rust or Go or in Python? And then we can point them at what they want to use, or what works in Go but what doesn't work really well in Go, right? And we didn't really fulfill our vision for this last quarter, because it was a little too underspecified. So I'd like to spend time just creating a bunch of subtasks and saying, all right, we should be able to have this kind of table for Rust versus Go, and then if we choose Python or we choose C or C++, add a column for that too. If we can just surface that data, I think that also would help people get started really quickly with the whole ecosystem.
Now, these two came out of our really awesome discussions in the past two wasmCloud community calls, and I broke them up into two separate things. I think people really need guidance on what protocol to use given this problem space, right? I get this question a lot, so it definitely means we probably need its own doc page, diving into that and exploring the full design space and all the different things that you should consider when creating an architecture that is component-native. Then the second side of it, which is deeply related but should be its own topic that we dive deep on, is coming up with WIT interface design and composition best practices — talking through my preference for how you design WIT, but also what do I mean when somebody gives me a new WIT definition and I ask, "Does it compose?" Just being able to answer that question, and us having a shared knowledge base for that, I think is going to be very valuable.
So that is the component ecosystem section. Is there anything huge there that I forgot to call out, folks, that I should add before we have our call next week? Well, if you think of something, add it here. We'd very much appreciate it. Or just post up in Slack or send me a DM or whatever. I encourage you to give me feedback, because feedback is a gift and I love to have it in any form. And if you're shy, that's okay. Any of these venues, super welcome.
So moving on to the next section: we want to make sure that our cache for precompiled component artifacts is very customizable, so people can bring their own solution for their own embedding of wasmCloud. We've had this one for a bit, just hadn't gotten to it. So I'm putting this one at the top as one of the first features we should ship in the next quarter. This next one, I think we should consider and potentially commit to, especially if we choose to take on doing a Python SDK. If we're willing to invest in a Python SDK, then I think this needs to be a part of that story. I talked about exploded components in our last roadmapping session as something that we could do, but we didn't quite have an appetite for it, because there were so many other more important things that we needed to get done. But I'm bringing it back up again. It's one of my favorite hits. It's something I really do want to build, which is essentially: in OCI, I want a separate layer for each component that I bring in. I want to be able to dedupe the CPython runtime or the StarlingMonkey runtime that everybody pulls in — the exact same version, the exact same bytes.
Bailey Hayes 16:29
So I want to be able to dedupe it at the OCI layer, and I also want to be able to dedupe it in the runtime layer. So that's why it's right there next to our precompiled component artifact piece. We also need an understanding between those two features: I've been able to essentially explode a component, and now I'm sharing precompiled bytes between, maybe, the same CPython in two places.
Liam Randall 16:58
And then just to be clear, what that would translate to is: today, when we look at components, for example, in Go or in Python, they're a bit larger, because each one has a full copy of the runtime. After we do this, the wasmCloud host could load, say, one copy of Python A, another copy of Python B, another copy of Python C, and then just load the data segment and execute from there. Is that correct?
Bailey Hayes 17:25
Yes. I will say "data segment" is a specific thing in WebAssembly, and technically it wouldn't be the data segment.
Liam Randall 17:32
Yeah, I used a random word. It would still be its own separate component with a stack and its own linear memory and data and all that.
Bailey Hayes 17:44
Yeah, yeah. "Instance" would be the word that I would use here. We would spawn an instance from the precompiled artifact. The reason why I want to make sure we disambiguate from a data segment here is that I have this other one of my favorite boondoggles, which is a streamable data segment. In the world where I think components should be your one true packaging mechanism that you use for everything — Markdown documents, prompts, models, all of the static assets that you load for your app — those, in my book, we should have a feature for. A data segment is effectively the layer when you load things into linear memory: how do I bootstrap a WebAssembly module? So this is a core Wasm thing. If I have the ability to make those segments asynchronous and load them as needed, then I can stuff all kinds of cool static assets in them and be able to do things like prefetch all the things that I'm going to load for a web page, or be really smart about going through a neural network and picking the appropriate paths based on the decision that I'm making given the inference that I'm running. A million and one different use cases for that.
Liam Randall 19:02
Now I'm glad I misspoke, because now you painted a much richer picture of how this all comes together. Thank you very much for putting that out there for the community. I think that's one of the super nuanced points. I was trying to plant a question with that, because I think your description maybe didn't communicate to all the people that watch and consume this the impact of that feature. That's one that I think is just super exciting. Sorry —
Bailey Hayes 19:30
Absolutely. So to be clear, this one is just for specifically deduplicating the module bytes that we load, which we're going to be able to precompile and prefetch. We do actually already have a precompilation cache that we use, but making sure that it's customizable, so other people can slot in and bring their own distributed cache, for example, or use different kinds of caching mechanisms rather than the one that we have baked in. So all that good stuff — which also means anytime you're touching that kind of stuff, you should be doing benchmarking. And our benchmarking milestone will be evergreen. We will always be doing benchmarking and expanding out that suite. We've had one that we hadn't quite closed out for Q3 yet that Jeremy's been putting in a ton of work on recently. Jeremy, is that PR already up? You want to —
Jeremy Fleitz 20:27
Yeah, I put it in chat.
Bailey Hayes 20:30
Well, I'm going to do it the slow way.
Jeremy Fleitz 20:32
Okay. Yeah, it's 5630. Yeah, there you go.
Bailey Hayes 20:38
You want to talk about it?
Jeremy Fleitz 20:39
Yeah, just really quickly. So there's this issue that's been out there, 5052, for I think two quarters now, and finally it's like, hey, it's the end of September, I need to get this in. So what this does is — we already have benchmarking that's out there, and that runs every time we do a release. This is more for doing a load-type test with k6, and then going through different types of scenarios, like a fan-out scenario, or even chain calls on the same host. Doing the chain call on the same host, it can also use local routing, which was one of the later features. And when I enabled that, that's when it actually found an issue with local routing. So this is very helpful even just for finding issues that occur only under a strong load-type test.
So this one's in draft right now. It should be ready to merge later today, and it also has scripts inside the same folder where the other benchmark ones are. But it doesn't have to run just in CI/CD. You can run this locally on your laptop, and you can also run it against your cluster where you have wasmCloud installed, if you wanted to test to see how the performance is.
Bailey Hayes 21:50
Cool, yeah. And can you describe what suites you have created right now? I see k6 bench, a smoke test. Are there other kinds of specs that you've added?
Jeremy Fleitz 22:05
No, this is very preliminary. This is really just a basic k6 bench type of test, and actually in the next quarter, like you called out, we have some more k6 enhancements that we're going to be adding too.
Bailey Hayes 22:19
Yeah, that's kind of the thing. Every time we add a new plugin or we add anything that involves the network, for example, we really should be running that through its own k6 spec. So we can very much be focused on expanding that out. So I'm really excited about it. Getting a spec, for example, on the NATS plugin that you added would be a super valuable one. Continuing on that front, it's not just k6 benchmarking that we're going to be doing. k6 is a macro benchmarker. There's a ton of things that are still super valuable with micro benchmarks as well. Fine-tuning how host component plugins load and how we do async and all that kind of stuff is much easier to do at a micro benchmark level, because you're talking about really fine margins. But then we really should be doing more different types of tests, like realistic load tests for long periods of time, that kind of thing. That's always a fun area to innovate. So that's our performance and efficiency bucket. As always, if you think of other things we should be throwing into that theme, add them right here.
Moving on to the next one, on integration. This one, I'm already mentally booked for Aditya, basically, as we talked about last week. Anything you want to say on that one, Aditya?
Aditya Salunkhe 23:51
Yeah, I already have the issue filed upstream on object_store, and I also have the pull request generated for that. I reviewed it quite a few times, actually, with the little time I got, and it seems good. I also tested it with a preliminary version that I implemented in wasmCloud, and it worked — only on LocalStack, though. But I'm sure it works well with the real cloud providers. It should be up for review by those maintainers, and once that's done, we're going to take a poke at implementing only the S3 part of it, and I think we're going to stub out the Azure and GCS parts of it until the upstream portion lands. After that, I think we should be good to go. It's quite close to completion.
Bailey Hayes 24:46
Yeah, so that's something I'm expecting early in the quarter, and that's huge. I think there's a lot of things in this integration bucket that somewhat overlap with the component ecosystem. You get this thing compiling to Wasm, then we can reuse it in a lot of places. But specifically, we have built-in plugins, and this is one of the built-ins that we ship, and for that reason we want this upstream project to be the one dependency that we really have to maintain that can talk to all these different kinds of providers. So it's definitely the right library for the right abstraction that we're providing with WASI Blobstore and WASI Key-Value.
Now, there's more things that I think we can do in this space that could be powerful and unique to us. You can see where we innovated quite a bit on the NATS plugin, where there are NATS-specific policy decisions that happen, and we do that at the binding level: we want to attach this workload to this plugin, and that binding is a capability grant. On these capability grants, it can say things like, you're allowed to talk to these topics, or you're allowed to create these buckets. And as I was reviewing that from Jeremy, I realized there's a huge opportunity here for us to do the same thing for Blob Store and Key-Value. There's a big design part that we would need to put in first for those, but I think it's also really valuable, because that's something that most people generally want. They're going to have authorization at play here already, which we already lean into, but I think people are going to really want even finer-grained control that's also declarative to the workloads that are accessing these things. So that's kind of the ideal.
Aditya Salunkhe 26:52
Do we have any early shouts for some blob stores or key-value stores that we probably want to port?
Bailey Hayes 27:02
Oh yeah. Well, S3 is the most important, I think, by far. Everybody's just kind of landed on that as being the lingua franca of object stores. Oh yeah, I should give that one a shout, and there's probably some more forked names that I can't think of. And I get in trouble if I say Redis, for example. Yes, those would be really valuable for the bucket policy — I thought that's what you were asking for. I did actually create a sketch of this way back when, with the idea of exposing what a bucket is: surfacing a prefix and whether or not you should be allowed to create it. Right now, we just create it, and there are other kinds of things that are policy-level that you might need to specify on each bucket.
We weren't quite ready to swallow this one, so it hasn't landed yet, but it's something I think we should bring into scope. Now, one more thing I'd like to give a shout for: a great way to expand our capabilities is to do host component plugins, right? It's very easy for me to say "yeah, add that one too" when it's not in the main wasmCloud repo, where we're doing cargo-vet and audits on all the dependencies. So I'm just saying, please go wild over here on integrations and expansions. There's a million and one things that I think we can start adding, and because we have the host component plugin feature, I feel great about adding them, because each one of them is sandboxed, right? So it's a really great way to expand the host capability footprint. So that was that shout there. But yeah, my number one, if you want my preference, is an S3-specific API. Yeah, Frank.
Frank Schaffa 29:16
I'm wondering — so you're asking for folks to test those things and experiment and so forth. I'm just wondering, why don't we have Claude or any other agentic stuff doing this, and we just spec those things?
Bailey Hayes 29:41
I mean, it should be totally possible. And look — you can tell that that's what I'm up to myself. If you look at my PR over here, we went through — I haven't finished it — but I came up with all the different ways that you could support a host component plugin.
So it's one of those cases where it does take a decision, right? And you do need a human driver to make sure that the output takes advantage of the properties that WebAssembly gives you. That requires a bit of prompting. We do have a skill that a lot of people are using that I totally recommend people use. What would really speed this up is, again, having this documentation, which would help these LLMs produce the right thing.
Aditya Salunkhe 30:36
Yeah, because it's a bit tough to actually navigate the whole WIT interface that drives the whole host component plugin, because at the end of the day, the whole idea is to have as much specificity as possible, which is a bit tough to box in, and that requires a bit of skilled expertise. Take, for example, if someone were trying to implement something like NATS — we would probably hand it over to Yordis, who would actually be able to give his inputs that would help shape the NATS interface better. But yeah, with improved documentation, that would certainly be better.
Bailey Hayes 31:22
I think that would definitely accelerate it. I think we could absolutely spec out some of these things. I think doing a good spec is the hard part, though, because that's coming up with this decision-making. Right now we're not quite at the point where LLMs produce what I would want. Will it produce a thing that works and runs? Yes, you can one-shot that. It's not quite at the point where it's something that I would want in the wasmCloud namespace that other people would build on top of. But I think by doing these two, and then getting some more representative examples and highlighting what those decisions are in awesome-wasmcloud, anybody can then point an LLM and go forth and create really rock-solid host component capabilities. So that's definitely why I want it on the roadmap, because there's no reason you should have to require me. We should be able to scale this out quite a bit.
So the next one is continuing our work on security, but I want to call out workload identity as being — you know, when you do quarterly planning, you should plan your big rocks first. And our biggest rock, in my book, is this one: actually having full, integrated support for workload identity in wasmCloud. And when I say that, I mean a combination of things, because there's a lot of aspects to this. It's more than just mutual TLS support, and it's more than just an individual workload having its own identity that it can use for a mutual TLS type connection to some other external service. I actually mean even more than that, in that I want to be able to do credential brokering in a rock-solid way in our host, so that people writing agents and agent harnesses are able to safely run them on our sandbox. And we're not just a sandbox. We're a multi-tenant sandbox, so the design for us here is a huge opportunity, and a huge place of complexity, but also completely worth doing, because this is what people ultimately want right now.
Everybody is looking for their agentic platform, right? What can I use to safely run my agents? And there's no better sandbox than WebAssembly. There's nothing that has finer-grained control. With VMs and these other mechanisms, people are aiming for isolation, but they're not actually truly aiming for sandboxing, and that's why they are failing. You can look at the one from this weekend, which was DNS. And pro tip: it's always DNS. And guess what we don't have in WebAssembly and WASI? DNS. So we sidestep so many of these problems that people are getting when they're trying to build on an operating system with POSIX.
So anyway, this I think is a big one. I've been working on an RFC. I have a huge Notion doc where I've been exploring the design space. What I'm going to do is synthesize that down into a human-written RFC for everybody to give feedback on. But the first big thing is that I actually have a series of drafts that stack on top of each other that will essentially get us to where the basics are for this. So we should be able to land some of the first parts of this next week, and I've broken it down into several phases that I'll take you through when we start doing the RFC for that one.
So I've lost where I was supposed to be — too many tabs. I even pulled up a window that didn't have all the tabs, because I feel like I scared Yordis last time, but yet here we are again. I guess I'll just show you how to get to the Discussions tab, and you can give me feedback, because I still see there's nothing here, folks. So please give me feedback on the things you want to see.
Bailey Hayes 35:59
But yeah, that is what I had in mind for what I think we should scope for our roadmap session. And welcome others to come in and talk about the things that they think are important. Anything more on this topic? Nice. Okay. Well, that is mostly everything. I know that we're starting to accumulate a bit of a backlog on pull requests. So even if you're not a maintainer, I welcome other people to come in and code review other folks. It helps me get confidence if somebody else has already had a look and done a first pass. I have a prioritized list for getting through some of these reviews and burning them down. But right now my current rate is maybe four PRs a day, and I need to close them all out. It's somewhere around 15 right now, at least. So I've got my work cut out for me. If anybody wants to help, super welcome. And that is basically it for the planned agenda today. Next up, I'm going to stop the recording, but I want to take at least the maintainers through a threat model that I've been working on. I want some feedback on that, so I'm going to hit the button.