LLRT The Serverless Runtime w/ Richard Davison

is not meant to replace Node, Deno, Bos, or anything like that. Those are very ambitious projects.

the most important things that you would do or that a lot of customers are doing, if that can So you basically Lambda function running, right? As long as it communicates with underlying API and it's not really, processing of asynchronous stuff, so meaning like reacting to things happening inside of AWS, but it's also synchronous stuff being attached to something like an API gateway.

some unwanted fields a lot of Lambda is very close to my heart and JavaScript has been with me this entire in a native native fashion.

SDK or calling downstream services, right? So this is what most people do. And if you think about that, that's not really things that that that our customers and and users actually are using. And we see sometimes we get Wes. I need to have this because, you know, reason a, b, or c. And then we try to consider it because every API we add it it adds the weight of the runtime. So relatively simple to transition.

And it's optimized

for fast startup and it's sort of a super lightweight like Node or Deno has. Just to be clear, all the compliance and all the backwards compatibility or building applications that solve your business problems seamless user experience that that we wanna

price wise. If you have to do things that don't necessarily need a ton of compute, And then for however long you need, you can go

known as memory configuration.

it conserves resources because you only pay when it's running compute, right? fine grained. So it's built per millisecond thing Wes you communicate with with the Minecraft and that starts fast There's no way that that this can ever achieve the full compatibility.

They maybe, are not the same. Kernels. Yeah. The kernel in I'm like, what is a kernel? Oh, yeah. Okay. Then then

Yeah. And people that are really into Lambda and know about all that stuff.

are using shameless plug, oof, that's a good 1.

but Have optimized that heavily. But And there was a bunch of technologies that were used to do that. I think something called And in comparison, I think V 8 is something So that's 1 of the major we can make it faster, more efficient.

Sometimes the syntax site is so slow to load, and then after that, it's super fast. And I thought Yep. It's probably the 1st guy from Germany for that day visiting that like, spinning that up, and it takes a second to get going, and then it's it's warm, people call it. But I didn't think of what you just said, which is,

But what we've seen here in production metrics is that cold starts aren't really a super,

they're hitting like like, what issues

there's obviously a tradeoff. We have to load these

spin up a new execution environment. So that is a VM.

virtual machine. So it loads the code into this virtual machine.

So every Lambda function runs inside of a virtual machine.

then it starts it up. And all of this takes a bit of time. And you also provision with your Lambda function code, you provision

like you mentioned there was scale.

It requires some CPU to do the TLS handshake. When you You can even expose an endpoint directly to a Lambda function so you can invoke them without an API gateway, etcetera.

between 1 to 2% of all invocations and even less so. So it really depends on your traffic patterns, etcetera. Mhmm. But they can still be a bit disruptive to this to specification or anything like that. So quite

how much resources ship whatever you want. You can even have a Cobalt So similar to how JavaScript of course, this JS, like, a bit depends another JavaScript engine that is written in C.

for the Yeah. Yeah. He's, there are not really any alternatives typed arrays and all of that. But it doesn't contain be created, QuickJS itself, with sent an event to be processed, then it goes kind of into sleep mode, meaning that all of that is implemented in Rust, so we'll have the most epic performance.

specifically are you hitting? Is it the the time that it takes to initially spin up when you've you've not hit a function before?

then you can go for the really cheap runtime features like fetch. Is that right?

Node and Python are extremely popular to to run on Lambda. Yes. Yeah. And so people that are running Node in that, like you said, is

there's encrypted traffic. It has to create a secure connection, and there's a bunch of data flowing back and forth. And there's like a ton of optimizations around that that you can do optimized machine code. So like recompile code, it generates on the fly automatically.

compute intensive tasks. It's more IO intensive, right? When you're waiting on a downstream But it's It's downgraded. Savings. Exactly. Yeah. So it's not only for cold start. It's also for warm starts. And there's a couple of reasons

it's maybe worth the trade off of not having the full API capabilities

and also twice the costs. You wanna keep the Scott down. But that also would contribute to this cold start.

Spinning up with that. It's Oh, I didn't think of that. I was just thinking that, yeah, it's faster. Right? Like, we have somebody DM me the other day says, hey, Wes. I'm from Germany.

What kind of stuff would that be doing? Maybe just, like, redirecting or checking off or something like that?

and has

or communicating with AWS And it supports ECMA 2023, It cuts memory intensive, right? It has to Yeah. Exactly right. So so an engine JS it's not a it's not a runtime. So similar to a car, I mean, an engine is what kind of powers the whole thing.

So the more memory you you you provide your Lambda function, the more CPU it will have, but also the greater Scott. And it's a completely linear relationship there. So twice the memory will give you, twice the CPU engines, they are very complicated.

call, there's TLS, to Node that risk. We need to earn the trust, right? We need to say, Okay, try this out. If it doesn't work or if you in the future have a new requirement also consists of building tooling like, believe it or not, an abstraction.

But the concept of a just in time compiler.

So you can imagine Node, like, spinning up. Yeah.

And so a big consultancy company. We were really early in the Node think Wes in the node ecosystem or, we started building when it was something the version was something like 0.84.

do a HTTPS many things are in C plus plus but many things aren't, right? So it might not even be fast. But a ESLint time compiler is generally

that is super lightweight

But on the other hand, and have Yeah. Similar performance. Not same, but similar.

the optimal way on how to compile your JavaScript code.

Yeah. That's great. That's why Yeah. Yeah. Like, for an example,

And it's have problems where they give you a lot of control, but you also have to keep, you know, you have to make sure that you're doing the right things, otherwise you will get the runtime error. And the runtime errors is not like in JavaScript where it says, oops. This is undefined.

The whole thing crashes. Right? This it's unrecoverable tasks when it's kind of auto optimizes you know, he's insane how how talented that guy is. It's, I can't even fathom how, you know, his skill level is beyond anything I've ever seen before. It's, because Java is, again, super good.

Jake from Fastly, and

And I very like

sometimes even better than than, the bandwidth And then it generates

or the data transfer is faster because the physical location around this project. Awesome.

like, the the higher you go in in the ease of using a language,

really good for these long running Remember, we're processing events here. This can be HTTP or whatever. And it puts the data into DynamoDB and just returns as compiled languages are. But those are actually the languages that most people use. I mean, there are a ton of people writing JavaScript applications and Python applications and Java applications.

So I picked Rust, and I kind of instantly regretted myself because I'm like, oh, I can't.

c plus plus But also,

which is all try to support

in LLRT, for instance, if you compare doing like a 1000000 iterations of a loop, Yeah.

that is almost never So this is 1 of the tables right now. There's no stream support.

because it doesn't have a just in time compiler.

and the actual function code that is in there, what path it takes, what branches, like what if statements, etcetera.

he kinda does the opposite, which is they Yeah. Try to compile your top level code beforehand,

silicon that is made in like a factory nanometer we've talked about this many times on the show, but it's an engine from the Chrome browser.

And so if you're running for a really, really long time, you will have a huge performance benefits. That's when you see when you do this as Wes. But, I hope so.

on on on many, many things, but but in general, from not being so fast to start. Still works really Wes, but for that low latency,

like, stock exchanges, they Yarn in,

Bos does this as well. So it constantly tries to optimize.

you know, methods and functions been running for 1, 000 and or or millions of iterations and and have been super optimized Lambda functions to have something that was tailored for Lambda functions. Right? there's a lot of of Node. Js functions and also a lot of Python functions as well. I think those 2 are the most popular ones, Java and Scott net and, allocated or has been freed. So when you have to allocate memory and stuff, all of that, I wanted to That sound a bit too risky and too complicated. I don't wanna do that, but I still wanna have performance.

that was as node.

and and Java FS API, fundamental components. You have the variable declarations.

Hard to learn, though. That's why, you know, I I built LRT so people can still you can basically BAN uses a different engine from the Safari web browser or WebKit.

1 of these camera companies that that provides, like like, very high end security cameras for, like, police cars and things like that. And he writes

so that you don't hit that that just in time, it's actually

Okay. That makes sense. Man, it's

That's super interesting. Man, I'm I'm learning a lot about how, like, the lower level stuff works. I really appreciate that.

underneath, right, like, it's eventually, it it it goes down to ones and zeros and and runs on the silicon.

that would suck in Pnpm, That actually boots Windows XP like the real thing. It's like a kernel in JavaScript.

like, I don't know, the c code that runs on the camera,

Since we're now the runtime makes assumptions This 1 is maybe not executed so frequently. Let's throw it away and then recompile it again. And now

And assembly is also an abstraction over like

that takes the the Java bytecode.

So Node also has 1 because it's inside V8 and similar Deno uses the same thing. So adjusting time compiler Yeah. I Node, it's- It's wild. Amazing how this whole thing is, you know, you know, card house or whatever holds together. Yeah. Yeah. Because it's so many labor. Anything works. I was talking to a guy who works at, So you can focus on building There was pretty much Rust, and I had some prior experience. And I could be somewhat then I will have faster startup by just by doing that. So if if Node gets removed a bunch of APIs, quite honestly, it will be much faster. Yeah. But they don't know, you know, they can't do that because they people are dependent on those APIs.

JavaScript benchmark, comparing 1 JavaScript, It's 1 of the most popular ones. Right? There's a lot of I don't think I can share even though all the the statistics, but I know I mean, there there there's basically They don't know what APIs And here we are writing people struggling it seems a bit risky. They get, like, undefined, and the whole thing crashes.

Wes talked to,

the runtime itself. So Java has a Java virtual machine. It's a runtime.

very likely to happen because the compiler

and then you can tell

the instruction set of the CPU because assembly is also, modern apps or app mod technologies. So that means serverless and container technologies. And of course, versus the Scott, right? So the more resources you give it, and this is I mean, I thought I knew streams. Then I started looking at the specification under the hood stuff. And it's like, woah, this is really complex. So it's like it has a flowing mode and a pause Node, that it depends on how much should be read into the buffer and has an internal buffer that gets full. And, yeah, depending on how much you read, that buffer will be it's it's extremely complicated.

for both JavaScript So I've been, not as

But this happens automatically very well integrate with AWS Lambda. So that's the like 1 of the major benefits that you can get quickly going and start building Scott it's a very versatile So it's less than 1 megabyte in size.

I I fought as much with with the compiler as it helped me or with the tooling. So the learning curve is very steep.

that for partners and customers that work with Amazon Technologies.

I've heard that a lot of them are actually Java And then just make it architecture or specifically LLRT has has designed its APIs

you always it's a trade off. Right? You always give up some stuff in order to make it easy to use. Right? And here Node we are, me and Scott, at the end of the line, just using JavaScript because that's that's the best our little brains can do. But

the fact that the Rust for have this low memory configuration. The lowest 1 is actually the simplest way of making things faster is to not do as much. So if I can move our APIs because you have to recompile in order to change something. And also keeping costs low

at least productive.

and, he's talking about, like, just processing. He's like, we do he's like, we don't even talk milliseconds in the office. You Node? It's nanoseconds here. And here I am on the opposite end being like, oh, we have WebRTC.

super fast raw sockets. Right? So they're not running behind the TLS connection. So that means that they have to be unsecure. But you can expose, like, a simple HTTP server ESLint internally and have a TLS in another layer if you want to.

similar performance, a bit,

it will be get compiled away.

I wanna ask 1 more thing about this is wild to me that we now

executed, the lower level APIs doesn't work really Wes, and they aren't properly tested. So it's still a Yeah. Very experimental work in progress. The streams API is huge. And, like, those you can if you start reading a stream, it's locked, serverless the runtime that only works for those CPUs

is like a cold path. So it's not for Cannot complain. It's about

Mhmm.

a compiled language. That was basically a requirement as a foundation. Right? I wanted to have a compiled language So we were quite early back then. It was at CGI, But to be honest, it's quite complicated to get that right, right? To to have service. So it's it basically works by letting customers special. So I did that for for a couple of years and we focused on building apps Wes that was, you know, the big thing to do, writing thing. I don't know why, but

compiled languages

the,

being held together by Yeah. So so yeah, in runtimes started working on that or the team hasn't started working on that specifically.

This is 1 of the reason access error, right? You're trying to read or write to memory And I've Deno some Rust projects before. This is not my 1st Rust projects. But initially, when I started with Rust, like, almost I think everybody else struggles a lot. But then it Lambda is is, is a very capable rather than focus on building like nondifferentiating predominantly with

doing hashing or something in JavaScript, I think there's room for something specific.

is exactly what it sounds. It's like a compiler that is built into No. That's that's the plan. And, also, I I forgot to answer this from your previous Wes. Some of the things that doesn't work. Right? So provide to our end users. So it will be better if we can get rid of them for all, for the best case. And in an LRRT, it takes 50. So that's like Yeah. Yeah, 1400 machine. If it's a win, or not a Windows, if it's a Linux or or Mac machine, no Windows support yet. It could be added, but that's not a priority especially

Yeah. It can be. It's not always. Yeah. Have you seen that the sorry. Better over time? Like, it gets faster? You said if it's been running, it's like a fine wine. My Node app just gets slower if they run for a while. Yeah. You got a memory leak. Yeah. Yeah. Exactly. They actually do because it's, like, a serverless or in a Lambda, I'm actually curious if if that's the most popular runtime, if if you know that. Do you know that?

talking on top of the stack. I don't know. Is there something

Go is pretty mature but it isn't as performant as So you tweak it down. And then the divided by 50 is like Wes times faster, right, for the fastest case. For the worst case, it's something like 15 times.

itself, thanks to the the just in time compiler that is constantly being running and optimizing the code. So if if, like, a Java program has been running for a really, really long time, it has world or whatever.

a WASM runtime. So in our runtime in, a runtime, and communicating between the 2.

Node, will you will you try to

still a work in progress. There's a few things in there, which is a bit weird for our use case like Otherwise, you can't, you know, download, or other things, right? It's only the low level stuff like timers and things and the event loop and all of that. That is implemented in c plus plus for Node. But you have, programming languages and and and sort of frameworks that can run on Lambda, you have a bunch of options.

target the winter CG spec, or is that too large of a thing for the the project?

That is like, with Node, This JS usually an effect of the optimization

FFmpeg or. Yeah. He even even created a Windows emulator in the browser that can actually run Windows XP in JavaScript, which is actually world be without FFmpeg? specifically tailored and built a just in time compiler JS something called a de optimization.

We

you said it Wes built with Rust. What what made you pick Rust specifically?

weird runtime, I think. Not Quick JS, unfortunately, but there's some old super advanced. It sounds advanced, but it's not really as long as it can do HTTP, you can you can put it on there. And as long as it runs on Linux, that targets a specific set of experience is absolutely massive.

like yeah.

It it's

Do you know what other

effects in in React than I did too. Right? And then Yeah. I can hardly I struggled with a set time out inside of a React hook the other day, and I was like, there's people running Windows XP in the browser.

this guy is yeah.

Yeah, yeah, definitely will do.

that is a good language for regular people to use. You know? Yeah. And now let me let me build something that is very hard so that regular people can write a JavaScript function and get some major benefit out of it. Yeah. And it can run, like, on on embedded devices or whatever. I'm I'm not sure if that was the implemented, or does that something it needs to be implemented?

So so back to back to,

with understanding,

If he sees a problem, yeah, if he sees a problem, he's like, I'm gonna do it better than anyone else. And he creates You have the whole scoping, you know, global scoping and function scopes and function execution and all of that. And you also have You have to, you know, basically everything that you import from Node. When you have an import or a require, that doesn't exist. So basically, what you have in global scope is what you have inside of of the JS engine.

he just he likes doing these crazy challenges that nobody else has done before, Amazon or AWS product right now. Node. It's still like more of an experimental package. So we're Beautiful weather here in in the Wes Coast of Sweden.

His face.

LRT,

why did he make this? Why did he make Quick JS? Was there a certain problem, or did he just think, I wonder if I can write JavaScript in

c?

use cases, like or I've been trying to get Fabrice on the show because I I wanna ask him. But I'm curious if you know as well as, like, Awesome. Thank you so much for coming on. Really appreciate all your time. This was fantastic, and I learned a lot. Yeah. Thanks so much, Richard. Awesome.

Yeah.

the majority of all APIs are in in JS, yeah, PC emulator in JavaScript. JS Linux, that's awesome. I mean, you should put that in the show notes.

the engine. So you can actually use it kind of standalone with the console Obviously, that comes with a lot of, like, more guarantees about, you know, SLAs and support mechanisms and structures hybrid in between. Right? So so the streams API is obviously a a must.

a million other things. Like, just just a genius. I remember

for the same use case. So all of those APIs

in Node for comparison, and also Deno uses V8.

But they also tailored Yeah.

You you don't care about that as much. Of course startup is important but not really as as important as it is for that long run long running sustained performance. So they're all built around that premise. That has some implications when running on startup. So I would rather save the compute and throw away the old compiled Node. And this is really expensive.

And the developer behind Cook JS is Fabrice Ballard, who is also the author of

the code that is running inside of V 8 or JavaScript core and if I can create runtime has to struggle in trying to figure out, okay, what what what method should I keep in my compiled cache?

And V8,

which kind of you know, if you remove some of the the features of the v 8 engine, to be able to use that. So you can run FFmpeg in Lambda with LLRT and get low latency. Right? You can even stream stream the output from Stream from that. Yeah. From SSmpeg encoding. Right? You can do that. Yeah.

kind of types with So running that with Node, with node Wes here, you get, most of the APIs are actually implemented in JavaScript.

like a just in time compiler, I'm sacrificing precious CPU, and it Bos Yeah. Costs money, and it takes time.

longer running tasks or they have best performance over things running for a longer time because they were built for web browsers.

there's Node, used for optimizing And depending on what arguments are being passed to it and how it's executed. So say there's like an if statement

and also some of them, And a lot of customers That being said, why you see sometimes in Node, you see fluctuating response times for doing exactly the same thing. You can't figure out why, oh, why does this take,

In a couple of ways of how that is done, real value in your product experience it like to your point, would make sense here since And it Node evolving all the time. And there's a huge I think I'm a lot more productive now than I was when I started the project because I spent so much time in it. But it's still I think it's a great language.

Like, we have a list of questions to ask you. And every single time I'm about to ask a question, you answer it already. So that's it's pretty amazing here. Just wanted to shout that out. Awesome.

the JavaScript language, which is things like variables and functions and a single weight.

anything other than that. So there's no fetch. There's no console even, Yeah. He's absolutely a geek, so I like geeking out in things, in technology.

It comes from Chrome.

in this thing? And it's amazing that You know? I can hook up my webcam and and get access to it in the browser. And it's crazy that they're they're so far apart from each other.

But all of these And ideally, I wanted to to have use Rust, 128 megabytes.

it is. Right? Like so they implement

to a lot of APIs. So you have, like you said, the underlying, like, around a very simple thing but also Wes common thing. It's put some data onto our serverless database, thing here

reason for the performance gains is basically this super lightweight engine using built in C called QuickJS.

but that's not being used in LLRT. So we're only using the engine because quick JS runtime is very lightweight and it's not really tailored Titanium if memory serves right, or, But I think it should be in there because it's something that people use a lot. We do have some basic socket support. So we have some TLS socket support, obviously with the lack of a full stream API that makes it a bit the streams. You can't, like, doesn't know always, right? that is happening. And it's especially prominent if you have this is a completely, you know, new project built from the ground up. So I don't have to what things, the node does this Lambda function have Wes would make it slightly, slightly, slightly slower for every 1 of those APIs that we implement. But it also naturally takes time to implement them. But that being said, we do support a lot of APIs. We support which means that they Yarn really, really fast. In comparison with that produces outputs TypeScript? I don't think it is. Like but say that that TypeScript is on top of a stack, then TypeScript low resources or few resources because it can't keep everything compiled and optimized.

are now implemented in Rust, and comparing that with something else. So running that run it on their local machine in something else, and it will behave and work the same way.

They're also very capable.

there's a bunch of techniques being applied. But the the main is actually also a small runtime that it exists

Wes

so a very recent standard, ECMA standard.

20, 30, or even 40 megabytes.

And then it's it's your job then to implement

nothing. No sockets, Deno, So you you mean you have nowadays, you're would have horrible performance And I think a lot of those C plus plus and other compilers also does Wes, and you can give it hints and say that this for a cold start, as node. So when we have like JS read file,

FFmpeg

a while ago when when they started started talking about QuickJS, I'm like, oh, it's kinda interesting. Like, maybe you can throw it on embedded device. But, like, is it really gonna, like are they really gonna implement all of JavaScript of course. No custom APIs, in LLRT. You know? Like, whereas, like like, Bun and Deno have their own there's no standard FS API. Right? So Bun and Deno have an implant to their own. Node has their own,

I don't know if if we can if we should go go into that, but

amongst

most of the things that you need so you can react web web assembly functions, right, executing larger files than than what you have in memory if you always read it into memory. Right? So this is something that we're working on. But we try to find like a sensible approach That basically Vercel lucky would would require us to embed, like, And can in in hindsight, you could argue if if the the product or the the tooling was mature and everything, but, is transpiled into JavaScript. So that's kind of a semi abstraction. I don't know. Right? And then you have JavaScript that sits on top required you to have a very I built some tooling around that to try to break them apart into more if I can remove a bunch of things and create a custom runtime

transformer functions or read streams, write streams, and beam pipes in between and everything. So, ideally, that should be native, but it might not be feasible in implementing that huge API, runtimes and languages that weren't specifically designed building with Node? It took 200 milliseconds this time, but only 50 next time Wes is happening.

III made, like, a semi light version of it, Minecraft uses this inside of their TypeScript DynamoDB. We have a a multi column key value store called DynamoDB run code, outdated. They had a bit of legacy. So so,

And even understanding how it works.

the person who has to implement that.

AWS, I hope. It's yes. It is it is an AWS. It's not a shell. But Right? Yeah. Okay. I'm working. No. It's not a media encoder. It's why why am I forgetting the name of this? It's running native stuff on inside of JavaScript?

Yeah.

limitations for file size, but it, the trade off was worth it. What about WebSockets? Is that

For sure. Yeah. That that would be great. We Wes have to run it in.

some features from streams. We we embed that JS architecture.

on data and things like that from child processes because I think it's really important And building tooling can involve everything from making their life easier to making their It directly translated to increase revenue which was also very fun and challenging, Yeah. But it's it's a ton of stuff actually because what we see most customers doing inside of Lambda, they they write ESLint integration code or or sometimes referred as glue code. They do some data transformation.

And you can imagine that wouldn't run very well without the just in time compiler if you have a lot of, you know, bytes flowing through, than than the QuickJS. Right? So so just just that in itself will contribute a lot to to the the fast startup.

a complete or, Scott lived running instances that quickly spin up and and, like you said, Wes, Yeah. Yeah. It's it's a similar with, like, stock exchange. Right? When they talk about if they have a millisecond of latency, people lose a ton of money, right? They have the They even move like the server run my code of the interpreter, so meaning that runs straight off the source code instead.

So the Wes streams API, picking a compiled language or or like a foundation for this project. I wanna have it to compile, to be super fast because performance is key, right? And I want it to be mature. So they kind of ruled out like sig.

clunky to work with. But we have TLS,

I think it is.

not TLS sockets, but actually, how much resources you can put it on there. Right? But the thing is that these languages and frameworks, because it's I think it's a bit nicer to use asyncawait rather than than callbacks yeah.

It's not super complicated to add in TLS, but that kind of Node on top of speed. So there's a lot of things that depend on having

You can actually pull in, like, the streams from Node. It's actually a separate package on Npm that you can use for the same implementation that Node uses.

and you have to t a stream if you wanna be able to to read it and, like, whoo, I do not envy

It might be a bit too complicated. So we we might have to find, basically my entire career. So I started out

WASM. WASM. We're running it inside of WASM, and there's there's a performance hit there. And there's also

for child processes, not with old features.

8, 000 lines of of JS and a lot of edge cases.

What do we run it in right now?

Yeah.

to load. But if you're if you're talking about paying by the millisecond,

this can be just charge when you don't need a compute. You don't need basically the CPU to do to do a network call. Right? That that is in practice. But in reality, it also consumes some.

not the design either. It's meant to complement them where you want to have this really low latency.

you I'm sure there's customers at Amazon who are salivating at this being like, you're telling me we could cut our bill by 3

versus 48. Right? And

I'm very impressed on how far they got all in all of these projects.

are being used. So I have an advantage here. I can say I always support this. And if it doesn't work for you, then you can use Node.

Yeah. Yeah. No kidding. The benchmarks

instead of JavaScript.

inside of Lambda, it takes around we use the same signature

which is a compiled language that is also very optimal and very fast I have not Whereas if you know I guess you know what it is. To say then, QuickJS ESLint for a sick pick. I mean, it's, or everything. Can I say everything that the the Lord, features that we haven't seen anywhere else.

SIMD, Maybe you shut it down when you created you know, you had to patch it or, deploy an update or anything like that. You did that in the middle of the night when there was no users, and then you started it up again and pretty much left it alone. Right? So the concept of building for that that,

briefly explained before is that you have a limited API.

that might not seem like a lot. I guess a 100 milliseconds is is a lot when you're talking about somebody sitting there waiting for the page

Wes you're really splitting hairs there paying by compute. I'm sure they could, save quite a bit of money. Is is that true? I'm just guessing here. Yeah. And it's it's actually even more more so. So if if if you look at the there's benchmarks on the repo doing

And, you know, what's the trade off?

that is single instruction, multiple data, very advanced like low level CPU instructions that can work effectively

in in the GitHub repo, at least, are are really pretty wild, especially for the cold starts like you mentioned. Yep. It's when compared to Node, it's it's shocking.

And to be honest, right, then And this is kind of you know, closer to the exchange because But it's actually been used. I think

that that node has. Right? So it's it's I've cheated a bit here. Right? I can build something from scratch, but I can also tailor

a cold start on a HTTP request is a 155

why

an okay response.

like What if those assumptions are untrue?

Yeah. You have a very limited API. Right? And you have to savings.

1400 milliseconds the promise of that.

so fast.

milliseconds

And that's We have bunch of different services that, all of these really advanced availability, or anything like that. But we we handle that for you. So there are too many sick pics there, right? So build Rust arguably. And it also uses a garbage collector. I don't wanna have that because it can add pulses. Right? I wanna have consistent fast performance.

But I I couldn't tell right now if that if that will happen. Hopefully, it will. And I I stay very positive. And Wes we you know, we are discussing this obviously internally as well Because we've seen that there is obviously a need. People So configuring memory low, So the runtime itself is actually implemented in Rust. And it uses

you know, product at some ESLint, or is is it just an experiment? You know? Yeah. I sure hope so. It's a bit too early to tell right now.

reaches a bit more maturity, But it doesn't have especially because you have you know, you wanna keep the memory low.

about the stuff that's a little deeper than clicking buttons and making loops. You know? So that's been fantastic.

And by the way, LRT also works on your powerful machines that have all of its, in most cases, this is not So The other thing that is currently not working very well is streams.

promoted to be, like, a an official AWS, I see. Richard, I gotta say, you've been hitting our show note questions.

Man, I feel like my my brain grew like 3 sizes in this conversation.

Sanity around it, an ecosystem, an Npm or JSR, and all of the innovation that is happening, I think it's fantastic. So, use some of the Yep. Yeah. That that that's about it. And it's you know, the great thing about them is that you're only paying for the compute that is consumed. And it's not like you're paying for the machine, underlying machine that is consuming CPU resources or anything like that. It's actually some data in an object storage or whatever. So it's kind of the events driven architecture It you can use, code that v 8 has stored in memory, you can dump that to a file and then you can restore that like next time you you start the program. So that will save a bunch of time not having to analyze and try to figure out doing hashing and things, And this kind of also works as It's a huge sick pic. Or sort of benefit. Actually do that. There's, like, a compiled cache in v 8. So you can you can And these Lambda functions or aggregate data from different sources.

So do you do you see a future where this will be

have been, specifically

shameless plug on people bashing on JS.

the JS Tolinski, Yeah.

Yeah. Let's move into the last section that we have here, which JS,

winter CG

once

I Wes listening to listening to too many of your shows. Can we put you on the spot for a sick pick?

I don't know, So this is sometimes referred to as a cold Scott. When it needs to provision the resources again, it needs to download your Node, that you co package that you published through Lambda, right? It needs to download that, the reason why. I don't actually know. I think maybe kind of loses up a bit.

No. Beautiful. I like it. I agree too.

Thank you, guys. See you. See you in the next 1.

tools around the language that wasn't purposely built for for running on the web or or not running on the web, but running on back ends or running on whatever. And I think that's a bit Shameless plug, I mean, people loved to use the language. They're really productive in it. I think it's great. It's easy to learn, write, and read.

I was very I'm always very interested to learn

really receptive. And, rather, you know, I could be affected with.

as well. Yeah. But, Node, Wes kind of also

sick picks and shameless plugs. Not sure if you came prepared with either of those. Oh, no. I didn't come prepared for that. That's all.

and and you're just simply just paying for the amount of time that you're using, and then they I I don't know if you guys use the word spin down, but, like, you've

to an API in API gateway, you can use a Lambda function to have customized code to have that authorization.

hitting up an endpoint, Wes technologies Lambda. Right? Mhmm. Exactly.

And Wes, what is it? Yeah.

of their end customers better.

the reason why I came up with this idea internally.

Yeah. Well, I'll just give 1 quick example for anyone listening who's like, I'm not sure, like like, what that might be JS, like like, imagine you

my job with you you build small functions and then you can expose them to the public internet or react on events.

LLRT

around running their code. So Node does this, Deno does this,

why is Amazon building yet another JavaScript runtime?

as the load increases or as they process more events, they simply scale automatically.

the it sort of closes out, and now now that computer is no longer running. Right?

trying to, dependency.

specifically applicable to that that context?

And just take a step back and and, talk about, you know, what it is and and the rationale around it is That may might be out of out of scope and a bit too complicated here.

a sort of an extension it for serverless environments. I know what CPUs it's running on. I can enable features inside of

So

you had an application, which at the end of the month, it would email a PDF of, like, a receipt to every single 1 of your customers. Right? Like, the old way to do it, it would be

I did a whole talk. I think I counted up to to 9 of them,

build something first, if you look at that COMFAT metrics, it's a bit you know, it has a few exclamation marks.

while you're waiting for it. With with, like, a with a Lambda or serverless, you can

can all run different programming languages. And the great thing about them is that you don't have to worry about scaling or you don't have to worry about keeping them up and running, or resilience, or, by design 25 degrees Celsius. Sunny's shining. You know? rather than spending a lot of time on on building complex infrastructure.

have way more JavaScript runtimes

puts something in a database or if someone puts It's being used, like, here and there. And then nowadays, they even run, like, JavaScript on in space. Right? The James Webb Space Telescope run some it really depends on what you're trying to build. Or So it's like 40 times more complicated people can pick and choose, right? So they can get more guarantees on their workloads. They can use LLRT when running on Lambda. They could put the same thing, like the parameters sent to the function It's huge. It's like it's enormous. It's a huge API.

basically just, like, let that thing sleep.

So react on things happening inside of database platform. Someone and others.

Yeah. In regards to LLRT, like, what makes

to a lot of other services. So if you want to add capabilities, for instance you want to add some custom authorization adhere to broad sense of understanding of different technologies.

a nano service. Right? If you if you take your microservice and break it down into even smaller pieces,

Yeah. Sure.

you would have a server running that makes those PDFs, and and you might have some, like, cron job where it takes

on the the server. You know? Like, we've obviously had Node, and then we've got Bun and Deno and and Cloudflare Workers. And

service.

you could spin up a 1, 000 of them and very quickly do all of that work,

glue.

3 days to generate all of those and and send them out. But with and then you're paying for, like, 28 days worth of server just to sort of sit there,

posting some data or requesting some data via get Wes or whatever. And then you can respond to that by a Lambda function and it will expose an API. This is a very common pattern as well. So it's not only also quite mature. I think it's been around for like 10 years Node. So it has a huge ecosystem. So that was some of the some of the kind of the ticks I wanted to have Wes we're picking

and that was before Amazon came out with this LLRT.

level. Right? So as soon as you get an an event in and that event can be something like Wes described here where, you know, at the end of the month, you need to create a PDF or whatever. But it can also be an event that reacts to like an HTTP call that is a synchronous event. So someone, Wes you can store data in a service way as well. So it works really well with with Lambda. So the example just puts the event that gets sent into the Lambda function.

yeah. To set up a scalable architectures.

of technologies,

So had their own demands and their own requirements. The user experience was the center around everything in e commerce, right? Because it directly translates it's allowed to have because it's a linear relationship between

Sometimes that can even involve building run times.

since the frameworks that we used to do at the side of the company that we used to work with with with on the side of the company was was

I love it. Love it. So

Welcome to the show, Richard. How's it going, man?

let's maybe just give if you could give us a background on, you know, who you are, what you've been doing, and and how long you've been doing it.

So without further ado, let's get to the interview.

to the revenue. So if you can improve the user experience for our customer's website, I'm a you started a server.

I wanna call it developer slash architect from Sweden.

ecosystem. Like, all of the packages were being made, and it's it's totally different space now.

Awesome.

Yeah. So we're here to talk about LRT, which is Amazon is building a new JavaScript runtime.

I'm at AWS building So it can be more efficient.

That was way, way before React even existed or React Native or something like that.

other types of APIs? Bag. Yeah. It's a it's a lot of technology. It's since I came from this kind of, like, diverse background where every, basically, every customer had different requirements and they Yeah. Sometimes yeah. Sometimes we also had projects that we took over or improved or something like that. So it kind of required this entire journey.

Then a transition to, the e commerce industry. So also very fun because you get to tackle a lot of challenges because basically every customer So the major reason is he has to be he has to be 1 of the world's, if not the world's best like programmer.

And

Yeah. So is it most mostly JavaScript that you're working on at AWS or,

maybe before we get into any of the LLRT stuff,

I don't even know if they're still around, but we use that technology to build like cross platform.

with Node, I think, almost 10 years ago at my basically my first real job at university.

quite, help you write safe code and safe that is being memory safe. So a lot of these really high performance a huge deal. And you might So it typically occurs So when when you wanna use like an API that is not supported, about microservice something that was kind of a lighter runtime, pieces. But you should not waste effort. That's a very important thing, right? It should be massage the data or remove remove Python was created in the 90s.

Thank you so much, Scott. Thanks for having me. It's it's it's great.

So you Node, Java was created in the 90s.

a sort of a cross platform approach, kind of the same thing. Those are it analyzes and that feature of having Scott fast startup, it didn't really exist. So they hadn't Mhmm.

you don't see the source code anymore in a similar way.

they are built for general purpose.

and provided by AWS I don't Node. Maybe people bashing on JavaScript too much, right, because they say that, Node, it's not like performance, it's not that good, or why are people trying to build to Node to kind of go crazy on scope, but also have good performance.

when we talked about these different of something in the browser that has a runtime that is written probably in C or C plus plus which is an abstraction over assembly

syntax.

the concept of of serverless and these ephemeral is executed in Node,

runtimes, as we call them.

I was just gonna get to that, and I'm glad you're Node. Good. Cool. Yeah. Yes. So it's a good, cool segue. So Yeah. I oh, that's a very good question. This this guy is a magician. I I think Yeah. It's crazy actually when you think about that. Like, how many abstraction you know, layers of abstraction that there JS actually.

But they all suffer from this sort of legacy and also Yeah. I think it needs to be implemented. I'm not sure if it's in the spec or not.

more of a compiled approach. But then you sacrifice on flexibility You can't rely on us, you know, and promises.

Let's talk real quick about Sentry, the application monitoring software for your application.

There's a value of on how the code is being executed.

But the exception can kind of be drawn here with compiled languages versus interpreted languages,

spins down again and are discarded, that concept didn't really exist.

or languages that has and building mobile apps. And we used JavaScript to build them cross platform.

applications, automatically.

is breaking or why it is slow. All kinds of great information just like some magic glasses for figuring out what's going on with your application. Check it out. Century.ioforward/

So you can run Java applications. You can run Python applications, Node. Js applications, .net This is a special virtual machine called Firecracker.

like I explained before here, node to dump the the v 8 compiled cache. So, like, all of the compiled infrastructure that you just Node. That is that kind of that, you know, accidental complexity, Well, then it has to recompile you don't want to What can happen though, and that often happens Sure. So, my name is Richard Davison.

and insights into why your application

You can also run custom runtimes. This means that you can build your own. So as long as it's a Linux program, c plus plus or or or c.

Rust or C plus plus or C, even Go, right, to have this It's partially supported.

and a whole heap of other predefined Not everybody does that, but I'm The trade off is is what I thing. The most common 1 of those things is if you have a memory Pretty weak 1, but yeah.

The general purpose programming languages and frameworks that exist, they they haven't been designed around that premise. Right? So they they weren't optimized for fast startup. They were optimized for long running tasks and have good performance when when running over a longer period of time. That's not true for every runtime in in, c plus plus or C or Bos, or anything like that, are the exceptions.

being able to kind of So it's not it's not actually, like, an a an official more complete streams API that than than we support right now. Wow.

except for going to Then you have the actual, like, machine code being executed, It has this just in time compiler Oh, yeah. Sorry. Yeah. So Win for CG is is JS like the the long term goal. Ideally, we could meet meet it this year. It depends that the spec is also

So that's kind of the rationale, I think, 2 and a half years ago, something like that. So a recruiter reached out to me and ended up here. So now And finally, I ended up at AWS. I was recruited, much, much faster over time. Hence, the Scott exchanges uses these really has Node? super prominent challenge, right, or super prominent problem because it mainly exists during the development lifecycle.

prominent to Fast Startup

Right now, people who run Node. Js in,

well or at all in, LLRT?

What are some things that would not work

for LRT. Ideally Wes want, but we don't want them and also arrays and array buffers and all of those APIs that exist Node, like, the primitive types, your source code is also what you execute. Right? You run straight off the interpreter as it solves.

most of the things that you need from FS promises. But we don't support the regular callback FS module, right? So you would have to Okay, I would have to change my code to use FS promises, which I think arguably that you should, Scott of data at the same time. I can also the time it's spent on processing your event. So it's very use JavaScript when you haven't invoked them in a while. So when you haven't When you open YouTube or open a website, you're not gonna stay in it for 1 150 milliseconds and then close the tab, right? evict things from its compiled cache. So that's often what we see inside of Lambda, They all have a bunch of legacy, but and back then in 90s, and organizationally But I saw that there was a lack I don't quite remember. But we were very early on there, so we kind of jumped straight into to JavaScript.

you can switch back to Node, he has a website that is basically just like a text file of all of these amazing projects.

the node compat. You said, like, this obviously is not going to be a node replacement. It's good for things that it's good for.

implementing that. Then you can switch back to node or LLRT that would would not perform very well because it's built around that has not been Yeah. Yeah, no file system, Wes.

move that out to to a break that apart into 2 separate that you have to add to your application.

it's actually pretty good in terms of, like, what APIs are are are supported.

Yeah. So still, quite fun. So I started doing that for a couple of years. Yeah. It was very challenging, of course, because it did it's quite different back then than as opposed to how it is now. It it was wild back then. There was no, like,

So when we say that Wes support FS, right, we're not supporting the full I don't remember even if there was, like, Npm or not. I can't remember. I think that Wes even may have been even before NPM. I don't know. But, yeah, it was pretty wild. It didn't ESLint. Promises or anything like that. I think it Wes, like, third party. Promises started arriving, but it was not async wait and not any anything that was built into it's still required. So as a separate piece of we call these functions.

downsides So no

with the same specification and you haven't wasted any effort. And that's just a major decision that we took early because we don't want people building API, that 8, 000 line JS, but not everything works. Like, you can't extend,

But if you look at the compat matrix on the docs,

and, like, you're not gonna come around and make a a 5th API that we need to do.

runtime for JavaScript that they're working on over at Amazon.

We get into deep all about the new LORT

Welcome to Syntax. In this supper club, we're talking to Richard Davidson.

how he's putting this together, what you can use it for, what are the ideal use cases for serverless in general, and just all the ins and outs about building a runtime

improvement in cold start time. So, we get into all of the details about how this thing is able to be so incredibly fast,

for a serverless platform.

This thing is crazy fast. In in some cases, it can be over a 30 times