Cloudflare Workers Are Next Level With Rita Kozlov And Brendan Irvine-Broque

TCP sockets API called Connect that we've been working on.

But may need to be tweaked a little bit or have a different context when you're using them as on them on the server.

We're trying to make this the API that is shared across Node and Deno and getting the Cool community on board and hearing people's feedback along the way,

And so Probably all we all benefit from there being some diversity in the ecosystem.

I I'd look at this list of of companies. I I see a bunch of big, big, big, and small companies, and I just, like, wonder,

Why don't we just do that?

on over the past year as kind of an example of where the This group is having an impact, which is the, kind of outside in refractors, where you can kind of take the outermost layer of an application

And so that group meets regularly. If anybody's listening and wants to get involved, it's Open to anybody.

or,

And so what we we did was we kind of worked within the community of Enter CG to say, okay, we're gonna support this on Cloudflare Workers, But that's not that's not where our kind of work ends.

developers is open to anybody, the Cloudflare Q's product that's deeply integrated with workers.

I would call out 1 API we've been working,

That that's awesome. And that really paved the way for, isn't in that case. It's kind of like a

what is the the winter c g. Right? Like, so do you wanna explain what winter c g is and and Cloudflare's part of it? Yeah. So winter c g is,

And so there wasn't really a a standard for us to lean on.

And the work workers' runtime and others that are, as you talked about earlier, kind of emerging right now,

making,

when we were looking at the ecosystem, you know, There's an API for this in Node. There's an API for this in Deno.

and also voice kind of representing that group of people when we talk about,

TCP connections from workers, to be able to connect directly to databases.

And Winter CG is kind of the the way that we make sure that we're listening and getting feedback and kind of working out in the open. That's nice.

We really welcome external participation. We I think that group would love to hear more, full stack applications on Cloudflare,

server side JavaScript run times all share, so across Deno CPU time per invitation.

who are all working towards,

There's been past attempts to make the an API work in browsers that haven't quite succeeded.

And since we wanted to adhere to WEB standards, we did not want to make up something proprietary that was Clafler special sauce a lot of people,

And this magical thing happened, which is that Developers, and we've realized that what you could actually do once you were deploying software at this scale in every single data center was so much more full than being able to customize a CDN.

we can make it slower. Right?

I think, you know, to run a runtime, and sometimes it is the case that you do need your own specialized runtime.

is how we always wanted it to be, but had all these constraints getting in the way of it. Wow. So,

you kind of mentioned this earlier. Right? A lot of folks know Cloudflare for things like CDN, Type in dot run and there you go.

let's start. Like, What is what's what is Cloudflare Workers and what are they useful for?

wrote an if statement somewhere that's like,

and enabled all of these different use cases. And so Well, workers in a nutshell are a way to run serverless code at the edge. I the way that we think about it is a completely different paradigm for building applications

DNS, DDoS, And so,

Yeah. And so we started looking at different approaches. How can we do this? And, you know, Oh.

Your server or your CDN

the rule of things is that the 0 to 90% Yeah.

How does something go from an a big idea like that to actual

So

A few years ago, we started running into this problem that I think every single startup hits. Right?

Does that take a long time? So from the moment that we started thinking about it to it being live, we

You could actually build applications that felt like they were scalable from 0 to To Infiniti, we're instantly really, really performant and fast because they ran really close to the end user,

I believe.

And That's just not acceptable And so the way that we prefer to describe it is generally, you know, a global network of while accessing that data. And the way that we do that is through one of our new products called Hyperdrive,

We had a very manual process for it. There was An engineer that went behind the scenes that Okay. I don't like Accruing things, and so I'm very picky about which ones I choose.

That's where we started. And

And I think that this story, this origin story is really interesting and really crucial to kind of understanding workers

but that's so much more

execution?

So we started thinking about how do we make this possible? And actually,

when we, And we've partnered with Hugging Face, so our goal is to start to support as many of the models in their Catalog as well as possible. Wow. Yeah. That that's really cool. We we want to do and

the past 5 or 6 years. Right? So Yeah. We had The initial prototype, I think running within a few months and released it, the the original version of it back in 2017, Exactly. It's not spinning up puppeteer. I think this is actually a perfect example of your question before about Trade offs. Right? I never thought about that. That's great. If someone at Wix is listening to this, I promise I can make your life a lot easier.

if, you know, it's syntax FM, then do this thing.

takes Mhmm. Not that long, relatively speaking. Right? And then the last 10% has been

And so we wanted to make it possible for anyone to be able to service themselves and be able to do this on their own.

say, hold on, let me let me figure something out, you know? Like, let me figure out, What country you're from? And if that's the case, let me redirect you. Or,

But that introduces this problem of, okay, now you have a 15 second cold start maybe.

when you're trying to make things faster.

In this way, that There's no way that by making it customizable,

it was, like, really interesting how you said, oh, well, maybe we could just run

the the user and the actual end result, And you can sort of jump in the middle, that's how I like to describe it, and and easy to query and probably cheap. I don't know. I didn't look at the pricing, but I assume so. That is one of our Staples. Yeah. So

Of you want to onboard larger customers, but they have So many requirements for you, and there are so many customizations that you have to make. Right? So it started out with if I want to add a header, if I want to cache very lightweight We have,

You were just talking about standards. Right? We turned to the browser,

And we thought about, well, This is the most battle tested software out there, the v eight engine. What if we deployed it to Cloudflare's network edge

clients cut at the edge here.

it probably

I've often talked about Cloudflare workers

And obviously we knew that that wasn't gonna be scalable.

let me render out something a little bit different.

or we do so much more than that now, and

because this Came with some requirements and constraints. Right? So if we are a CDN and one of the main value props that we provide is that we make your site faster,

on like there's kind of like 2 ways you can use it. Like, first, you can use them as

CPU heavy. Mhmm. Yeah. We so we just rolled out a news and tax website. Yeah. And 2 things that were pretty big is, 1, we,

And for context for if you're you know, you haven't thought about how much CPU time your worker might be using, you know, most workers are using, you know, somewhere around

object instantiation We appreciate it. Yeah. I got your job. Okay. Yeah. Thank you.

In AWS, But,

increase the limit to several seconds or or maybe up to 30 use cases? Like, what were you seeing from your

And

By cashing your queries. So would it be safe to say this is like a planet scale like product?

And then

The initial connection to the worker is great, but

Quite right for our model.

CPU time is is the time that's the

data is a big investment for us. We are constantly rolling out. We rolled out d one, which is our own database. We have r two or object store, Ingvar shows up and goes,

works across different things, so that you as a developer, you're not constantly facedown with, Oh, I want to use this library. Is it going to work in runtime X or Y or Z? And I think Cloudflare was incorrect me if I'm wrong, but Cloudflare was very early to this,

is not going to be optimal neither for your performance nor for our memory footprint. And so we'd rather Offer up a service or an API that makes sense within the context of the environment that we're running this in. Yeah. So I guess even And allowed customers to deploy their own code on our network.

as possible that developers are going to need. And so When we hear about something like, oh, I wanna be able to run puppeteer in a worker, realistically,

of data that the

And then I

And I'll give credit actually to, absolutely everywhere. So I thought that was pretty cool answer. Well and I think that it touches on to your point of And so cache API provides a lot of flexibility. You can do things like cache post requests.

Like a cron request, but that is the perfect use case. I I really wanna move it to Cloudflare queues because that's perfect use case for something like that. And we have our browser rendering API,

Like, that's the stuff that that that, you know, you you wrote a a for loop that is a little bit gross and takes a lot of time, or you brought in some big dependency that's doing extra work when it doesn't need to. That's the stuff that you control as a developer, a bit of a higher,

that's actually spent by the CPU

to actually do that. And I often go over the 50 milliseconds, but it's not like I don't think it's a hard limit. Right? Like, if you go over sometimes, they're not gonna just Shut her down. Right? That's correct. We we do a little massaging of the limits,

What am I Scott, who are we using? We're using PlanetScale, but there's also several others. Was it Neon? PlanetScale. That's that's a special sauce that PlanetScale does is they They do the pooling on their end, and then they'll they'll connect it to it. So you're saying hyperdrive

you guys have fat pipes between all of the different Cloudflare locations, which makes Connecting to wherever that pool is fast? Exactly. So things run through our own network and kind of we're figuring out where that pool of connection should actually live, Making sure that's in the optimal place for your worker, and you don't have to think as much about, like and if you think about so many database drivers,

large amounts

and suddenly, you know, it's spending browser standards, because there's some things that, you know, may make sense in a browser,

I went and fired up a Cloudflare Worker.

Ingvar who was on our team at the time. So when we were trying to figure out in the 1st place, okay, how do we do this? We're not using containers. We're using the V eight engine.

it's actually the most performant thing To run your worker as close as possible to that back end

A connection pool and then you can connect to it. You can hear DB. I'm assuming

Import whole existing

Then I have to parse it. I use the package called LinkDom, which is worker friendly,

or

on your behalf. We are managing

so that we're kind of On the same page so that the ecosystem of libraries across npm that we can then build something like a hano on top of. And you'll we'll see that. I think that you know, I I mean, it's there's so many opinions about this, but, like, people have thoughts about routing. I've seen so many of the arguments around rails routing. Right? Like, that's there's so many spicy topics around, like, oh, it should be file structure only based routing or, You know, I have this one weird middleware constraint that I bring to the table. Yeah. So I think that there's an that's a space where

And I was looking at that being, like, oh, Cloudflare is is doing a pipe to everybody's house, but I had nobody to tell that joke because nobody would like Nobody nobody's gonna appreciate that joke.

But one of the things that we're working on And it it's

that's not just like one one off feature for us. It's like that's how we think about scheduling on our network, is Not just, oh, we're gonna run this as close to the eyeball as possible. It's your problem. Go figure it out. But, like, we wanna make those decisions for you. And this Might not be the

Like a tough love kind of thing, I bet, being like, well, why is your your thing taking 200 milliseconds? You know? Like, Of course, we could make it larger, but it's probably better for

rather than running that worker as close as possible to the user.

and using web standards.

Constantly on their mind is GDPR and how do I meet compliance regulations.

Okay. On a per worker basis, so that you can say, Hey, I know that, you know, p 99, this should only ever be two 150 milliseconds of CPU time. I wanna set the CPU time limit a little bit lower just as a safeguard in case there's a scenario that I haven't thought about I've talked to people who are building applications where they want to be able to,

And so the other thing that we found is that it's actually much easier to take something that's running globally and scope it down.

IO or or that stuff. Yeah. Your request.

of wait time are

Is that you do sometimes We're not running Node. Right? But we do recognize that people do exactly what you guys described. Right? Which is Use a framework that relies on some random package,

I'll always give them a Cloudflare Worker URL because if anything changes, I can always I can always massage or fill in that that data at some point and

never give learners their or simply just return the fetch request from that and it will proxy it. And then I also I added in my own CSS. It made it look cool and like

10 milliseconds, like far less than 30 seconds, so it's plenty kind of per request.

there's there's always trade offs that you have to make when when you get something amazing like that.

which is called hyperdrive.

platform for databases?

The actual

people are processing not just 1 input in there, but a batch of different inputs.

there's a model for this already. It's called service worker.

your database is halfway across the world and

And so what that means is, like,

not just the infrastructure, but we are Making it so that you don't have to be responsible for that server. We're thinking about, you know, where problems like I was alluding to before about where you should Run where you should operate.

We're going to make a standardized JavaScript runtime because you basically took the

Is that a database that is Distributed,

Totally.

People always look at the 50 milliseconds like that's that's not a lot of time. So are you able to explain the whole wall time versus compute time? You don't know until you deploy the thing and and yet, like, that's that's the point where people go,

What are some use cases where you might want to Secret sauce with a worker on top of it.

And he's actually been the one really

the actual

A Cloudflare Worker only runs for I think on the free plan, it's 10 milliseconds. On a paid plan, you get up to to 50 milliseconds. And, do you have this cookie set? If so, then

You figure out why. And that's what I did in that case is I I some guy from the React team actually helped me figure out how to use the dev tools to figure out What's actually taking

a special sauce that

a 100 milliseconds? And it was that thing taking, like, 99% of it, and the rest of it was 3 milliseconds.

parsed the HTML output, and then at the other end, I gave the exact same structure To these models with the import statement,

Background processing of of work, and there's real computation there. And oftentimes, developing the set of standards that, a community of, we really care about people building open source libraries as, like, kind of our primary customer on our runtime team,

will allow you to, like, right? But it's not A Node. Js runtime, right? Right. And so, you know,

we're recording this on Halloween today, and we just are launching our new workers' pricing model, Oh. That,

I have a course that hit a recipe API.

of server side developers building applications in JavaScript,

and you're going to hit very weird consistency guarantees. So d one actually does generally live in a single point of presence today.

Combine a bunch of audio clips before we send them off for transcription and into a big buffer. Right? And we're doing that with a WASM package, FFmpeg.

queries where it can. Cool. Okay. That makes sense. Yeah. Okay. Because so we did an episode on Serverless databases,

jamming all of puppeteer into a worker

feature that we recently announced

to take, like, way too long, like 80 milliseconds or 100 milliseconds. And But it's based on web standards. So instead of you can still do middleware and all that, but like you get a proper request and response and you can use all of the

Something that that you can have you can really shape. Mhmm. Yeah. That makes a lot of sense,

Which allows you to connect to your database. It'll handle connection pulling

But, you know, you shouldn't, as a developer, have to think about, you know, where your that backend resource is. It might not even be something that you control.

but trying to make sure that there's an ecosystem of different run times that All kind of represent the shared needs

and then scale it up to additional regions. And many companies started in a region that's Not even compliant with where they're trying to be. So Excited to be here. I'm Rita. I'm senior director of product for our developer platform workers.

And I guess the jacket that they fished the fiber through is orange.

one way to think about it in your head is that we use V eight under the hood just like Chrome does, and so a worker is, in some sense, akin to a tab in Chrome.

That's a really good question.

reason about how their application works without being a distributed systems engineer.

or is it just a

don't work well in serverless because and I put them in some database right now, Zeta, but I was really excited to see that you launched a vector database because I'm sure that will be nice

And so, yes,

your bill at the end of the day, your users who are visiting your website, that a, like, a question. We took every transcript and fed it into a embedding.

to 70 milliseconds

on, again, the different use cases and what customers are going to need. And try to make it as

are open like we keep it warm as long as we can. But even just taking it, loading a page, take a screenshot is is a little bit slow, and we're still running that on a We'll make that we'll make the connection. But, yeah, yeah, that would be a really interesting use case for it. I never thought about that. So hano. Js is

And so that's a that's a kind of a responsibility that we take very seriously, but we know that it takes that trust From somebody who's building on our platform to trust us to do that for them. That's kind of the way that I would frame it. Also, like like hard limitations

once we deploy the actual thing. It's got Interesting. I never thought about the the untrusted COVID. It makes sense because browser runtimes for JavaScript now, which is wild. So,

that was too much, so I had to go to a different server server to to actually host that thing. And I've hit that a couple of times where I'm just like, ah, like, there's a There's this hard can't do it on Cloudflare and like Puppeteer is another good one that now I'm part of the Puppeteer beta. You can take screenshots with Cloudflare. Like a lot of those Pain points are getting solved.

and even just that layers on some complexity, and we're trying to To create a world where you just don't think about it, kind of in what you're pointing to about PlanetScale, where you really don't have to think about your connection pool. Yeah. Yeah. In my neighborhood, they're laying fiber.

One of my packages

That's a really tough challenge. But you can still reap the benefits of deploying your application itself on the edge Sometimes when people import large packages that just seemed like a really, really perfect fit to us.

Cloudflare Workers runs JavaScript, dog slow on,

which is designed exactly for this use case also. Oh, yeah. That's the so it Has like a puppeteer API. Right? But it's not actually puppeteer.

has much higher limits than what you were alluding to. So we actually have 30 seconds of CPU time per request.

we want to provide as many of the tools

that people talk about rather harshly as being,

There's been a couple of projects where I'm like, I'm going all Cloudflare for this thing. You know, Cloudflare stack

to to go along with some of the things that developers are often needing,

where if your worker talks to a back end and makes, you know, a couple of requests to say an API that is either yours or a third party API,

How do we handle restarts of that instance in that server so that you don't have to?

Simple as possible for anyone to be able to So the way that we think about, I would say data in general, when it comes to running applications with workers, we were the 1st to use this model.

Accurate? Are you familiar with them? So this really lets you connect to any database, even an existing one. So let's say you had a Postgres database And so to maybe break down that difference For listeners, around wall time, and or duration, what's often referred to as, like, gigabyte seconds, which is kind of a confusing metric Mhmm. Versus CPU time,

So another one I have is

The even the switch to, like, the serverless mindset is is a bit of a hump. So, like, What is the trade off that you have to give when you think about building with a worker? I think that we respect how much trust people are putting in us is the way I would frame it. It's like, mhmm, there is a trust thing, right? Like, we are taking on more responsibility Postgres database driver, and then there's a separate database driver that's the Node one, and it's this forked ecosystem, and it's impossible for All the wonderful open source maintainers who, you know, put their work into this for free to have to, like, deal with all these differences.

Then they'll distribute it all around the world. I just was thinking, like, I don't care, you know, make it fast, you know. And the same thing with the with these edge functions as well is, like, Hopefully, in a couple years,

And so I'll shout out James Snell, who folks may know from from Twitter, who's contributed a ton to Node. Js over the years,

your existing, like, relational databases. And often Then those relational databases

So one thing we've built is this feature called smart placement with workers, run time that they've experimented with, who wanted to to get involved over the course of time. So I I don't know if I'm gonna be able to solve the TikTok mystery, web standards, and that back end is in a particular location,

just opens up so many different use cases. Mhmm.

So it's important to remember that workers aren't just being used out there for

Surprise, surprise. That's the Cloudflare Worker API as well. Right? I think I can pretty safely say,

Data structure. So what I did,

I've seen the the Cloudflare d one.

There's always that triangle of of pick 2.

Yeah. So we actually,

and then I can loop over the divs and scrape the The name and the image and all that stuff, and that takes anywhere from 30 seconds to 70, 70 sorry, 30 milliseconds what does that look like so that it You ideally get as close to as a worker environment as possible. Yeah. So when you, I'm going to certainly dip into that next time I I build something.

doing actual real work,

The thing is it's not 50 milliseconds of of it running, right? Just a server. Someone will hit your thing. You can do some work and and return the data, or you can use them as like a something that sits between

everything.

the ability to do read replication

To make sure that there's no case where, like, my work will never be taking

And so for most use cases that you're going to use a database for, you actually don't want it to be live everywhere all the time.

Let's talk about large monoliths

I guess, like database products. Right?

and I employ this sucker to Cloudflare or whatever else, and they're gonna figure out all the sort of hard stuff for Yeah. And on that point of what are the hard things, we're starting to really push the bounds of, as Rita was pointing at, this the kind of scheduling requirements.

So many of the the times that we end up having,

especially for me personally because I I would have just you see that the time limits

one because your worker's always talking to it, and we know that that's gonna get you the best performance. Oh, that's that's the thing we get all the time, anytime we talk about Edge Functions, we get tweets that say, yeah, it's great until you realize

And,

That doesn't really make sense,

this flexibility started onboarding larger and larger customers with web assembly, especially if people are trying to

you know, be more standards friendly, to

And

like, when I 12 years ago when I launched my 1st product, I I launched a thing, and then I I woke up the next day to Bunch of emails from Germans, and they all said, your downloads are slow.

5, 10 full seconds of CPU time per request.

versus CPU time.

or in the EU, it's a lot easier to go, okay, my application needs to be compliant by these laws. It's much, much harder to do the inverse, which is take something that's running in 1 region

And so, that's kind of the difference between those 2 and why, you know, we really think it's the right thing for for you to to pay based on CPU time, because that's

programs that they had running somewhere else.

What for someone who's like a traditional Node. Js developer coming at this, Alright. Well, you got key value stores and then you have,

And we asked the folks like, if you wanna dabble with AI stuff, certainly check out the their workers.

incoming HTTP traffic, but also for processing, you know, batches of messages and actually doing important

really,

and a bunch of other tools and features and products. But we also recognize that data specifically is one of those things that's really, really hard to move. And as an organization,

you might even spend time re rewriting your But probably you're not going to spend the time moving your data.

and everything like that, but it will also make your data feel distributed.

which is kind of explaining.

Yeah. And, you know, one risk that comes up as you introduce higher limits is that people you're kind of subject to denial of wallet attacks or that Somebody finds some infinitely complex HTML tree that they send to your worker,

And I I had the thing on Amazon s three, and I learned You got to put CloudFront in front of it, and then they charge you for that and they charge you to download it. And,

Right? So if you want your data to always what are we gonna do? Are we gonna run node? That doesn't seem

Is that

And so what smart placement does is we understand where your worker is making requests to, and behind the scenes, we're automatically figuring out, Oh, we really should run this really close to US East

And I said, shoot. I don't wanna rerecord the video,

Provide a few different options depending

that kind of goes hand in hand with that logic,

just kind of blanket time limits. But,

sort of just, like, run

sexiest thing to think about too, but if you talk to any developer, especially in Europe, one thing that's so that you still, for things that are frequently read, get The best performance possible. Cool. And I'll mention one other

serverless

had, like, a weird load time and, like, not even network. Like, literally, there was some bug down the Required tree that was causing

Let's talk about that, the flexibility. So you get speed, you get flexibility.

very specifically in Germany,

That, we're scraping the data. That's a fetch request. That, I'm not paying for that. Right? Coming back, I have HTML,

as a developer, Why are they making me think about this? I have so many other things to worry about. So I I've I actually have not heard the Regent Earth thing. That that's super cool.

servers service worker API, which a service worker in the browser allows you You control like, if you make a request to a URL, a service worker can literally, like we say, jump in the middle and say, uh-uh, I'm I'm going to serve you back a cache or I'm going to modify the response in some way. And that's all based on request, response, fetch, add event listener.

Duration of wall time, those are things that are often out of your your control, especially if you're building an application that talks to other services that aren't yours,

then it's gotta go across the world to get your data and and come all the way back. So, oh, that's really neat. I didn't know that you had done that. Yeah, and we're trying to apply that same kind of thinking across our products, whether it's durable objects or d one or some you know, how do we manage pools of connections To databases,

So I'll give an example of, for people listening of limits that I've hit in the past. And

that. So definitely go play with our products.

or drop a review if you like this show.

fixed. Like, I I remember once I made, like, a a worker on the website, and then I wanted it locally or vice versa. Like, I wanted to push it, and they're like, well, You kind of choose 1, and that's where you work on it, right? And

So, the standard way for deploying code somewhere has always been a container. You can spin up a container.

into your AI deployment. So if you're using OpenAI and you're like, what are all of my tokens going towards?

any past Feedback when whenever we do start building our features. So Beautiful. Well, thank you so much for all your time. Really excited about this one. I think people are really gonna enjoy it. Yeah. Lots here. Thank you. This is great. Thank you for having us.

Those are great starting points and lets us have kind of an open dialogue with you and other people to chime in and support along the way. Also in our Discord,

Find that some of the best feedback comes from there, and people give great, like, reproductions.

caching, and rate limiting.

Fixed. So I appreciate all the feedback. Where where should people cloud

repro

Shout out to my bench scraper. That's that's great. I I was teaching my daughter how to use a a bench scraper the other day because she's getting into baking, and the countertop is disgusting.

give you feedback? We actually really love getting feedback on our GitHub repositories.

DK for Wrangler, the CLI.

are looking in there and or what's a part of my application

m for a full archive of all of our shows.

shout out there.

So a few weeks ago, it was Clafler's birthday week. We like to give back a bunch of presents to our community.

our database. So you can use that alongside

for the workers' runtime and Cloudflare/workers

Head on over to syntax.f

for building, you know, all of the Chat apps and rag use cases that we're seeing blowing up right now.

to get observability,

we have a great community there both to ask any questions that you have. Someone will answer them pretty promptly and also provide feedback On our products, our PMs pretty actively

I'm really, really excited about them. Oh, that's awesome. I forgot about the vectorized thing as well. We had built

For your AI, you can use our AI gateway

Really, really great tool for demystifying

part ways? You already touched on it, but the AI announcements that we made recently, we're really, really excited about.

So, Cloudflare /workerd But I what I would what I would say is, like, you know, met with a bunch of people who are who have worked with Wasm, but also people who are just developers who are used to writing Python or Ruby or any other language We really wanna make that a a great path for people. I was gonna say, what we often

sharing kind of their feedback about our runtime,

of what other providers you're using

And don't forget to subscribe in your podcast player

and so we really are listening and Value everybody's time they put into to sharing where they want our platform to go. Yeah. I've I've tweeted about several Wrangler things, and they have all been or you had a MySQL database living out somewhere, you can use HyperDrive on top of that, and it's handling the Serverless connection pooling and caching And it'll actually pull in, based on, you know, compatibility dates,

And we have all these vectors now, Web APIs that have entered JavaScript in the last couple of years.

this year, we announced 3 things in the AI space. First is workers AI, which enables

And regardless

And I was like, you don't wipe this up. Use a scraper. And she's like, this is amazing.

the workers' AI embeddings,

shameless plugs? I think I think Rita plugged All the right things, I would highlight the AI stuff, and please just give us feedback. Like, we really just want to hear from you. And more than anywhere I've ever Worked in my own career. Like, this is a place that we really read it and take it seriously, and so many of our internal discussions are literally like, Hey, this person is talking about this thing on Twitter sent a great GitHub,

scour it for developers To deploy models in the same serverless way that they deploy code to workers today. So you don't have to think about scaling up GPUs or Even getting access to GPUs or GPU capacity or anything like that. As Wes mentioned, you just kind of pick your model and Or similarly,

Been at Cloudflare

kind of everything that's developer facing that you work Within in a Cloudflare Worker. Nice.

for 7 years and, particularly focused on building out our developer offering.

JavaScript runtime, which is like like, obviously, we have Node and then we had Dino, and you probably heard of Bun, and there's We had Jake on from Fastly, and yes. Great. Brandon, do you have any other middleware runs on on Cloudflare workers, but this was just regular service service. And we I I scraped some other website for recipes,

serverless

rolled out winter dotjs, which is another

CloudFlare. It's going on the Internet. A lot of people are using it. You wanna make sure you have some error and exception handling tracking, and Sentry is a great place to do that. So head on over to century.io.

all lowercase, to get 2 months for free from Sentry. Yeah. You can, Sentry's got a Cloudflare Workers integration. So And that takes yeah, that takes a little bit of time before we actually send the thing off. And the other one is we use Puppeteer to take a screenshot for our Open Graph images,

You can always use coupon code tasty treat,

if you're pushing out code anywhere,

edge in Cloudflare Workers, like your entire business on it?

Cool. Well, before we get into it, I do wanna say,

Is Cloudflare Workers, which is their Yeah. When I did my

server runtime, right? Like we're starting we have way more Server runtimes than we do actual It's so good getting flour off of Yes. Anything. Yeah. Yeah. Big fan. Nice. Good. That that was a sick pick. I like that. Alright. We'll move into shameless plugs.

Brendan Irvine Broek on today to to talk to us. So welcome. Why don't you give yourselves a bit of Instruction. Say who you are. Hey.

So really excited that you guys wanted to talk to us about that. Cool. And I'm Brendan, product manager for the Cloudflare Workers runtime. I focus very deeply on the APIs that we offer in workers, versus wall time or duration or gigabyte seconds can encompass time where there's no work happening. You're just waiting on IO. So maybe you made a A request to an API, and you're waiting to hear back. But until you hear back from that server, there's no work happening in your worker.

Let's start with there's kind of 2 things I want to get out of the podcast for the listeners is the first I Everyone listening to understand, like like, what are workers and, like, what are the use cases and why might you want to use 1?

I forget if some company, don't be shy about letting us know about that. That's good. Because Because you do polyfill a bunch of of node APIs, right? Because the same thing with Deno is like ideally, yes, everything web standards beautiful, but real world, A lot of these packages like request JS, which is like hasn't been updated in 10 years. That's one of the most popular packages in the world, right? but there's no other recipe API that has the exact same PHP app that's 37 years old,

There you go. They got something for everything.

we'll talk a little bit about the effort to

the other day on Friday, somebody announced, quite honestly, I'm really excited to hear that you've increased it

on Cloudflare.

because you can control the outermost layer, you can control the the kind of routing of what,

Into like a Keep those standards based so we're not having IE

And then they, like, started making it look crappy Anything you'd like to plug to our audience audience before you,

blog posts talking about that externally.

one, I a a web standard

a couple of years ago they took features out and then they started making The it basically was for screenshots and sharing images and and redirecting URLs. And they took the ability to they took the read URL redirects out entirely because People probably using it for spam or something.

They let you have your own custom domain name, but

eventually, they'll get rid of the The old monolith, is is that a common use case, or is that just a a couple one offs we heard about? I think that that kind of architecture you're describing where there's kind of this islands approach is Definitely something we've seen a fair amount of, and we've done some of our own,

There's some stuff here and there that, like, often whatever, sometimes gets into trouble. But I thought it was so cool that you could just like proxy an entire website, Do you also see people rewriting

Add in your own,

of the entry point of your application. And so I think that's really powerful for teams that, you know, maybe feel a little bit stuck today, that really want to And so when you run Wrangler dev, You're actually not running does with them to

Like, it's all kind of to that end of how you build something complete. But then there's also this really interesting angle I like to talk about with, works on the open source Cloudflare Workers runtime called WorkerD.

that's awesome. I also wanted to to just give a couple of different use cases that I found Super handy for Cloudflare workers

1 hunk at a time,

and I said, all right, whenever somebody goes to it, it was west. Ioforward/abc123.

accessing data, working with,

that basically just sat in front of it. And I took my domain name back and that's always the thing like, oh, should we build it ourselves or use a SaaS? And this is kind of like, well, I'm using a SaaS, but I can sprinkle on my own little

Do people use Cloudflare Workers for for both of those things?

we heard it from a couple of listeners where they say I use Cloudflare Workers to we take our existing

And start to kind of peel back the layers and put the outermost layers on cloud flare, on workers, yourself.

see is people start their journey With 1 of 2 things. 1 is what's a part of my application that's really slow that I would like to have it be faster,

and that's really where we're trying to go, right, if you've seen some of the announcements we've been making around allowing you to connect directly to your database,

is this perfect thing that sits between the user and your origin server. And once you offload it, you unlock both a lot of scalability and performance.

but don't see necessarily that path to incrementally migrate.

paid for the SaaS called CloudApp for the longest time and it was expecting. And now I know it. Never

to our listeners because,

Like do people can you run an entire

Totally new greenfield,

Rebuilding their website

and we still need the header and the footer, and we need different parts of it. But Like, I want to be able to fold in my fancy new application, but we simply do not have the infrastructure

that I'm having a really tough time scaling,

Just go fetch

Yeah. I think so so much of what we're seeing right now on our platform is people really doing that and building run Wrangler locally, which is the CLI for CloudFlare,

See the benefits of serverless, see the benefits of workers, You have to manage that connection pool yourself,

to do it. So they were simply just like and, eventually

And so, yes, definitely a thing that we've seen happen quite a bit. Okay,

and then start to build maybe the net new Pieces of what you're building on workers,

where they put their logo all over it. And, like, it was just like the product It kept getting worse and worse and worse. And so what I did is I I wrote a Cloudflare Worker

Application So

And then if it's a redirect, redirect them. If it's an image, download the image

and those often do sit at kind of that in between layer. So I find Auth To be a really great example of that, where we have so many customers, including NPM was actually one of the first ones where The auth server is always this source of choking point

the unbranded version of that asset.

because nobody wants that bug where they're like, Oh, yeah. Like, I was building this thing, and it was in Node, and I deployed it to this runtime and, like, why am I seeing different behavior? from people about, you know, what are the Things that maybe you've relied on in Node in the past when you built applications that you'd love to see a broader set of run times all support.

You did a talk at one of the WASM conferences.

And as you can imagine, that got Bigger and bigger and more and more challenging to maintain,

trying to do Vercel only errors, runtime. So Brendan, that that's that's what you work on, right? Like

from supporting that package because,

out there,

you know, the way that the API that you

And so that's why we've done that. Oh, yeah. We we are very familiar with

Those APIs need to be Polyfilled for the time being. Yeah. And we actually go a layer beyond polyfilling where we're providing built in versions of those APIs that are native to the runtime.

the exact same thing that we're running in production that they're having to each build in isolation themselves That makes sense. So so better less on the client side. But there's some interaction that Happens in near real time between client and server,

we just we can put more things directly in the runtime. We can give you Exactly what runs in production, and that's what's gonna give you the the kind of guarantee And,

and so it became more of the direction we're going in now of, like, how do we just give you the actual thing, the actual actual runtime.

And we'd love to be part of that story and making that easier for you And having less bugs reported to your project, where developers are saying, Hey, like, I tried this in local dev, and I'm deploying to Cloudflare, why doesn't x work? We really think that if we We have some APIs that we can give you to make that process just so much easier and reduce reduce the burden that's on you. Oh, yeah. Cool. That's really cool because, like, Our our new Syntax site, we built it thinking And so if you take a peek at the Cloudflare Workers' runtime API's docs, there's a number of different APIs, including, say, Nodecrypto, JavaScript standards with TC 39

you know, have some form of local dev experience, and it was kind of this shim layer that said, Okay. What's running locally is node itself, but we needed to to have some way of exposing,

That style of bug. That's why. The last week

And we're actually trying to slim down that layer a little bit because, a couple others,

like, it's probably

that you can trust. One thing I would shout out about before around, Do things that are kind of like CAD, but, Something like worker threads,

And that's been a big effort that We've been pushing towards, and it's one of the many reasons why we open sourced the workers' runtime,

this will probably run on Cloudflare Workers,

you know, we'd love to work more with The people at frameworks of all sizes and scales

not necessarily the best fit. Like, I would just say generally, like, we just want to hear the feedback, So don't hold it back. Don't think that, like, Oh, like, this will never work on Cloudflare Workers. The chances are that Mhmm. We're thinking of something along the way, And there's probably an idea marinating in somebody's head at Cloudflare, and we'd love to to meet with you and talk to you. Okay. And 1 I know we're running out of time, but I have so many questions.

and that's more the direction we're going in is rather than you having to think about your own polyfills or what you're putting in your own Code that's shimming something in.

to

How do we just provide that for you via kind of a compatibility has some some serverless functions

Rely on the workers' runtime in local dev. So many frameworks have their own kind of local dev server that they've built into their projects, and When I talk to people, that's often something that's kind of painful to maintain

scrap it. You know, let's let's go put it and throw it on Vercel. So that's really cool that you can run that locally because then you get that immediate feedback. And also when you're building the thing What does the local dev,

under the hood that actually uses

layer? Is there anything that, like, Anything right now where you say, oh, yeah, that. If you're if you're doing this in node, it probably won't work? Like, is there any, like, heads up that you should be thinking about?

and you hit a use a package that's not supported or A node API that's not supported, you'd know right away until before it's too too late. And we want to know about these errors.

but Cloudflare is not just a DDoS thing. Cloudflare has tons of really cool products behind them, and one of the really cool

Here we were committing, pushing, sitting on our hands for 1 minute and 21 seconds.

buffer a thing that we cared really deeply about at Cloudflare is kind of code portability

So,

as well as kind of shim some different some differences along the way.

a a node based development server. You're running a development server that uses our own runtime,

WorkerD, the open source Cloudflare Workers runtime.

to make it easier for your users

local dev being consistent is if there's anybody listening who's building a framework, draw things to a canvas, or do client side inference and do that maybe in a worker, out of our need to, So we really wanted to to be able to support

grab something from an R2 bucket or to talk to queues or some other service,

I've seen red red errors for, you know, all day long, and yeah. I just know that Worker D was a thing. I I always thought it was Mini Flair that ran locally. What what is Mini Flair? Yeah. So, it's a bit of a funny story here. So Mini Flair kind of evolved

the, you know, ver the appropriate version of the runtime that you need. And so you get the same,

Vercel's

Used to,

even if you can work around it, etcetera, etcetera, I think there there's nothing better than, You know, hitting deploy and that deploy working on the very first try.

when you're running locally.

Is that people do have the this association with edge being something that's

to figure out where is your traffic and Optimize it for that. And so that's our conception or how we think about the edge. I love that. There's so many times I ask myself

because, first of all, I I think it's really hard to wrap your mind around. So to your point, it's not all if you don't know what it is, it's not a very intuitive thing. Right?

that a 100 a 100 milliseconds of compute or a 100 milliseconds of latency might have on an entire company's revenue. Right? Especially in ecommerce. So latency is super important. But at the same time, it does make it slightly more niche in terms of, like, How you think about it. So you start to narrow down your use cases by, okay, well, why do I want this to be really, really fast? I think that the thing that gets lost about How powerful it is to be able to run this network

with the word edge

I've always found that to be such an insane question. I don't know on day 1 where my users are going to be. Maybe I'll start in On the East Coast, but maybe my application is gonna take off on the West Coast or in Australia. Who knows? Right?

The idea of the edge, right, is that you can get as close to your users as possible or as close to device as possible Those generally tend to be a bit more

Where do you want it? I Wanted everywhere, please. Yeah.

makes the edge important.

a bit of a love hate relationship

something really latency sensitive, really high performance.

And don't be wrong. These things are super important. We see the Impact We also announced Vectorize,

300 data centers.

And that's where the kind of concept of region Earth came from, which is that That's not up to you for the developer to have to think about. I think the onus is kind of more should be on the provider Only if this cookie is present, if I want to do something only from this country.

and flexibility that we get in how we schedule things And how much of the cognitive burden in terms of what a developer has to think about, it takes away. Right? So if you've built an application before on a traditional cloud provider,

tricky part of it, especially when it comes to talking about compute and workers,

what region are you gonna run this in? Where do you want it?

is just the scalability

The very, very first thing that you actually have to think about is

But the

What is the edge? Where is it? It's such a funny question.

Yes.

and designed for kind of niche purposes. Right? So It's what you would reach for if you want,

without being

No. I was gonna say, do do you wanna give the audience maybe a little bit of 1 on 1 or 101 for, like, what Specifically,

But most of the time,

Fastify

And is there an opportunity for us to kind of work with the winter CG,

that API for, you know, what is a JSON schema? How does it get transformed?

What we've learned is everyone is trying

random people running random code because they're gonna mine Bitcoin or do something nefarious with it. So

I was surprised to see Cloudflare just went and hired the Hano JS guy, which is you shouldn't think like, that's a good use case for an an edge function. Hopefully, in a couple years, it's I wrote the standard JavaScript, And then 2nd, I wanna dip into, like, the alternative

APIs across run times and have a bunch of open source run times that doesn't make it easier to reason about. But operating your own run time is still and scaling it and Having multi tenancy and all of that is a really massive challenge.

From

you know, like Bloomberg, right, where like Bloomberg doesn't have a runtime, but there's many people on TC 39 working at Bloomberg who make big contributions, who get involved. Well, they do have a runtime. It's just not something that Be kind of at the leading edge of thinking about things, not just selfishly for ourselves as our own runtime, what are the set of APIs that All of those run times share and support,

what happens is said, okay, there's already a way for doing this in the browser. It's called Islets.

amazing.

has built, Is,

do you if you want to like I should say if you want to write the code in your Versus Code editor on your computer,

If you want to

So one, for example, is,

And one conversation that, you know, we've been having a little bit ourselves is like, Is there an opportunity to standardize

Let me can can we just preface this by saying, we've asked Ryan Dahl. We asked Jared from Bun. We asked And that recipe API went down.

performance optimization for its routing layer.

frameworks,

you, Wes, might use. But there's a runtime that operates They're systems. Right? And so for them to be able to take advantage of what are the different APIs and how do we make sure that, yeah, as libraries and things batter built. They're reusable.

But my question behind that is, do you think we'll ever see Some Wasm company, Which was I know

side question to that. Where does where does ByteDance and TikTok fit in? It seems like a lot of these people are hosting companies.

That was what led us to eventually release workers for platforms, which actually Shopify itself

lower level primitives and that is

to to to shape that? And that might create, like, one of many primitives to make it easier

1, I think that there are pieces, if you peel back the layers of the onion, that are opportunities to standardize.

I sat on a board for Wix, And pass the

And as it turns out, running an it's one thing to have

And so as we started having these conversations with more and more customers,

Jake from Fastly,

And so to be able to do that, they want their own customers to be able to run on trusted code.

companies encounter a similar situation to us where they want to be able to Customize something infinitely.

do a little bit less themselves and rely on more standards under the hood. And so I could imagine, you know, a future a number of years from now where there are enough base primitives that

Some of the different server frameworks can kind of level up in their thinking and worry less about the underlying,

Logistically, how does something like this, 1, come together? Like, this is this is wild that all these companies can work together on something like this. But also 2, like,

You can run serverless code on there, but every time you want to use a package, You have to get it approved by them. Every time there's an update, somebody's got to comb through and make sure that it's probably fine, you You know? Mhmm. And it's because, like, you just can't be letting

That's fascinating. I answered the 2 questions in order. So Okay. I just have to shout out, James Snell on this podcast, because he's really been the driving force behind this effort, and when it comes to how this has been even possible, I think it really speaks to, like, you You know, the importance of being able to build community and build relationships and build that trust over time, and that's something that he's really led the way on, I mean, helping people see the long term, right, that, like, we're not all in it for ourselves, that, you know, our work is gonna outlast any one Product or platform or company, and, like, this is really helping everybody see the bigger picture behind it.

for

for some of these different frameworks to

thing look like. Because I like that a lot. Wow, man. There's

or maybe have an internal

that's one thing that we learned very quickly as we started to embark on this is How surprising it is how many people are trying to do this exact same thing in various ways. Run their own JavaScript runtime. Yeah. We, On the device itself.

you are running in a constrained environment when you run-in Cloudflare Workers,

the folks from Bloomberg are coming on. They they're just going through the whole legal thing where they have to get it all approved. But we also asked the folks from Shopify, running basically running server code

I also, all over again.

It it's surprising how many So all of that good Visibility

the deal? Why is Shopify on this thing? And they said, like, we just want Our A long list of some things that we wanna we wanna especially around, like, images and and art around each of now you can sink them or push them. I forget exactly what it was, but

of a faster router based on that schema.

Were that, like for servers for doing routing? Or is that kind of like outside the the scope of of what JavaScript should cover? That's an interesting question.

Hydrogen on top of. That's one. Yeah. Is To exactly satisfy this use case of just because you want your customers to be able to run untrusted code doesn't mean you have to become overnight an expert in,

we said, like, like, what's

and nobody can tell us why TikTok is part of winter c g. So we are. You're our last hope. Why is TikTok part of winter CG?

around JSON schemas and what,

One of my favorite ways to build a server lately. So it's it's it's kind of like the Express,

primitives

Application, we wanna make sure that runs

but this was before the remix acquisition, but we basically want it to run wherever, you know, like we don't if you're building a headless Shopify

And so I think to That end, I think there have been people, even who aren't working directly on their own run times, Where they're doing kind of image processing and processing of GIS data, but it's very latency sensitive. And just the fact that GPUs are

running a massive multi tenant system.

and Wix is now rolling out their own.

So we've been working on adding support for the WebGPU API, and I believe, Pushing and making a big push on our side to, there, but we just generally don't think that you should hit that. Yeah. Yeah. Fifty millisecond limit. We're trying to make sure that every everybody There's, you know, some lighter weight AI model inference that we've explored there.

uploading your Python script

And so That's really what we wanna go out and solve. It's like, you know, how does it how do you just think about

Also work, but I know that there's a lot of people across a lot of Across a lot of companies, And there's a number of different libraries for kind of working with JSON schemas and kind of doing the cogen,

We're one of 2 run times to do so. Deno did some work on this maybe in the past to unlock some of the things where, you know, you can do, 15 seconds every request, and you're getting billed for that. And so what we're doing is, we're introducing limits that you can configure

at some point. And so we're really trying to push the bounds there on, like, what APIs we can offer and bringing web standards into our runtime.

so much I want to talk to you about, but we're running out of out of time here. The one thing I'll tell people to go and and research is all the AI stuff you just announced a couple of weeks ago as well. Like, that stuff is super exciting. Being able to just have access of the thing is they're used to having a file system, They have a long running server, you know, like they're just thinking in that approach.

on Cloudflare? Stay tuned. We're working on some exciting things.

and who are like, What's this WebAssembly thing? This is like, why is this harder than when I deploy to other places? Why do I have to think about, like, compile targets and all this stuff?

What's the story for, oh, I don't want to run JavaScript. Can I run Python on, every time you run a function, it needs to set up a connection. And a A lot of times you can't have that many connections to your database. So the answer to that is you have to have pooling. Right. And that's

As if you're just writing pipe Python and you don't have to worry about all this cruft around WebAssembly. That's somebody else's problem. That really is our problem to to help you solve. The other thing I would maybe highlight, though, is, like, I do think

Very, So I think one product I might highlight is Cloudflare Queues.

Have you if I may ask, have you tried it yourself yet? Not yet. We, we did a show where we did a couple little updates on it,

and I thought it was really nifty that you can simply just like, you have a version of a lot of these models. Right? And then you also did a hugging face thing? Yep. We are constantly expanding the number of models that are offered through us, and that's another thing that we're constantly seeking feedback on and are reacting to it pretty proactively.

in making some things that are today only possible in a web browser

there is an interesting opportunity with with web assembly, because we are

on workers.

web standard centric and very browser centric

Wash those hands, pull up a chair, and secure that feed bag, because it's time to listen to Scott Tolinski and Wes Bos attempt to use human language to converse with and pick the brains of other developers. I thought there was gonna be food,

This

is the Syntax Supper Club.

Hoo. I'm starving.

so buckle up and grab that old handle because this ride is going to get wild.

I've been a Cloudflare

I sure hope you're hungry.

Welcome to Syntax. Today, we have a awesome episode. We have 2 folks from Cloudflare

And in fact, I think we've done a couple episodes on

Workers on today.

fanboy for many years. I've used it for quite a few of my projects.

APIs that you're excited about? Could be browser APIs. Could be Just straight up JavaScript APIs. It could be anything. The WebGPU API, I know I said it before. I think this is super powerful. It's early days. The types of things that people are building in browsers are Crazy and being able to jump between client and server really easily. I just think there's there's tons of opportunity there. And what what other kinds of stuff Could you use that for on the server? So I was actually just on a thread with somebody who, because we don't think it's the right outcome for there to be, like, You know, you're Cloudflare

and so be just being able to,

Cloudinary,

like, KV, a key value store can be really, really helpful for cases when you want to persist something. Sometimes you need to just stash something away in the same data center because you might need to come back for it.

when you need that to happen in a particular And then so that's where you really see,

without having to think about infrastructure,

A sick pick could be anything, something that you just Enjoy in life. Could be a podcast, a TV show, a product of any particular kind, just something that you are into right now.

But buying a bench scrape has been a massive improvement in my life. Just the efficiency

the episodes. So,

So and we want to understand what are the APIs that we're missing

of moving things from my cutting board to another a group that standardizes

but I think the cache API is actually really cool.

Fast at some types of compute,

I put Cloudflare in front of that so you're not paying for pin

is unmatched.

Do either of you have a sick pick? I'll pick a kitchen gadget. Yeah.

to do that kind of visual modeling,

which does image resizing and whatnot.

we our new website is a bunch of AI stuff, but there's So we have them on. We've got Rita Kozlov and

I think that's really powerful.

I think is is a pretty powerful pattern. It might be a bit of an underdog API,

place or where you have some latency sensitive needs,

So, you know,

on that. You know? So you could stay on the free plan. Smart. Yeah. That's just smart, Wes. And then if if it's if it's cash, you can you can cash it in, CloudFlare. Cost savings is a legit use case. Don't knock it.

There's rendering.

Alright. So we move into the supper club questions real quick, Scott. Yeah. Let's do it. We got a couple of, quick ones here. What are

Yeah. I I think it's really neat and solves a lot of problems, and people tend to sleep on it a little bit. So I thought I'd give her a shout out.

Mhmm. Alright. So now it's the part of the show where we talk about sick picks and shameless plugs.

My, one nefarious use of the cache API is, sits between

It's an API that's available within workers, and