AI Superpowers with Kevin Hou and Codeium

You know, what would any developer do? It's like, okay. No. This is what Kevin How would want to write given that he is working on, you know, Kevin How personal website using basically the forefront of,

to the AI to get such good completions? Because, like, sometimes I delete something. Sometimes it knows my tabs. They were able to tell us it knows what tabs you have open, but, like, it's gotta know more than that. Are you able to spill the beans on What? You're sending to the AI? Okay. I can leak all this

Are you sending,

We can infer based on, you know, what your actions are, what your most recent edits are, things of that nature to deliver A few things here.

internal to perhaps either your organization Coffee, I wanna say. It was one of these it was literally an npm library that you would install. And you're writing in I believe it was React. And in in the interface that you would use to to chat with an LLM But there are some brilliant people at brilliant companies working on those sorts of things, and what we have decided is We are product first.

like, a a chat because it doesn't necessarily know. And we had the I don't wanna bringing in Copilot because I I understand. But we had them on, and they weren't able to tell me this. So I'm gonna ask you and Alright. You might say I'm not allowed to tell you either. But, like,

X libraries.

with preface.

That includes things like knowing what The reason why that's important is so far,

very relevant results, I mean, the

or, you know, your individual individual projects, has not cracked, I think, the full Work to be done

to you. And then finally, I think I think, What are you trying to achieve?

the generations on top of that, make the experience not just a

Yeah. That's that's one thing that we keep coming back to in AI is that context is king in that The more information you can give it about what you're doing, the better the actual result is. And, obviously, there's there's a limit to how much Context you can feed something before you have to go ahead and and train your own own model. And that's why these these coding assistants are likely better than just copy pasting that into,

what

rice maker. And and they will find every time we've ever mentioned rice maker, and then we grab That's cool. And how big of a code base

Yeah. But on the scale of of code bases, they are

years years of source code. And they so far have been quite happy with that with this setup, and we haven't had any issues with with scaling. And I think what this comes down to is Having very intelligent

There's probably only 1 button in the Vercel design system, and so it makes it a lot easier.

And that's the like, That's what makes it good. Right? That's the secret sauce of figuring out And I I know OpenAI has, like, a vowels where you can, like, you can,

tens of billions of tokens detect where objects are, where collisions could occur, all that sort of thing. And so Exafunction was building was managing, you know, tens of thousands of of GPUs to basically make that inferences as quick as possible.

I have a question about, like, preferences within your code base. Does it pick up on things like code styling? Like, for instance, I always I write JavaScript in snake case. Will it preserve my snake case when I ask it

what are the best pieces that we need to send along as contacts. You're not sending The entire code base along for the completion. No. No. And I think that's a a a key differentiation.

the Three sentences before, 3 sentences after, and then maybe we'll take the, like,

Abstract syntax tree. So an abstract That's kinda your classic. As you're typing, we try to predict what you're thinking, lowest latency, you know, user in the loop sort of sort of workflow. We offer the fastest, level, like some of the things that we were describing with could you crawl that much? Or there there's obviously a limit to how much you can embed. Right? Embedding is is quite cheap From a Yes. Computational and and literally a cost perspective. So relative to the LLM workflow, that hasn't been a constraint so far. And you actually allude to kind of an interesting point of of scale. Right? And, again, this is, I think, where our exafunction background and a lot of our ML infrastructure comes into play here. So we currently But then you also think about

Use that to inform the model. Okay. Here's what they're actually trying to import. So you're suddenly and writing kind of reranking, you could call it, The public becomes a bit easier. You can say, okay. You're importing. Let's just say you're importing from really hard to break It's huge. It's huge.

the embedding part is actually it JS it Wes okay. There is actually no constraint there. We haven't hit a constraint. And to give you a a sense, one of our marquee customers is Dell.

We have an episode title on the calendar that says, whoops. I'm a data scientist.

of those and send it along. So so you're instead of Sending around transcripts. You you have some sort of algorithm that figures out

syntax man, say that 3 times fast. What an Wes. What an AST.

are processing on the order of, We're kind of at this at this point where we're we're receiving a lot of requests that Number 2, we have a chat assistant, and this chat assistant is actually code based aware. And I think that's something that really sets us apart from, I guess, the competition or other products on the market.

50 different button components, Let's say if you're if you're Vercel. Yeah. Fifty different Yeah. Files named button Scott tsx, and you're working on a project.

I I recently dipped into algorithms. I'm a data scientist now. I'm not. But, like, I Nice. I was like, k means, and now I'm, like, reading papers about what k means. And so, like, I'm way out of my league here. And stuff.

20,000 files.

can you run this on? So I think, like, somebody listening right now said, alright. Are Wes have,

the prompts are big.

Like, I think Vercel the other day said they have something like 40,000 JSX components or something like that. Like, Could

A day. Somewhere along that range of just sheer amount of for questions? I might have to hard code that, if your name is Scott and you're using snake case variable names JavaScript, they're not allowed to use the product. But Yeah. Yeah. Right. Yeah.

we can then say if it's a local file, let's actually open up that file. Let's summarize that file and then use that as context.

So, you know, massive company, massive, So I still think that the autocomplete like a graph, being able to

the top 3

So being able to we do this thing. It's called an abstract

open source library source code that's living either on GitHub or on Bitbucket.

I don't know,

imports.

we grab,

tiny. And so you really have to be selective about what you're sending.

a sequence of nodes that that have some sort of semantic meaning. And in this case, we cared about imports. So now across all the different languages, we're looking and analyzing what are you importing One of the features that we're rolling out very soon is the ability to you know, what is what is the the 10 year horizon look like for outcomes.

like, with the syntax TypeScript. We We converted every single sentence that we said, each of them to an embedding, and then you search for something like,

you boot up Versus Code during a particular project. If you have the setting enabled, we'll actually go through and run sort of a an embedding store

was looking at Vercel slash design system.

runs what's called a language server. A language server is this binary that sits behind the IDE. So unlike a lot of the other products on the market, we are

Then at that point, We send that over to our model. The model generates something, sends it back to the language,

That's 1st and foremost. So whether or not you're using Vim or Emax or Versus Code or JetBrains, you're gonna have a very similar experience because a lot of our logic is inside of this language server binary That's shippable across any operating system. Now that language server binary is responsible for sort of orchestrating,

let's just say, an autocomplete, work to make that possible, and then we'll worry about, GPU time, like we've discussed, is incredibly valuable.

And pull in what it deems relevant. We rerank everything to try and fit it within that kind of valuable context window.

There are a number of things that you can do to get hyper advanced context. And I think Copilot having used the product And talk to some of their reps, It might start PR reviewing itself. So you don't you don't really

it'll run through all these different sources Okay. Yep. Yep. Get the checkbooks ready.

That's super interesting. I I Wes always so curious about I was like, oh, maybe they're, like, look in the clipboard and and see what's in your clipboard and but it's it's obviously a lot more than that. Like, it's,

on what's inside of your code base, Which means every single file will get indexed, every single function will get indexed, whatever your documentation, all that sort of thing. And this all happens locally using an embedding store. Now we also have other sources such as active tabs, where you mass where you last edited. You can even look at things like Git commit history, which is quite interesting to see what sorts of things you're most recently doing. You can imagine in the future, and we haven't gone to this yet, you can imagine in the future, you have a GitHub issue or you have a Jira ticket.

you can take my word for it, However much that means to you. That as soon as it gets our stuff, goes through our models, and it'll leave quite promptly afterwards.

Like, I assume with JavaScript, you could probably crawl the dependency tree, right, and see how Exactly. Files interact with each other. You're spot on there. And I think one of the one of the first things that We did,

Crawling, anything that it might deem relevant to what you're doing. So for example, what it does. And I think So code styling JS picked up on that. Yeah. Code has picked up. And I think on a case like snake casing, where you look as well. And so every time you invoke, Imagine you're creating a PR, for example. I mean, that's kind of the holy grail of what a lot of Raise some money.

indexing workloads,

But, of course, we, you know, we don't we don't on anyone's data. We are very we are very secure.

We we actually have 0 day retention. So Whatever the research can spit out, we can inherit and and benefit from. But the very basic thing, we want to build a product that is is rigorous and useful.

but for Yeah. Working with AI. And is that chat? Is that in line? Is that comments?

And as it's going through, it'll await that with higher Higher importance. So that then, in theory, the language server, and then we deliver it to the ID surface in the form of an autocomplete, So, you know, you might think with with AV, a lot of it comes down kind of the decision making, the car's intelligence, all that sort of thing. But what I found very interesting, Exafunction got started doing ML inference workloads. So the whole idea was or because they don't scale well for errors and especially when we're talking on the magnitude of replacing developers.

UI, not, like, aesthetically,

context bit, and you'll say, I want Svelte

the best

segue for a question I had too. You mentioned having those different interfaces for interacting

generations across all the different surfaces and as a reminder, we have Autocomplete as a surface. We have,

future

pin a piece of of, like, quote, context. So I guess in your case, Scott, You'll be working on your project. You'll have a sidebar up. We call it Codium Brain. It's like the ability to, to kinda do these just more advanced tweakings.

What what's your take on on, like, how we interact with AI as developers.

And you'll actually go in and and press the add The the file that you're in probably has existing snake case variables, and so it'll it'll kinda it'll kinda do its thing there. Where this gets a little bit more interesting one of the Copilot reps was on stage as that tweet was kinda circulating, and he had to, you know, do a bit of of damage control. But I think, ultimately, what that indicates, you know, is

have we hit Svelte five code. Or,

with the AI. You know, I I've always wondered a little bit personally, like, Oh, he takes To drink. I know. I know. I have to prepare. Here we go.

it's important for everybody to get to try the tools to experience them in the same regard. So, yeah, I'm sure that's a a tough balance to, to walk.

That that hopefully, you know, Codium can can Yeah. Can be.

Everyone but at the scale of what is enabled for an enterprise, if that makes sense. We could do many features model quantization, And People are pretty quick and eager was Crazy idea. I personally don't know if that's repositories, kind of what autonomy engineers wanted, I am from the Bay Area, been a builder all my life, engineer, born and raised. I've kind of been in the startup world or was exposed to the startup world very on.

hey. I came up with this,

when you're developing tools like this, is it, Is it like

Interesting. I never even thought about that, but it's it's true. Like, we we talked about the the embeddings and all the experience of all that. That stuff matters a Scott,

I'm curious about,

Or or or or going to be the next frontier. And then we IDEs, getting LLMs into the developer experience. And as someone who is really interested in building tools,

What, like, what languages sorry. Go ahead, Scott. No. I was gonna say, like, how many of these ideas that that make its way into Codium. You know, I mean, as developers, I'm sure these things just pop into your head.

knowledge

for both for the user experience as well as I'm sure, like, if you can save Okay. So to to give the audience another example of the embedding, because I always I'm always the guy that talks about embeddings whenever AI comes up JS,

I don't wanna say gimmicky because I think that that's probably an unfair term because they're probably very cool, and and they could be useful. Right? But if we if we went for after every single So I wanna say I wanna say yes, but I wanna say yes with a. We are working on all the different surfaces that can make this possible. So

like, an enterprise developer? And then we kind of use the individual tier as a way to validate these sorts of Products. So the individuals are actually getting tried to reconcile these 2 is and, honestly, a lot of exploration to be done. I think it's is unfortunately just, what you wanted in that markdown component. And then it would Yeah. It would update its own, like, source code. Oh my gosh.

Of of maybe in this particular file that you're working on.

where is the You're the fan of the of the podcast, so it's cool to cool to be featured. Like Scott said, my name's Kevin.

Mhmm. Mhmm. How do we make something very, very useful?

or listening to what users want? Like, how how are you all coming up with

file gripping and didn't put it in some sort of scalable infrastructure. That product would never have made it to the enterprise because it just doesn't support that level of scale. And so we're very meticulous about deciding what we work on such that it can eventually make it to the scale that we could deliver a product to people is by by putting using our own infrastructure that we had built at exafunction you could think about it this way. If we went through a repo and, like, deleted a random line or a couple characters from a repo and then ran its unit Wes again. Like, would they still pass? And there, I Wes,

the ideas for these types of developer tools.

10% of your compute costs, I'm sure that is very significant.

thing I would really like to do. Let's see if we can hack it together, or is it based on

Not a very clean way of answering your question, but, like, we have many different streams coming in.

to share what they believe

idea that came into our heads.

you hear a lot of people saying

And we start from those primitives, and we we kind of do the work there. We do the hard engineering

you know,

community

the AI is getting worse. It it used to be amazing, and it's It's no longer good.

And we do our best to try and find something that that works for was The tooling and the visualization specifically that went into how do we make this car drive. And so I was spending a lot of time thinking about just

where is the use case for,

that sort of thing. So we mentioned the language server binary, Being able to compile local files and figure out context and do the prompt construction, that is a very that's a meaty part

You build an AI model, but, like like, what is that? What is an AI model? How do you make one of those?

are trying their best to predict Based on a sequence of tokens, like, what is the next most probable token? That in a nutshell is, like, have have Svelte of that commit as kind of a prior.

And the performance there and the number of parameters, all these things reasonably benchmarked across the industry. I mean, you'll see I guess GPT has moved beyond this. They're now on SAT scores or whatever, LSAT.

So you sit down at your desk. How do you make an AI model? You know? Like, everyone's always talking.

You need some goal.

compress and and make the actual inference as efficient as possible.

You've got the IDE surface where you're like, okay. We need to be able to interface with said language model,

You need data that can make that I mean, what or, you know, any number of reasons. In embedding, for an example, if we did that the incorrect way Kind of orchestrate You know, able to to give you the best possible experience, the lowest latency, I won't dive into too much detail on sort of those technical nuances, but at the end of the day, you need I think voice had a had a moment there, you know, where people are like, I just wanna dictate Those sorts of features are probably only gonna benefit the enterprise rather than the consumers.

Achievement possible. Mhmm. And then you need a lot of compute. I mean, essentially, in a nutshell, these LLMs basic ingredients Us AGI companies are kind of going for it. It's like, how do we go from a GitHub issue to a PR and have that get merged and tested and all that sort of thing? The amount of steps that have to go into that process specifically related to AI.

That is only the tip of the iceberg. Obviously, the model is incredibly important.

It's a very it's a very complicated and and often nonlinear

Scale with the with the product quality.

or said said language server. You've got the infrastructure to support getting LLMs into

important that you can have

Of of what you need for the model itself. Now what we've noticed is is variable and immense. There might be multiple iterations. There might be multiple stages of testing.

the actual inference on the model. So for example,

Wow. And so this comes out of Exa function. Is that the the company that is that is that the case? So I could actually use this with the the latest versions of whatever I might using instead of

You gotta the request is probably irrelevant. You probably won't see 95, 99% of what makes this product experience magical.

and deliver that that product for for actually, for free for individual users.

about 2 years ago, the The CoPilot numbers. I funnily enough, when that announcement came out, I was actually at the AI Engineer Summit in San Francisco, and, You can break your files and and your programs into,

how do we maximize and squeeze the most amount of compute out of GPUs? So we these sorts of workflows.

owns

installs in JetBrains.

It has

the best way, later down the road, but you can think of it as GPUs are expensive.

to serve a product. And and that sort of infrastructure layer is still kind of the bedrock of our of our product, which makes us, And cancellations, and being able to kind of assemble all this and and then run

262,000 and just in general, how we can get the most out of these tools and a little bit about how they work.

startup. So, We'd move at a at a very quick rate of of shipping, but I don't think that a lot of those would end up making it to the enterprise Either because they're not useful

Wes, so okay. I guess that's a a good introduction then to to maybe even move it into Codium itself, which, That's a fine line to walk with enterprise versus, you know, your your free customers that are they're they're not paying. They are free customers who are essentially freeloading. But,

installs on Versus Code,

there's this thing going around. People say, like, Microsoft is losing money on GitHub Copilot. I don't know if you know if that's True or Scott. But, like, it maybe you can confirm. It is it is very expensive

And all the different infrastructure that goes behind From a from a coding perspective, I actually saw a really interesting I think it was called IDE agnostic.

something like a GitHub Copilot.

I'm I'm assuming you've used Copilot and you've used sort of these auto complete tools. Once you once you go, it's magic. You can't you can't go back. And so we wanna deliver that experience to as many people as possible.

So those Yarn some wild numbers. But, like, can you tell me, I guess, what exactly makes Codium different than other, you know, language assistance inside of your code base? I feel like I've used a whole ton of these different ones. So what what sets Codium apart here? Sure. Sure. I think there's there's 3 things. So

was the original

How do we get how do we get the best bang for our buck? And so we were building infrastructure even if you have hence the infrastructure, the fastest product on the market for that sort of thing, which means every single keystroke, we're trying to we're trying our best to to figure out what you need.

Think of again, a lot of us have backgrounds in in self driving. Think about the workload of an of an autonomous vehicle. You have limited compute, What's been interesting is we're catering to kind of an individual audience and then simultaneously an enterprise audience. And, of course, as you know, when you start

these large scale workloads.

Limited dollars.

Totally. Totally. I mean, the amount of compute needed for for these sorts of, language models is is immense. I don't I can't fact check exactly, higher level will now be Now, you know, a year later, multistep reasoning,

561,000

It is expensive. And I think even for a company that has relations like that to We have a very active Discord, and I think

That's really cool. It's like, I've heard that that

So there's a variety of techniques. Happy to talk about those, slowly, you know, validate those things and then push them over to the enterprise. I guess I is the next logical feature or you know? And you can kind of do your own, Microsoft and OpenAI and that whole kind of trifecta, an incredibly expensive product to put out there. In my opinion, you know, the $20 that you pay for Copilot is actually

to run a,

There are

There's things like human eval,

why do some things get worse? Is it because they're putting less compute into it? Are they turning the knob down so it's a little cheaper?

They're they're on their own thing. But for code, for example, like, something like human eval is is there are there are metrics that are kind of accepted. I think the tough part with human eval

Is there any way it's it's kind of a 2 part question. Is it first, like, how do you guys,

more traditional evaluation The previous your generations will be relevant to to your use case. Okay. And and and that's actually a really interesting,

Yeah. Yeah. That's that's a great that's a great question.

check that it's not getting worse over time?

pipelines that are And so I guess the way that we that sort of autocomplete form factor is the way that you can Can best form, courseware.

And and second,

And developers rejoice because

spatial computing. You're gonna be waving in there. Spatial. That's true. Get out of here. Come. No. Come back.

chat is going to be the lasting

ideas that you've heard?

everyone had a couple beers, and, like, there's that 1 guy in the corner that goes, alright. Hear me out. Have you heard like, throw the throw the text editor out the window. Throw the laptop out the window.

was literally opening a, Like, a markdown component there's no limit on the amount of complexity that can kind of go into something like that. I think from a user experience standpoint, I don't believe that when building a product like Codium, You know, maybe you're using Node Similar workload patterns. A lot of these LMs are, Codium as as a product. There there's so many different facets, and I think It's gotta be perfect. You know? It's gotta build that trust, And you've gotta be very quick with kinda course correction.

Is there any, like, wild

ability to to put this in the IDE.

well understood and fairly easy or intuitive to pick up.

On the The more traditional ways.

completion. Would love that. I love nonstandard interfaces like that. That's what the the voice thing was really interesting to me as Wes. It's like, yeah, we have a A keyboard,

we have not been replaced.

people have all sorts of ideas. I think

GitHub Copilot's

So far. Yeah.

abstracts away a lot of the complexity that I think you need to understand in order for the workflow to work.

user experience.

What's the, like, Silicon Valley, you go to a Sanity,

Mhmm. You have to be able to go in and and sort of manually tweak. What if it did step a Deno. Right. TOEI. Yeah. Interesting. But what we've tried to do is we wanna give this to everyone. We wanna democratize this product.

product.

what I'm doing. But I think what people have realized JS, So far, you really do need the human in the loop for the sorts of performance that we're talking about currently. I mean, GPT 4 and all these language models are are really great, But they are still prone

yeah. What other inputs can you have like, Programmatically test if it's if it actually is getting worse, or are you just being spoiled that this robot in the sky is is doing all the answers for you?

surface. I think there's a lot of external sources and internal sources to then give you the best the best completions.

We have a funny, we have a funny thing in the office. We actually have a foot pedal that we've programmed to Bos, so you can just kinda Yeah.

when you're working with coding? So foot pnpm, voice,

I think the autocomplete surface is is brilliant so far. I think so so far, it has been the most intuitive chat as a surface, and we also actually have terminal as a service. Mhmm. All these different services

and you have a mouse, but, like

a chat based workflow.

We have to hit tab.

Alright. I'm I have a question here.

you know, all sorts of industries. I think GPT, sort of that chat interface, is great. But I think the first success that we saw

Varun Mohan, who's the the CEO and and and cofounder, kind of combining all these different third party libraries you're using, documentation the kinda ergonomic that I would want, but, you know, people are thinking of of everything.

he approached me with this kind of new idea of people start paying for you, Their their opinions matter a lot more. Yeah. And so Yeah.

how is it going?

a little over a year ago.

Got my computer science degree, and naturally, like any other Bay Area native, kind of returned back Spectrum of things that that you could do. So, for example, you mentioned neighboring files and kinda like what tabs you have open. Or I guess you mentioned what tabs you have open. That's just one of the things that you can do to say, okay. I they're they're looking at this particular folder. Our service

building developer tools.

Kevin,

Do you wanna maybe give a quick introduction to who you are and what you do and how you ended up at Codium? Sure. Yeah. Thanks for having me, Scott, Wes.

So Codium has been around for a little over a year now when the Codium product first got started.

So kind of Node quite sidewalk robots, but Small Scott Wes we kinda try to boil this down into Or if it's an external file, we actually rolled out a service. It's called Codium Live, but it allows you to actually look at

to my roots. And, I actually started my Career in the self driving car industry. So I was working on Oh, really? Nuro. Yeah. We were doing, package delivery.

So,

I thought it was a great Great opportunity. So I've been here for since its inception, that are A chat message, whatever that that surface may be.

what could we how to understand what the car was thinking and and why it was making the decisions it was making. So always been in this sort of, like, tooling, product, user experience world, Yeah. So I think is that it doesn't the examples don't often mimic

Look at what you already have written and use that to dictate That form is Because compilation of, like, what that means. You Node, you hear 10 requests. You try and figure out what that what that feature could look like.

let's just say you're using a specific version of a unit testing suite Mhmm. Or a specific version of a of a language.

So you're saying that Codium will actually look at that dependency and then scan works in your Node.

That that's something that's always been a problem with any of these tools. Like, for instance, right now, I'm I'm using Svelte five. It's in pre release candidate.

easy thing to pick up on just simply by Because of the reranking context that we Talked about with okay. What project are you working on? You Node, what repo are you in? We can very quickly filter down into what we believe is most relevant. I guess in that case, it Just Tap to Codium. Is that is that accurate? Or Yeah. Yeah. It's a bit of a it's an it's an interesting story.

But but, yes, to answer your question, it will

just being SOL.

is when

kind of these higher level things about your code base without you explicitly telling us. And so that means being able to answer the Wes, like, are they using or or node 20 instead of, like, node 18. Right? And there's some there's certain things that are are unavailable.

Deep context really comes into play, and that's where we're actually inferring Now suddenly, you have kind of this high level intent of what you are trying to do as a developer Oh, yeah. Yeah. Node dictate to So to answer the first part, how do we actually measure the success of the product? Basically, Exafunction first of which, autocomplete.

the dependency itself to understand,

AI superpowers

Welcome to Syntax.

in your code base, in your text editor,

In this

all of the the cool type of new tools that have arrived on our doorstep,

Friday supper club, we're gonna be talking to Kevin Howe of Node, and we're gonna be talking all about

Scott always you Node? I think research process for this sort of thing. So I think there's a few components. And, actually, on your podcast on of the AI primitives that you were talking about. You nailed a lot of the points. So real life users and usage would look like.

One of the very first things that we did for the Codium product is actually implement the end the end result of a product like that just simply because it

for LLM products.

as a first class Might benefit kind of an organization way to work with an LLM that we've seen across, So I think we have a bit because of the autocomplete form factor, As soon as you type if you're typing it faster than our latency let's just say our latency is 10 milliseconds. If you're typing it faster than 10 milliseconds, the relevant unit Wes in the way that is runnable in their code base.

functionality.

a lot of really hard, like, infrastructure I kind of alluded to this, but, like, personalization in general is something that We take a lot of pride in at at Codium.

can get canceled midway through. You now have slots for 99 other people to use the same GPU.

what our team is very excited by is just solving Jest? What version of Jest? And what's that testing infrastructure look like? So then I can go And actually generate This is, I think, when you could do the naive thing of running of of searching to do. But I think the nice part is there's plenty of companies working on this sort of thing, so I think Someone will kind of arrive on Yeah. A a logical, sized robots.

problems. It is of of code that we are trying to either index or generate on. And so for this example, if you have 20,000 files, I think when the LLM boom kicked off, we realized, oh, you know, developer tools, AI, a single model on a single machine and just Taking requests in serial. Right? But when you're serving hundreds of thousands of users like Codium is, it's very more and more complicated changes like, our quote research or like what the features that we believe massive number of Files, for example, hundreds of thousands of files.

of the completions that come through. And so what we're doing from the very beginning is because we kind of own this infra product to infrastructure to model layer, we can implement a way for the model to kick out, Save that GPU time so that 99% of the requests parallelization, 3 out of 20 wrong? Now steps 4 through 20 are now now corrupt. How do we go back and make that change? It that that doesn't feel exactly like a of this version. And I guess in your case, it's prerelease, you'll probably do Svelte at this commit on GitHub. And then for all we had worked together in the past, and Exafunction was the the perfect infrastructure play to make that happen.

cancellations very, very computationally heavy.

I mean, the math doesn't quite work exactly like that. The general idea of, like, what it takes to build that is a fairly

has a play it it you know, research is immensely important