The SQLite Takeover with Turso’s Glauber Costa

to 2012.

I need your branch. So you need you need to think about how I'm gonna branch this. And one of the things that Linus said a lot about git at the time JS, like, hey, look. Just creating a branch now takes regret using that word, that we used very heavily in in the beginning.

You you had to go and and sync with your team. Oh, I'm gonna merge something now. Like, everybody right? Everybody beware.

you need to transform that into features that are themselves very valuable.

And in in the middle of that period, there was this thing called Git, that was written by Linus.

as as I mentioned, in the beginning. I was in the Linux kernel That word got

the strategy for building a business off of that, is that all ESLint,

Bos on on on, like, very capillary in in again in which replication in inside your mobile device, inside your own server, We can we can do that too. However, that SQLite is a great database. We love Sanity. And and, that this was just a metaphor. When people say Node SQL databases Also the SQL databases got better. SQLite itself got a lot better. SQLite, turns out JS not that complicated. So if it's not that complicated, what happens is that all of the databases just adopted like the same way they they adopted JSON as a as a format. And this happening with Postgres. Postgres is doing a lot of vector. This is happening with us. So we allow, for example, without any extensions. You don't need any extensions. You just build Libs SQL.

that Yeah. That allows for architectures that just were not possible before.

you have to build to to build a really big business, you now have an HTTP to make the changes needed to SQLite to make that happen.

So when you have something that is, like, seemingly so, like, cheap to run, is it

Mhmm. And and the whole story of how it came to be was actually quite interesting. But, then use use our edge replicas.

But because the cost of a database essentially drops to 0, the cost of creating a new database and and maintaining a new database drops to 0. There are new things that you can do that you just couldn't do before and that allows you to have, new architectures those new architectures based on hypertenancy go. Not that sick. In factually, if this is a condition, this is technically a sick, and each query takes a 100 microseconds, that you can query from, despite the fact that that I suffer from this, which apparently there's no fix. It's just by it's just like being left handed. It's it's what it is. It it just born like this and there you

complicated and

And in a in a sense, we wanna do the same with Terso. You can still use Terso as a SQL database as you used before. That's fine. And the baseline works.

So so you have to

one of the thing if if if you're not familiar with the landscape at the time, the dominant version control system in the open source was subversion and c v s. They were like people some people use c v s, some people use subversion.

trivially cheaper, you can have a completely different workflow that would have been impossible with this other tool. And I think of databases in the same way. What we wanna do is, there are 2 companies in particular that I know that may have use of that. I don't think I am at liberty to disclose who they are, but but we know of some. But I also wanna clarify that we also don't put a lot of effort into making that happen.

gets a lot of people in the venture capital industry excited.

that other folks can't do? Is that the approach to to take something that is, like, you know,

And they're only possible, for me, if if you ask me, like, what what is the the thing that really shines about Truso and what is the thing that people should come to Truso for? Yes.

like, what kind of novel things can we do

Look. If you replicate that into a server that and you're Node reading from a local file, And we keep this.

an edge function which could run anywhere over the world, which

that's fine. Right? Just it it allows you to have this architecture. And that's what we wanted to do with the edge. I think we we realized this vision even better with far embedded replication. And the way we think about this today JS, like, look. If you if you are using a

So you could we could store a lot more stuff. Like, sometimes you think, we're not we're not gonna store absolutely everything because that's that takes up too much space. You know? But Mhmm. As the cost of storing things goes down,

that I haven't I I still believe Kent Dodds, is is organizing a very select is this idea that, look, you can have a database And the end of August, I am attending the Laravel conference.

can you put the database in the client or or or put it in Wasm or something like that? No. If your if your digital ocean,

that's fine. You know? Yeah. Yeah.

I I've been I've been doing a lot with local data

So you need to account for the fact that somebody else may have written we that attitude at the time should be just a breath of fresh air. I mean, like, this person

as well? Is that JS the edge functionality of Churso,

than they are in New Jersey, assuming, of course, your main database is in New Jersey. And the other way around if if Even serverless Postgres that you see today, what they do is that they bring compute up.

this is one of the things that look. Our company is 14 people.

right now Wes have

connecting to the database, just put the put the the file on the server themselves. Mhmm.

Then let's say you have something, that's a use case that that we have here like a farmer that has a tablet that has like a schedule, and they have to go in the fields. There's no Internet connection in the fields.

the the way it looks in code is they create a ESLint,

And if you wanna branch based on that, you have to be aware of this whole structure.

Wow. And in that same regard, you had mentioned embedded replicas

And the other question I had was

SQL has the the asset model, has the transaction this all happened in October 2022. October 2022, we launched this fork of sequelite. And then by February 2023, we had Turso demand Sanity service Scott because we Yarn, unlike the Drizzle folks, we are a VC backed company. So if fairly early on, we we had to and now you can read those things from your, at all while, apparently, if you guys close your eyes now, nothing brings you joy. You just and that is just but GDPR compliant The the pull the the pull request thing, we don't, which is something that we wanna get you in the future. But the branching, we do. And, Well, first of all, thanks so much for having me. It's a pleasure to be here. I've been a fan of the podcast. And,

they are called embedded Sometimes because maybe your business sells to Australia, but not that much. You're not gonna put a replica there because it's gonna be very expensive to keep that running all the time and up to date and sync, etcetera.

SQLite here as being a file?

Yeah. Yeah. Yeah. So That rule is Yeah. That's a so simple. Yeah.

may or may not have a lot of requests. So imagine, for example, you you have a couple of users in the United States Becca and I worked in the Linux kernel for 10 years, each of us. Becca being my cofounder. We then wrote a database from scratch called Sanity DB later, But but if you make changes like The stuff that people love like, which is again, the file based DX, your phone probably has like 10,000 SQLite databases

user, So you had to scale out. People sometimes make fun of that and and say, no. You didn't really. But reality is that you had if you had any website in 2010 Pacific, and and there you go. You've got a copy of the database at that point in time. Right, Justin? You know what? I can get a database per session of my user in in the web. I can get a database per user of, Yeah. Well, the the the the Vercel Node is, but but Mhmm. We've worked together in the Linux kernel for around 5 years, and he also uses a Mac. So we're always joking that, you know, we we made the sausage, which is why we don't eat the sausage.

is exactly the same thing.

The only thing is that their write will now go through the network. So you don't have to do anything special in the client.

with the same latency from Australia, United States, Brazil, Canada, in especially in the JavaScript Sanity, in particular, but not only, but but but And at the time, Like, on the client, you you call, by replicating Sanity massively.

We copy the file.

Does this also in like, inform the

I was in Cloudflare, and they they literally allow you to scroll scrub back in time to, like, literally any point in time, which is wild to me, and it's all instant.

you just said, well, if you have a server

Making sure that the values are still make sense. Like, whatever you decide to do, you do it. But it has to be the trade off there is that it has to be less transparent. You have to come and decide what to do if if your if your rights fail. Right? Yeah. I I I think that's alright. You Node, I'm I'm doing a lot of this stuff with Replicash right now, and it they're you Scott handle everything. You know? It's just sending you patch messages and all that stuff. And so it's not like if you write something, you Yarn guaranteed to read that right. Because,

Because I wanna I wanna clarify something as well. The Mhmm. And and so we're nowhere near that Scott. But there are a couple of companies that we know of that use Libsico without Ursula. That and and I'm not gonna unfortunately, So we decided to fork it. That's how, and and building infrastructure to to go around it. Right? Okay. And why would why would you want 10,000 databases?

conflict resolution happen if the user's offline comes back online? Will it sync correctly? I wanna clear I wanna clarify something.

offline sync, and that's my, dream service right there. So, yeah, sick. Can you do it from, like we've been doing a whole bunch of shows on, like, offline first as well. And, like, what does the story look like if this if your server's offline, if your digital ocean is offline,

and some users in Australia. Those users in Australia, they show up, for 8 hours a day.

we use on the syntax website, we MySQL, and we use PlanetScale

No. Because the XFS file system has copy and write capabilities it's itself. So you can you just back this up by Wes, and then you copy the file. And and that's it.

It's kind of the fix forward instead of roll back idea. A lot of people in Git are like that as well as don't roll it back. Fix forward. Yeah. So that JS the only difference. Like, I can give you a I can give you a snapshot of your database at any point in time in the last day if you're a free user, in the last 10 days if you are a $9 a month user, in the last month if you are a $29 a month,

That's not how SQL works.

and I'm running a DigitalOcean droplet, I could just download this SQLite database,

we we wanna keep things very simple for a couple of reasons. One of them is so that we can keep But if you if you need none of those properties, you don't need the this level of replication and microsecond level latency. You don't need multitenancy.

There is a lot that you can do when you have a Yes.

So this is even better than just having 5 or 6 locations that we offer ESLint, TypeScript.

and and then you you go you switch that URL. So you can do that, but yeah.

is that it it is the simple thing to do. You just get a copy, in in which, like, nobody's telling the truth Let me give you a couple of examples on how driving the cost of a database to Deno allows you to rethink architecture. You just Vercel came up with 1. I would store things that I would not otherwise store. I'll give you 2 more. The first one is just a hypertenancy maybe people Wes they say visualize, they mean people try to get you to visualize those things very vividly so you can be transported into that environment.

with low latency, which is still how you use it on serverless. Because on serverless, you don't have a file system where to replicate that file. But if you do, that's what we suggest. Oh, man.

either 40 0 or very low latency We had this company and they only go as back as far back as 2005. So it's it's not there. But then this guy that was very high profile, very high profile in the Linux Vercel, he was the maintainer of all the virtual file system interface all viral.

that client can read data in the past So the developer experience of developing with SQLite is absurdly good.

you define a period for the sync. Let's say 5 seconds, 1 second, 1 minute. And then what happens is that if you sync more often, that's more expensive.

that comes back in a 100 microseconds. So that's the only like, it's all transparent, but but you one of them is slower and and they want the the rights are faster. The the rights are slower and the reason faster.

you now have to keep servers running, I I'm I'm I I really wanna stress that because some people understand that, like, if I don't do multitenancy, then and intrinsically But with Thurso, to begin with. Yeah. So so it's and building

given the prices of this right now. You know? Because, like, I think

that in which you decide what to do. Like, so do you ignore this data or you try do you try to write it again? I suffer from a condition But once we release multi tenancy it will cost me, like,

we our database is 1.7

what a lot of databases that go for massive replication they do, like our previous NoSQL databases, that you don't really have any guarantees. You you read, you write, and you read, and you may read what you wrote or Scott. And eventual consistency in the future, those things will coalesce.

you can just copy the file. And does that take long and that that uses up a lot of space? to do, You you can you you have the copy of the database in your device.

And I I think that's such a powerful way to be. It empowers you to to to do more with the data. You don't have to worry about the cost associated.

like, conflict merging strategies Yeah. Whether you're versioning tables, whether you're just overwriting, whatever. So it it's all very, is

Okay. I got I got 2 questions after that. So, like, the first one JS,

they just had a bit of a rug pull or the other,

situational dependent and and complex.

is that related at all to the the way that we're talking about,

that I as my cofounder was from, essentially, 2000 to 2,000, 2002 But if I keep the if I can keep the file in sync, in what we have today we have a concept in terms of schema databases and child databases. So when you create a database, you can if you'd say nothing, it's just a database.

I just wanna clarify I was a college student, as it was, There's gonna be a 2 day conference,

will this, like,

So you had mentioned it could be stored on on device and that will sync like we just said.

in the He responds to to my first contribution you managed to introduce 3 bugs in a single line of Node, and people like you should never be allowed to get close to a keyboard again. So that was my welcome to the Linux kernel.

on device, and it is. It changes the way you think about saving data.

in New Jersey and I have,

That's right.

then the only way to do this is to make things less transparent. Because,

the edge functionality

to serve those requests from those other locations, I don't have smart stuff in my Node, It's Dallas. Yeah.

you instead of going into a server, We're not just after the people who are doing multitenancy.

like yesterday. Right? Just then and then you'd at 3 PM because the the the the problem there is that, like, some people just shoot themselves in the foot. So you you we give you a new URL with that database in that point in time, and then you can go verify if this is really what you wanted, and we really had that symbiotic 5 years ago. It has the the infamous Durstree will allow you to create a replica of the database and operate in replica without disrupting the main service. Right? So one of the things that we do wanna allow people to do is, like, have this flexibility and and and go wild. Don't fear your database anymore.

Mhmm. And, like, that

in in our defense, I mean, Pnpm and I come from a very different background.

gigs,

LSM trees and and b trees. There are very directly.

if I have a database That is really cool. And

And the reason we knew we could replicate SQLite massively is exactly because this file based, low cost nature.

in microseconds.

accessible thing that you can access, journal mode before, which Wes, forced you to have a single writer.

on ESLint.

So you can still see all the data in your database there.

we wanna change that and we we in fact have it on our Node map by October this year to change it to allow rights in the replica as well. And now you need some form of conflict resolution. Right? But the the the one of the reasons we started with just read replicas It takes the same right? It JS just the file. So this architecture is very beautiful. Mhmm. But but you you can at any time you can at any time in Turso, right now or are you? No. I just Wes when I joined, when I joined Datadog, Like, they were all, you had to scale it out. And Mongo was at the time the best choice and and to do that. Like the SQL databases, you would have to manually chart chart it, which is not a pleasant experience.

a a huge problem to understand what's going on. It's funny when we were looking through the Replicash docs, and it's like, there's a whole massive, like, 4 pages on,

simpler architecture that allows you to do like, any other database, you have pages and then and then you have because, you know, it's free and blah blah blah. So that was and and I loved it. I didn't care about that because I didn't have kids. So it's like if they starve, they don't even exist to begin with. It's just me.

Well yeah. Give me that, that,

But you can still see the data you can still see the data droplet is offline, it's the same story. You you can read from it. You will not be able to write.

a VPS, an AWS server, anything like that, you're actually better off don't use our edge replicas. I can offer you Node better. Just replicate that inside your own server.

in in the local endpoint,

let's use Drizzle as an example, like d b dot execute and then you put a insert query. That works.

a very unfortunate meaning because what people people came to understand edge as being And in a sense, I was hap in in a sense, I was blessed because it did not get, you know, it doesn't get much worse than that. And that was already like the my first impression. So things actually got better after that. Right? Just I am pretty much, on the, win win relationship You can encrypt that database of different keys.

you think, oh, what what more can we store?

you know what? I need to be very careful how I I I construct my tables and how I query my tables because I don't wanna have an n plus one problem read from a file locally in 200 microseconds,

it it's it was the right choice for us. In fact, I would love to know your opinion, since you're as as users. But we don't roll back to the same URL so there's some bandwidth cost in in there, which is fine. It it's not the most expensive part. But if you're not querying that replica, you're not paying anything,

run it on my droplet, and it will sync with the rest?

to host it, or if you have, like, a, like, somebody's phone or whatever,

Yeah. And I'm I'm just thinking, like, how awesome it would be to how much more you can store

Right? So one thing we really like about,

And our approach to that will be essentially

instead of

something that is not compatible with Node. Js, which was mind boggling too. Because it edge edge in my mind is a concept that is geography. Responsible for that slightly.

my serverless function or or edge function or whatever is gonna connect to the closest version of that database. Right? So if the serverless function is running in in Australia, it's gonna try to find the Australia one. And if I update any of those any of those databases, they all sync with each other. Is that right? Yeah.

And that Tershow does that. Like, I'm able to just like, if I have a a 1 gig database

completely for free, something else. Node real visualize being a metaphor. And then I found out that actually apparently 4% of the population are like me.

6 months ago or so. Like, they took away the free plan. They jacked up the price on everything. So I'm here to tell you to please don't do that to us as well on.

then is that SQLite can be a lot more affordable Now and now we can offer you like those 10,000 databases, which is what we offer for $29 a month. So for $29 a month, you have access to 10,000 databases.

because the the the data hasn't reached that replica yet. And the reason that works well is that this is the same consistency model that a transaction offers you. If you start a transaction now, You can be more aggressive with your back end optimizations because you know that nothing that you do

understand what's going on. So we have a model in which there JS a single primary.

But Wes despite that, I still really do like Planet Scale because of their, like, branching feature where it's like Yeah. Oh, yeah. I wanna work on a I wanna work on a feature. I'll just, like, take a fork of the database so that I can work with the most up to date data and then sort of submit, like, a pull request for the the schema updates as well. It's is there anything like that? Like, what's the what's the flow for that when you're working with SQLite?

that is a pretty small database in terms of the pricing for.

Right? Because you're transferring frames more often and you have less coalescing opportunities.

which is it's relatively small.

the Scott that we wanted to tell And we wanted to implement Scott.

easy it is. In in fact, that is the one thing the the one thing that we did not talk too much about.

So all your rights are gonna go to your main region, which is likely like New Jersey in your case. But your reads can be served from any other database.

Is there anything about Terso specifically that we didn't get to before?

are not a great idea because your your database is always gonna be your your bottleneck. Right? If you have database.

JS that work for, like, rollbacks as well? Because I noticed,

Maybe we take some some questions that are are not Terso related.

Look. I I think we cover, Node one of the things you wanna know, is, correct me if I'm wrong, is people really that harsh and and are people really that harsh in The Little Sparrow? Mhmm. I I don't I cannot visualize anything.

And again, SQLite SQLite is part of the reason why we can do this because do you know how our branching feature works?

in a special way, you can the the same way you can still use git the When you go into conflict resolution, when you go into, like, okay. Now I need to allow because you drop the baseline cost. Alright? So so there are many examples in the industry across time in in which, Europe, whatever you are. If you have a server,

Will it

Yeah. There still changes that. There there still changes that radically. You can just Yeah.

yeah. That makes sense. And because a lot of people always say, like, edge functions

but using a lot fewer resources. So that's number 1.

What what exactly makes SQLite,

as well as as JS Wes as cheaper. So I think that the the the fire Bos model of SQLite I think it depends on what you mean by how lots because SQLite is used by potentially trillions of of devices.

SQLite being just the file, the file is always there. If you send an HTTP request that that gets served, about what we offer is just like, hey. Start with a file.

there's no buffering. There there there's no extra anything. You send this HTTP Wes. You serve this one request into that file and that's it. You didn't have to bring compute. You don't need Kubernetes. You don't need any of that. So you end up having a cheaper infrastructure to run. You you end up having why 3? both in terms of cost and and pretty much time JS as Wes. Because, you know, fire 10,000 queries And number 2, because it allows for for those architectures being just a file.

And because it doesn't cost me too much to to to serve those reads. Right? It's just that, like, a Scott of things got simpler, So let let me start with a story. Let me start with a story. That was around 2003.

camp of

in And

but I can now create a replica of that database in a file that sits in your own server.

like, a cheaper to run than a traditional MySQL or Postgres?

making it runnable in toasters. I mean, the toaster thing is mostly a joke, but at the same time, it isn't because yeah. I bet I bet some toaster out there runs Sanity. I mean, just, Man, fighting the big guys with a 14 people team.

First of all, the 20 years of optimizations this understanding Allow for those architectures where where, for example, if I had developers

a dollar a month just to keep the database online in in a container running.

Wow.

at the same speed or faster,

are are enabled by the SQLite model. One of them, which is my favorite, in fact, is is what we call embedded replicas. So embedded replicas allow you have your database accessible over HTTP, You end up with a service that allows you to do a 1,000,000 reads per month for for free.

being charitable other transactions will come in in the middle and update the database, but you will still read the data as if you were when when you started the transaction. Okay. So so you have that. But your rights are always gonna be slower in Australia create a new database do it by all means.

environments, which are low memory. So people were very excited about the idea of Trussell. That's when we were releasing our private beta.

a a a query that uses fewer resources to be fulfilled, That is how a lot of people develop. You can develop humanly expensive procedures. Like, you had to stop what you're doing.

And that file is kept up to date by the server. So now you can very well, the messaging and let's keep I understand most of your audience will be a JavaScript

So if the if you use fewer resources to, and runs.

we've we're now we're now on the AI craze. You might remember a couple of years ago, you had the IoT craze in which you had to have smart everything. And I'm sure at that point, somebody, came up with a toaster that was a smart toaster So it it is you you don't write to them. You you you write to the main database. So you still need, Today you still have the single writer model, but you can still read And creating branches and merging branches in those tools were really

to make it runnable in in in those So first of all, that was a long time ago. I I started as And why not 10,000?

to give every user a Postgres database, So you had, you you you kinda had to do it.

and and the very few things there JS smart devices on my home. I keep a gun around. If they do if they make a noise that I don't recognize, I'll shoot it.

So there is there is a general sense in in in which Sanity is just cheaper. Because again, it's simpler. It it will do fewer things, but those things they will do, it will do like but but, I'm gonna be there. And then in September, mutations.

a database costs as much as the resources needed to fulfill the queries. Mhmm.

It's not all about being cheap as well. We also have a couple of features that I think are very unique to and every write that you do in the database took a full lock. You could do nothing else when you're writing.

Beautiful.

my joy recently have been trying to figure out more about this condition.

from Fantasia to databases to in the Linux kernel. I would love to connect with with folks, there Yarn there.

And when you wanna connect to the Thurso service, then you can replace that file URL But I give you a new URL with that with that database. And now you can go verify and see if if things are the way you want and and switch to that URL if if if yet if if not, you just discard it. Right?

Do you use Linux as a a desktop

A lot of things got simpler for me on a Mac, so I just keep using the Just Yeah. I Node that.

Cool. Let's move in the last section of, the podcast here, which is, 2 sections. We have our sick pick and our shameless plug. Did you come prepared with those?

wearing, like, nice clothes and and But the one thing that we got from the private beta users is this is really hard to consume because it doesn't work with the ORMs. Right? Mhmm. And then we had this this Drizzle was just popping out in the scene at the moment, They that compute stay up for 5 minutes to handle 1 request and then it shuts down.

No. I I feel like I've heard of it, but I'd have to look up. Yeah. So all my life, peep people would tell me things like, oh, visualize this and picture this in your mind and make it like, if you're trying to do meditation, for example, And the technical answer at the time was that we'd because the way we architectured the service ESLint, again, in 2 months so we could release it.

how the mind works, how the brain works, and and why is it that I I cannot see a apple based on another database, which is a branch.

no. It's not a metaphor. Apparently, normal human beings can visualize the damn apple, and I'm the one who can't. And and there's something wrong with me. So, But you can create a schema database and then you can create child databases that are connected to the schema databases.

as a which is why like our messaging today is not even though we do replication very well, even though we do, hypertenancy our Discord community JS discord.gg/turso.

and and I'm asked to imagine an apple, I don't see an apple. I can describe an apple to you. I can remember what an apple is, but I don't see an a apple.

that allows you to get the good stuff about SQLite.

Peace.

So she it's just if she if she's watching this, which I'm sure she's not because, you know, she's not a techie,

pick sickness pick, except there's nothing that I can do to to cure this apparently, but I've been reading a lot about it. And I've been,

the empowerment you get

I'm not a front end developer. Again, I'm a Linux guy turned database engineer, now turned database company CEO. So I do not understand React.

being able to understand,

completely, and I I I don't even try to read books anymore. And it's made my life so much better. Yeah.

Yeah. That's awesome.

I I did not like hearing that.

but but that was, that was that was my entry point. And, is the story in a sense that we are achieving today with embedded replication.

I does not have pictures. If I close my eyes DigitalOcean droplet transaction model. It's not like a NoSQL, s. Right? Nothing. Like, my mind is You have things like InfluxDB.

feel free to, which local the thing is that I, at the time, I actually appreciated that. I I, Don't don't trust that don't trust that too much. But look, it runs everywhere Wes it's it has, like, 20 years of optimizations if if not more.

So so audience by and large. Yeah. So you create a client using Drizzle or using Prisma or just using the raw client, Libsico serverless platform, you as the user defines what happens in this case. So if we detect the conflict, we call a callback that you defined When you change the schema of the schema database, it propagates atomically to the child databases.

what makes it all so special. So thank you so much for all that. It was a really great game to chat.

I very proudly And then they say, look. I mean, if I can guarantee this at the database level already, a lot of my back end become a lot simpler.

which maybe will be my sick pick. I will I I was trying to the other day to do meditation, etcetera, which is read your write semantics. So in the same client, reads.

and multi tenancy, hyper tenancy JS we've been calling it, and put JS in production. So is that you create a client, And but also but also because we believe in simplicity.

But but I I still I still have a bunch of Linux servers. I use Android for phones.

I I I had a quick question. You know, you had mentioned that you had done a lot of work on the Linux kernel.

to develop.

it's not about the label. It's about

she she will be,

it's it's not about the label that you get.

And then you have a token, you have a sync period, you have sync URL. You have, you know, for for you you have extra things, but the main thing is the URL.

And for a long time, I was trying to force it. Like Mhmm. I feel everyone says you should be reading. I feel like I should be reading.

and I was very interested in open source, and I was very interested in low level systems. And Linux fundamentally There are 3 things.

And one of the things that she talks to even with parents

It's about

And then at some point, once I realized it's not good for me to try to do that, I just embraced audiobooks

there was and

pick, except that there JS nothing that I can do to once this becomes But I like the fact that he was not telling me, Push your data as close as possible to you, make those latencies very small to nonexistent, and that can change the way you think. However, what happened is that

I I got a Mac and then I just got used to the Mac. And and, what in my life gives me joy? That I I'm I'm happy that my wife is not listening to the podcast because, like, she's always complaining. Like, nothing brings you joy. You are probably

and that other human beings are different and and how they're different and and what is the baseline Wes interacted later in more pleasant ways.

The other day the other day I found out that, Because

I I suspect I Node.

I I will be in August in, thanks be to Node,

you know you know what? This is not a sick thing, but I wanna share something which which which again is related if if I may.

Absolutely. My pleasure. Thank you so much for having me, folks.

you you may find, like, a Stockholm syndrome or maybe I'm a masochist or whatnot. But

on my especially now that I don't Node that much anymore. Like, I'm just managing the company.

20 year 21 year old guy, like, just trying give this answer to ourselves and our investors and our employees and everything. Like, how do you make money out of this? And it's not by being a story.

We can't visualize And Lambos Lambos for for for them and the West Coast for the JavaScript community. I'm happy to put it that way. But, something that was very expensive now becomes very cheap and then allows you to build things in a different way and allows you to build things in ways that you wanted to do before but wasn't possible. One example that comes to mind is Git.

visualize yourself on the beach.

Or you don't or you keep the file URL and add a sync URL and now you're reading from the local file. Right? So one of the things that is very good, But when you pass a file, that's just that it's just this. That's a file that and one of the things that people love about Turso, and if you go in our community, that is 10 milliseconds away.

to use, to use their own databases. Right? Just that without having imagine that which is by the way, one of the things that we do not talk too much about.

and maybe things got better. I think I have, you know, when I was leaving, I think people things were already getting better.

your application in a file. You do not have to connect to the service

cloud database. But it's it's been really awesome to dive into

look, I think, I I don't follow Linux that much these days, the the development, for some things will need.

to understand software.

the worst person.

and I just know I'm physically incapable of doing it. Right? So just I have to find another way, which is Yeah. That resonates a lot with me. I mean, I I for a long time I'm I'm a really bad reader. I have dyslexia.

I'm just curious about that. What was what was that like?

attaining any meditative state.

And his response Wes, this is the worst piece of code I've ever seen in my life. You managed just a open source SQLite clone or fork or whatever they run on device. You have to have something.

a bunch of Century folks are going to be there as well. Awesome.

local SQLite. I mean, it technically speaking, that's SQL, the fork. So you can do vector search. You can do all of those things.

I thought petabyte scale workloads.

Thanks, Adam. Peace.

called aphantasia. Have you heard of this before? by doing multitenancy.

And for all my life, for all my life, I was always thinking First of them, just how incredibly you know what? I I wished, in in future contributions, we should strive for a more synergetic approach to get our OKRs in in a better place to,

I love it. Well, thank you so much. This has been awesome. You know, I really illuminated to me, like, what makes Terso different because it's easy to look at things and say, oh, it's another,

But will cause by accident you to mix data from different customers.

You don't have to spin a container to develop. You don't have to, like, keep bringing databases up and down on CI to develop. You can just put the flat file there.

by and large, people do not believe in open source. Right? Open source was the thing that you if you do this, your kids are gonna starve Put in put in your user's mobile phone so you have 10,000 users. Keep a database per user and replicate the database individually to each of their phones and encrypted with different keys.

And in in a sense, that these are the database well. So you can say give me give me a copy of this database we have a product decision on top of that

and I completely failed. And I think because of this, And this is probably one of the smallest heavy hitter database players that you've ever seen. Like any other database company out there is is at least 5 times our size.

my cofounder, just had like, if if you had anything remotely serious, you would go for NoSQL. And the reason for that is that the SQL databases didn't scale very Wes, and you didn't really have a lot of hardware. Your phone today probably has more compute and storage capacity than a server had in in in 2010.

speaking on a very corporate We truly believe in simplicity as a goal. We it it's easier to cognitively But the reason we even say, like, create a database based on another database is that you can create that based on any point in time allows me, for example, to try because I I was always fascinated

at any time. And, like, I was starting to see, like, the folks that were later in in in college, like, 4th year, 5th year starting, like, their internships.

was just the perfect intersection of this.

Mhmm. I say, oh, that must be a metaphor because if I close my eyes, I see nothing. And and and how resource efficient and memory bounded it is to serve your queries allow us to build a service that has all of those properties.

And I just consider Go go

reads which which is we decide, okay. So we're gonna go all into this. And we had the platform right after it. And and one of the advantages that we knew we were going to have So which is why, like, And if you pass a file, trivial to run? Yeah? Yeah. Yes. Because at the end of the day, it's trivial now to extract the copy of the database.

finding, a lot of fun in trying to understand,

I learned recently, like, 2 months ago that statistically speaking, you will see an apple if if I tell you I and I don't.

It it's it's all you Node. Like, it's the simplest getting started thing possible.

Yeah. I I think that is a sick pick though because, you know, what? You know, my wife's a doctor of psychology.

is telling me that I should never get close to a keyboard again because he thinks I should never get close to a keyboard again.

Learning c by reading the Linux kernel source code, and here's my contribution.

Imagine you're but you you hinted at this, Wes. Like, you you you are not you are a front end developer. You don't understand databases very Wes, at least not as much as, you know, the core database people in the company.

the way I saw the tech industry back then

to understand yourself Yep. Because you have that that that bigger picture now of why you're like this and that. So it JS funny because, you know, parents sometimes they because she she's mostly testing children. They get wrapped up and, like, I don't wanna put these labels on my kid. It's was, you know, a big factor in some of the direction you went.

sort of a

To have a good time series database, you have to have a completely different on disk representation.

not just based on the words inside of it, but

numerical representation of the data that's in it. Right? Mhmm. And if you want to, I could take, like, a whole bunch of photos or a whole bunch of text and convert them to embeddings. And if you wanna find similar text,

unequivocally Wes. And and, yeah, I'm I'm I'm hoping, 200 milliseconds and you just do this. And that's just a hash, and that's a branch. And and, like, eventual consistent database that you always write like idempotent again, it's open source, so people use it. And we know Node people using it as just a replacement for SQLite. We add a couple of things new workflows were enabled.

There are very complicated technologies that allow you to query time series.

what do you call them, vector objects

like SQLite adding it as a feature?

you know, obviously, people will disagree. And if you talk to, Every database was in a VM. So we didn't have any multi tenant capabilities.

very early on, we saw a lot of just, like, custom vector databases.

And and I think now that everybody is looking into that because of the AI boom, But our users who Yarn at the time on the private beta so I I I found that refreshing in in the beginning, if that makes sense. And and later I got tired of it, but but I found refreshing. It just it's not just a financial thing. Like, we really believed in in what they're doing.

And I think you could tell me. Are they getting their lunch eaten by just, like, regular databases,

that are are close to each other,

And people have been using, one of the cases that I am aware of for people using Libs SQL, nothing to do with Tersu is exactly for vectors. So you have this vector thing that is just there. Right? those changes, you have to be off online to make the changes.

somebody from a vector database, I will expect them to disagree. That's the they they they're financially motivated to disagree, which which is fine. Yeah.

And our build of Libs SQL has vector, fake way.

a NoSQL database, and now you're working on SQLite.

The meaning. The semantic meaning of what those things are. Yeah. Semantic meaning. Exactly. Or, like, pictures of of what is in the photo, etcetera.

to query I think you sponsor the development of Drizzle quite a bit.

Interesting. You had mentioned that,

Things are not represented as relationships. They're represented as series, and then you you you have to be able to very quickly do aggregations across those series, which is a completely different set of queries And the reason we decided to fork SQLite is that we figure And in that client, you're gonna pass a bunch of options. One of the options is the URL.

And embeddings are,

Why the shift to relational

vector,

which is now available in the Thurso platform, but it came first through the open source.

which is pretty cool. I know that it's probably a lot more than that, but the idea is with with AI right now, a lot of people are creating what's called embeddings.

I think the answer is the

vector

you know, your prior experience was in a NoSQL, So just understanding yourself. Is there anything you would like to plug? Any events coming up, anything specifically, or, just anything you'd like to plug at all?

because the isolation is now guaranteed at the database level.

Like, why would it make sense to put every single user in its own database?

I Tursus is not for me, and it's not true.

and it it gets propagated to the other databases. So part of what we do is is making that model, We, something before you did that invalidates the transaction that you had.

are made simpler

lots of things in your back end become a lot simpler And some of the things that those databases do can be quite complex. Think for example, in time series.

We don't we don't mind. I mean, do do you have all of those databases. I'm not gonna take offense if you use only 1. That's fine.

But some architectures are made simpler by this. Imagine, for example, that you are a company in the financial space.

Right. So you can essentially do that schema change in a single ESLint, as one of the core contributors

And then we add the capabilities to our server But but I don't think vector is look. I I worked as an in in a Node database for 8 years before that, which is a columnar store. I'm fairly familiar with with NoSQL rally in August.

Yeah. So offhand, you had mentioned that, SQLite being cheaper

And those are the architectures that I believe should do multitenancy. I don't believe, as good as it is today. It was essentially like, okay. Now we have this this this thing that's doing well. Our fork of SQLite got 1500 GitHub stars in a week, And if you're syncing less often, this is cheaper, but the data is less fresh. That's a trade off that you make.

a lot of architectures and and what we see a lot is people in the health care and finance space being heavily interested by this because those are heavy compliance based, industries in the exact same way you would have used subversion or CSS back then, of yours.

making the model feasible, which starts with here are the 10,000 databases for $29 And then my left 1 hour yeah. And and apparently, At the end of the day, SQLite is just a file.

Right? So Yeah. The way things work with Trussell But then when you do d b dot execute Vercel, all of this is transparent in. So built. And then we built the managed service, around it, registering, which is what Duracell is today. And if you want to see all of the errors in your application,

in in which data segregation is very important.

The first thing, I mean, I I wanna clarify is that this is not what Thurso is. This is one thing that Thurso allows. And and, And then when that is when when that is done, you shut it down. So you you really can can go take multi tenancy because those databases also come online in 500 milliseconds. Right? So in 500 milliseconds, your database is there. Wes for an hour, shut it down. That's fine.

or I I I as I've been calling recently, hyper tenancy because you don't even you don't need it you don't even need those databases to be long lived. You can create a database that lasts for an hour to host the session.

If you can't like, it's not because you can create that a 100,000 databases, doesn't mean you have to. So if you prefer to put everything on a single database, So now you you are faster, right, and more if you are on pro and enterprise.

If you if if you are, automated, So I'm always trying to get close to those communities, understand what matters to folks like you so we can provide those things to you through. So so I'm gonna be on conference, Vector is not like that. Vector is relatively simple.

You can have multiple you can have a database per client come chat. It's always a pleasure for me to to chat about anything really, and in a company that that we joined that is is a database focused on the NoSQL,

Tursu started with us.

in which, hey. If you guys can support us, we will be happy to sponsor you.

of a fork of SQLite called Leapsychl. So this is, you know, the the elevator pitch, and then we go into the details of that. But the open source project was found a what I consider to be it wasn't even a bug. It was like, hey. We could and But but but but you can. But you can. And like, this, you know, this is how the relationship started, but we've been very impressed with what they're doing. I think they've been very impressive what we're doing. And Yeah. You know, we we keep sponsoring. We we wanna keep helping. We understand that they're not a company. They are a group of dudes from Ukraine Yeah. Which makes the whole thing even more impressive, right, Justin? rights to happen was actually the maintainer of the memory management subsystem for Linux at the time.

managed version, Because if you're replicating any other database, And what we've done later JS okay. Now we build the the level of isolation. CColite itself is really good with that. So c cola, you can you can determine how much memory maximum you're going to use.

And, you know, very early on I go I hope you guys are Scott, mad at me, for playing both ways. I do No. We love we love Laravel. Yeah. I've spoken at that conference many times, and, what I see is that some architectures and then you get your on ramp to run this thing in production with with with replication, with, with backups, with point in time recovery, with schema changes, every every episode of my podcast gets its own database. That's fine. If it simplify things in architecture, you can do this. And the third Node is every time you were thinking, than a relational database would do. And those things are very, very, very complicated. You have companies,

Totally. Yeah. We just we just talked about that. Cool. Well, let let's talk about Terso. What is Terso?

yeah, like, we all we were, in fact, the 1st corporate sponsors of Drizzle when right when they were just getting started. We're very happy with the with the partnership. We're very happy with them.

Welcome to Syntax. Today, we have a really good one. We're talking to Glauber Costa about

Yeah. Yeah.

Terso

distributed databases in general. We're gonna be talking about SQLite and why people are using it these days. And a little bit kinda kind of a little bit of a glow up recently in terms of, like, its public perception, it feels like. So we're gonna be talking all about that and more with me JS always is Wes Bos. What's up, Wes?

And I think it's safe to say they'll correct me if if they disagree here, but slow.

and SQLite. It's a distributed SQLite platform that is really just getting a lot of attention from a lot of developers who've tried it. So we're gonna be diving into

managed offering,

And, kept asking us,

Hey. Not much. Excited to talk about SQLite and Terso and everything. The, the dudes from Drizzle had good things to say about you. Probably half because they like it and half because you,

and I'm happy to go into into more details there as well. But, like, very early on, we had this issue in in which by positioning ourselves as an edge database People kept telling us, okay, that's great. But if I have to change the scheme of those databases, I'm kinda screwed because that sounds like a tremendous maintenance burger.

and became this specialized thing that some use cases that is incredibly fast with 0 to low

around 2010, you which is invitation only.

that? Awesome. So so look. This this is great because when we, going back to our history and and and how we forked the this database, At the moment, we're actually planning to change that. Then I'm happy to tell you the future road map in here, which will answer your question fully. But at the moment, replicas.

ceased to be the default, The write is just Wes we're gonna make a little bit, we're only gonna make a little bit of money because we charge you only a little bit. It's not something that than any other SQL SQL database. So this this is still something that underpins what we do.

why while you you are writing, which means, okay, that's that's good enough for the for the vast majority of of read mostly workloads. So the software work better, improved.

Postgres improved as well. MySQL improved as Wes, and the hardware improved. So you don't have to do NoSQL anymore. And I think what happens that NoSQL with our initial edge positioning and a user requests a copy of their data,

So we offered we offered people at the time 3 databases for free so you could come, and and you could just at the end of the day, Park City, Utah for React Rally. So again, I don't, I please be gentle with me if you're there.

And and that's fine. So you don't have to necessarily scale out anymore.

sign up on on, get an account, and you would get 3 databases.

that had any mild capacity, Yeah. And and if there is any if there is any way that I can still, But I always like to be, you know, those are, we we have a lot of users of coming from those communities. I think Turso is a tool that empowers, Terso is a I mean, I am a a little bit of a dinosaur, visualize to fulfill that query, it costs you less money.

your default choice.

you can literally give every single user their own database, which Mhmm. Seems cool, but then also I'm like, well, then that seems like a bit of a maintenance headache and whatnot. You wanna talk about

databases?

Today, Sanity does not

And I was just more interested in in, you know, in going forward the core, and and that's why I landed here.

instead of having 1 big database,

I

the software in SQLite today is not the same as it Wes,

Mhmm. And so so what I what I found in the market is that by a large group together as NoSQL.

You can have a lot more for your dollar with Sanity. And this is true for every SQLite offering than you would have for other databases.

But fast forward to today, the harbor, advanced a lot. So your phone has the same capacity that a server had back then, but a Vercel, you can go on AWS today and and rent a behemoth of of machine Mhmm. On with a click.

I wanna talk about the the whole multi tenancy or per tenant database because this is something I've been hearing a lot about recently where people say one of the benefits of SQLite is I wanna explain real quick for people listening. So vector search is the ability

we came up with the managed platform almost immediately after, which means it wasn't transactional schema changes query away. Like, query away. Your your your queries are now free. First of all, because we don't charge you for that queries. We charge you for the sync. So we charge you to keep the data in sync, but you you now can do as many queries as your heart desire well, Datadog being one example. At the end of the day, they are just a giant time series databases.

I'm not gonna say that I don't believe in NoSQL And then we have something special on top, which I think makes it usable, But then there is a very smooth on ramp to use this in production, a

anymore. It's just that I don't believe in NoSQL anymore as in which I'm querying the database all the time. This is gonna end up taking like many seconds and my API is now busted.

Yeah. Cool.

Not necessarily those users were adopting this pattern, but they were saying, I like the fact that this exist. I would love to experiment with that pattern.

which you can use for site projects if you have 500 site projects. Some people do, in fact. You could you know just Wes and I. Yeah. And and that's fine. But you can also use for multi tenancy.

SQLite already isolates fairly Wes. Then we change your architecture so that now we have a big VM with a lot of files on it.

For free, you have access to 500 databases,

And and and

and look, this is what you should do as a startup. You you're always listening and and iterating. So once we released multi tenancy, now we can have 10,000 databases.

but kept ESLint us that this pattern was a pattern that they saw done on SQLite.

Just the why why like, with the Thurso URL.

So why can't you if it's just a Sanity file, why why can't you give me 10,000 databases? I am terrible of being prepared for anything. I just Okay. So I definitely think I did not Anything in your life that you're a big fan of It's giving you joy. Coding related or Scott? And I sent my 1st contribution to the Linux kernel. This again, it was either late 20 2003 or early 2004. I don't remember. I actually tried to find this message on on in archives,

By the way, our vector search implementation, was, like, this very fake place

I have this pattern that so the users themselves as they came, and and things that will require visualization reshuffle this code and and make things a little bit better.

Like, goes back to goes back to that conversation we're having about architecture. The thing with multitenancy in in a single point. So what we're doing essentially is adding all of those features and functionality in in in in a hotel in Provo, Utah, and I'm gonna be at that conference as well. We at Thirse who are sponsoring that conference. So, like, if you wanna chat and if you are in any of those 3 places, And edge for us was just

And we weren't really happy with the solutions that existed at the time to run SQLite as a server side production database.

that and that translates into essentially a cheaper offering.

My toaster, like, you run SQLite and your Apple Watch runs SQLite. I mean, it's everywhere.

capabilities what you can do with this model is that you can do offline But what we wanted to do

We were seeing a lot of people trying to run this on the server, us included. We were using this in in our product.

to use it as a as a production database for server workloads. I mean, SQLite has traditionally been used in in appliances, mobile devices, and we don't use the fastest algorithms. We use the most memory predictable algorithms exact exactly to be able to keep offering this multitenancy thing with vector.

was creating a product that was not a database, And I don't know if I ever will, but at least now I know that if I am to, I have to try a different way. I can't just try the same way I've always been told because it always involves some level of visualization, essentially saying, like, the machinery on top to make the model work and and and make it seeing where the pinpoints are,

felt like we had the technical

And just one day, we decided to fork SQLite.

but it was a product that used the database like many other products and and still in the data infrastructure space. And we used SQLite very heavily.

So we looked at the code and said, hey. Look. I mean, just we we Sanity really a bunch of changes that we can make sure this database And in in the same client that you created, if you write, you're gonna read that data back. But a new client that is that is just getting started now, Okay. And and and we all this happens, you know, just people once you put a term out there, like, people understand the way they understand. Like, I'm I'm not I'm not assigning blame to any specific individual or entity, or anything. But the fact of the matter, is that we had many many potential customers before when we talked. They would say, I love the idea what you're doing, but I'm not using the edge at all. I like Node. JS, so I'm not gonna use it. So, so it was a word that is was very confusing.

you'll want to check out Sentry at century.ioforward/

syntax. Again, we've been using this tool for a long time, and it totally rules. Alright.

that would allow us to run this on par if if not better than Postgres, MySQL, Mongo, and and any other offering really. But SQL like the project doesn't take contributions from from anybody.

So do you have a a lot of folks using Lib SQL

as just Lib SQL without, you know, using Terso at all?

syntax.

You don't want a production application out there that, well, you have no visibility into in case something is blowing up, and you might not even know it. So head on to reduce entry.ioforward/

One of them being vector search. So you have native vector search in the Libs SQL build. Oh, cool. Yep. And, you know, I know of some people using it. But Wow.

to that fork that have nothing to do with the TURso platform.

And so all of our efforts, that Yeah. That's the name of this feature. Yeah. Yeah. Embedded replica.

Mhmm. So torso as a company, you know, what's going on. So Oh, my god. That Wes liberating. That was liberating for me because, again, all all my life, I thought back then, SQLite, is c colite for production. JS you you start with with this c colite idea. We start with with the file,