Local Data: Sqlite, LocalStorage, Session, Cookies and IndexDB

that data in local storage. And same same with,

always send that data back to the database every single time that it changes. Right? So one example I'll have is on my course platform,

don't necessarily always have an account for somebody to store that in, or it doesn't make sense to But, we had Johannes Schickling on, and he's one of the devs behind,

they'll they'll get synced to to the back end. But,

immediately.

it's nice to stick that in local storage or or in the browser when you need it

the videos, your your data between users of the same device,

I forget how long. Every every time the video current time updates, which can be every 2 to 3 seconds,

I'm

or even maybe paying for data syncing. That's like a thing where people pay to sync the data, and they say, oh, actually, I need to have this available on all my devices.

Yeah. Another thing too often that, like I think you see this a lot more native apps, which is something that people, might be doing on the web more and more these days is letting you use the thing before signing up for an account or even any sort of login system. Right? And I'm gonna mess this up. I, in WASM.

putting

as people are watching and it just knows what to do. It's sending messages back and forth. You're not being like, oh, you're not writing the glue code yourself to sync it and query from the right place because that sounds like a

every

o p f s, which is, the private file system.

So

whereas session storage would as long as they close the tab,

that's where it exists. Yes. So that's all great. Right? You can store data in any of those numbers of ways. So, you know and and primarily Wes you're doing this stuff, especially for no particular reason other than I just do. But nowadays, if I'm storing

email us, whatever you got. We're at syntax f m. And,

then the data's there. It doesn't take any time to read it from the local database. It's infinitesimally So, man, all from the client side,

SQLite

if you close the browser,

Can we rattle through a couple of the the possible options for this? I know we did a whole show on it, but,

how how you can use the 2 of them together, again, to do your drizzle DB selects and all that stuff,

Electric SQL has kind of undergone Drizzle does have they even have a whole page, which we'll link to, which is, Drizzle and PG lite, Rx DB is a cool one as an option for again, the ones that make this the easiest are the ones that kind of combine the 2.

storage, byojs.devforward/storage.

is that instead of, like, the naive way I've done it the naive way a whole bunch of times. If you need this sync, by the way, So a couple of these things are are using WASM. So one of them is SQLite There's

pull something out of local storage and assume it's an object and assume you can JSON parse it, but

of you don't know JavaScript increasingly more of an interesting option.

IndexedDB

is

is is where I'm at. Authorization is like a like a credit card thing to me. No. No. No. They're they're they're,

And the stated goal of this project is to have a simple key value storage API which, like a lot of people, I think,

WebAssembly SQLite.

HTTP cookies, which can only be set and read via

Yes.

is a use case

as well.

Reason being is, anytime you have a request to that server, you need that information there at the initial request. That way, the server can be able to authenticate it JS you go or authorize it. Man, I don't know about you, but I I flip flop authorization and authentication yeah, basically, it just gives you the same API to work within all of this stuff. And I thought this was a really neat API that you don't really hear a lot of people talking about. That's cool. It seems like it's only key value, though. Right? So you wouldn't be able to key value. To query. You know what I really like about some key value stores? Deno's key value does this, is, Firebase competitor that does local storage by default. That's pretty amazing.

things that would piss the user off if, if they lost their data in between save or refresh.

And you're only ever writing 1 Node set of Node to work in. And and pretty much anytime I need to store structured data, But You wanna get the client side HTML and JavaScript as fast as physically possible,

but I think it could be, you know, greatly more used in terms of, like we mentioned, keeping persistent

via WASM, HTML.

obviously, because that's what it's made for. Query local storage. Right? You can simply just check if a key is there or Node, whereas

I'm curious how large that actually is.

by a 1.14

they say that it's hard to work with. And, honestly,

a little bit more flat.

the your application is reactive, and it's it's it's your whole state store.

the way these things will work, I believe, in the future, RepliCache,

So I I guess you don't have to, but filter and do what all the stuff you're used to. And I I picked up Dexi. Js for that background removal project I did, via WASM,

I I've talked about Replicash before. It is tough. Replicash is becoming 0 sync, which, PG light, which you know, the big question I have with some of these still is that because I haven't used any of these more intense ones JS is migrations and how do you handle that kind of stuff in a client side level? I guess you're

megs if you want the async version. I'm not sure what that means, but that's Awesome. Well, that's local data. Hopefully, you learned a thing or 2. Let us know if you have anything to add. Tweet us, Scott,

you're writing 1 set of, Although, you know, who said, I I believe there is, like, a penalty, essentially, like an initial download Sanity, where the 1st time you you load it up, there is a pretty large hit to use something like this. You have to download so the way

or

Alright. Peace.

in the space because it comes with, auth and MagicLink auth and all this stuff. Like, you could have it up and running in no time. You're just not able to run it yourself at the moment.

Oh, come on. Dexi I already know. Well, I, like, I I can learn it, but you know what I mean? Like, I wanna be able to just use the same first one is cookies. Storing data in your user's browser, you can throw them in cookies. There are

We talked about how it can get really complex with conflict resolution.

Because if the data JS living on the client, right, I've used it just, you know, raw straight up API, and I didn't find it to be that awful.

is a really cool option, but it isn't like I can't spin up an Instant DB server myself. And and that's the only bummer about that. Right? You're you're tapping into their ecosystem to to get going. But when I wrote an app with the Instant DB, it was probably the closest thing I had to get 0 to 100,

database queries and user settings.

fame. He has a library. He has, like, a kind of a a bunch of new packages called BYO JS that are all kind of cool. And one of these is a storage package called,

logging in. Right? It's the process of logging in, and authorization is the process of making sure you have access to the things you have access to. That makes sense. Yeah. Totally. I always just goof it up, though, when I'm I'm trying to say it quickly.

authentication

you close the tab, that's the end of the session, and it goes away. I I've never used session storage. I'm going to be straight up about that. I just haven't hit a situation where I needed session storage, but I I'm sure there's plenty of of these reasons to use it. I think a a good example of session storage would be

And it's exactly what you're just explaining. Right? Like, it's it's it's local, but you can also

So it's only sending the data that's changed. So it's asking for what's changed since this version. Oh, this has changed. That way, that sync message becomes smaller every sing Scott every single time, but it's smaller than it would be if you were sending all the data, meaning that that trip becomes much less, meaning that you don't have that crazy pop in or anything like that because it's not sending a massive bundle of data that then needs to get updated into your local database.

and and

And I think Drizzle even has support for pg lite now. Man.

to, like, a Cloudflare

going between the browser and WebAssembly, there's, like,

there's also a host of really,

and the only difference is is where you import those from. So you import it from IDB, for IndexedDB,

it's not,

mutations your local database. That database updates the UI, and then that information is in your your local database. You're just trying to keep those 2 in sync. At a basic level, for an application that 1 person is using, you're sending a message off on 1 device, whatever. That's very simple.

and objective data, I'm not picking cookies or local storage. I'm picking, which could then be encoded, syntax. Sign up again 2 months for free with the coupon code Sanity treat.

very distinct in web development

Yeah. 558 It's probably not the best bet, but it's Mhmm. It's a really simple API where you can simply just say local storage Scott set item, local storage dot get item. You can pull it out.

the author of tiny Bos, And

you'll be fine. It's cleaned up. Yeah. That makes way too much sense. So, thank you for that.

is a database, and you can query it and will just do it entirely in the browser.

Next up is IndexedDB, And, that's being created by the folks over at Electric SQL, which is one of these cool new local first platforms.

a cost you pay for going across that bridge.

much smaller than I would have thought to be able to run SQLite in the browser And that way, you'd be able to query all the users or just query user 123.

And via WebAssembly. I I think there's also, like, playback rate, volume level, like, all those settings. Right? I'll stick that in local storage, and then, periodically,

he knows the stuff, so I kinda kinda trust him with this. Yeah. There's also, another one, pg lite, which is Postgres

directly from wild new options

you will but let's talk about why you might wanna store data in the browser first. Right? Probably the first one is, authorization.

project. I'm a big fan of it. I wanna introduce you to a library from, yeah. It's tough.

binary to run yeah. That's a a great question. You'd have to run it in the in the user's browser because that's Yeah. I'm I'm sure more key value databases are like that. Sure. But that's Deno's

Oh, holy smokes. Four 10 KBs? How big? 410 kilobytes.

There's also session storage. So session storage, just like local storage. However,

Cookies have a limit to how much data you can stick in them. So, generally, cookies are only used for things like

in my opinion, is way, way, way easier than Replicash.

They fill out the thing, maybe they don't finish it, and then they close the tab. If you put that in local storage, somebody else comes around language settings, session IDs, just like identifiers in short little text us.

you can set an array of keys.

Session storage will persist

That that's definitely super valid, so you don't always need a sync. The the naive way is just to send all the information every single time, and that's too much information to send every single time. But if you're not dealing with a ton of data, not a huge deal. But the more and more data you add to the system, the larger and larger those sync messages are going to be. So a lot of the things that the the new platforms are doing is would be, like, patch messages.

that's it. Yeah. And let us know what you're using local data for. If there's anything that we didn't mention in this episode that you think is a good use case for storing local data, Leave a comment below, YouTube, Spotify, wherever you're at.

get set whatever. However Mhmm. I I do use Dexi quite a bit. Dexi is a a small library on top of IndexedDB,

That's what I want. I don't wanna learn IndexedDB.

And it's it's a really cool

in applications And so,

in the browser. Mhmm. So you have to download the entire a really hard thing to to do yourself.

when you were explaining it to me, I was like, I don't really understand where this JS, and he said he explicitly added a diagram to the docs

and it's basically just giving you a nice little API for creating, getting updating data. And,

and Flip flops to my dad for something. That I would use the word

that are loading this data, you can think of a lot of the times that this data, since it's living on the client, you don't necessarily always wanna wait for the server to hit the database, to do all the database stuff on the server before returning that is how you get it done with local data. Local data, I think, JS, widely underused,

settings for You

some changes lately Wes they're they're kind of, like, they're changing somewhat significantly. I I know it was, like, become version 2 now, and I haven't checked in on it, since they announced that it's going to to change a little bit. But Electric SQL,

everywhere. Totally. And especially even if you're sharing schemas.

ESLint drizz schemas.

potentially on a shared computer. So you imagine a job application. Somebody goes to apply for a job.

key value store is is 1. It's built on top of SQLite. Oh,

you could be fetching the data from the database. That database process on your your server side database could be kicked off right away. So that way, by the time that that comes back, you you initially have data. You initially have a loading thing that's beyond a blank skeleton screen, and then the rest of the data can be filtered in in any sort of way. Now you could have, like, a little syncing message to let people know that data's coming in, or you could just pop it in, which might not be a a great solution if you're you're getting your UI popping all over the place. Or you could have a button that says, you know, there's new data refresh, those types of things. But in reality, I think what it comes down to is that once you get into syncing data to a server, you're gonna wanna do it a couple of ways. Here's how I've done it specifically you're shopping at a store, you're you're you have a draft of a tweet, those types of things. Right? Anytime you want that state to persist is a perfect opportunity to save your users some headaches if you need it in the background, You can store that data. You can let the the people use the application even before signing up for an account

the server and server Wes, and then there are just regular cookies, which can be set and read via JavaScript

PowerSync, man, PowerSync is one of these ones that handles the syncing. But every single time I look at this platform, again, I I there's too many graphs and diagrams to the point where I'm like, alright. What are you doing for me here? Yeah. I I think there's a lot of different opportunity here. My the most exciting one to me is Xero because I've been, it's kind of the thing I've been, like, really hoping and wishing for. Working with. Yeah. And I've been using it for a little bit. Granted, there could be other ones. Like, Instant DB So you have a basic git pnpm, running the migrations on boot up or something. Yeah? I don't know. Yeah. Probably. I I bet it's you try to keep them Yeah. I think anytime, like you said, anytime Instant DB is a closed sourced is with storing things in Dexi.

It's it's not like the most fun API. It's not as simple as just,

you were just talking about like, you're filling out a form or job application.

decoded, those types of things, or a simple string value. Or you might use a local storage for preferences, anything that you need to just save. Again, typically, just a single string, save it, retrieve it, those types of deals. Yeah. Local storage is you can fit a lot in there, but the the thing about local storage is that it's it's only text. Right? It's key value store. So if you are trying to store something that is not text, you have to convert that to text. So if it's an object, you JSON stringify it. If it's an image, you you can store images in local storage by converting to base 64.

You wanna be able to refresh the page and that data to persist.

and it's stored primarily in your your client side, database. When you make a change, you push that change up to the server. When you load your application, again, you fire off a message that gets then pushed into your, if you hit the refresh button for it to remain in the same state as when you left off, you're filling out a form, for not waiting for a long server response.

innovators in this space along with the RepliCash folks and many of these platforms. Honestly, tiny base as well, for sure. There are a lot of these options that we've talked about in here. Yeah. I'm gonna paste in local first web dot dev. It has a whole section on different software for these things.

Like, it's a local database ish, synchronize.

small to load from a local database compared to, loading from a database on a server and building the HTML. So you're doing a lot more client side rendering, which is one of the reasons why I've been talking about client side rendering more. It's that you you're sending the information to the client. The client is then grabbing the data, loading it up, getting you all ready to go. And then, occasionally,

And, I I believe James was on the Local First FM, Local First podcast, by putting that that information in local data. Let me tell you. There is nothing more frustrating than filling out a big ass form and having the page refresh for some reason and having to start over.

typically.

in the same tab if you refresh the page. It'll not if you open up a secondary tab, and that's really important if you Yeah. Somebody is

You store things in Dexi, Yes. Next up is local storage. Local storage is often more used for single values like, a JWT, state for things like forms, surveys, People are wondering, what can we do that is beyond that data

or something like that.

opens up the same website, then there's a bit of a privacy issue because now you're sharing what it's doing is it's literally running SQLite

data, like, beyond single values or whatever, I'm I'm using, Yes. And one of the cooler things that many of these new libraries are doing from local storage, your session storage cookie, like, normal databases that you work with. And for the most part, I found Dexi to be cookie or local storage typically, whether it's a session token or a JWT, queries

both to store files, which the browser also has file storage. We did a whole episode on that, but you can also just store, like, blobs of data in IndexedDB. Yeah. And the cool thing about it is it has, like, React hooks and and Svelte hooks so that as long as you save it to IndexedDB,

like we mentioned earlier in this episode with the, PG lite has definitely been one of the bigger,

Let's talk about where you can store your data. Alright. So

not always the case. Right? Sometimes you accidentally set undefined and then try to JSON parse undefined, and you're in trouble. I'm almost always saving things in cookies over local storage, that specific user.

k kilobytes,

IndexedDB, as an access token, those types of things. These are stored locally on your client.

that, people are exploring in this area right now. As we start to get, like, more down this rabbit hole of using local data more frequently in applications, so much. I'm I'm talking with the XeroSync folks all the time, and I'll use the word auth authentication, and they'll be like, you mean authorization. Right? I'm like, yes. I do. Gosh darn it again.

Authentication

for all of the storage mechanisms.

Kyle Simpson,

So you could say, like, user and user 123.

IndexedDB?

and I, like, refreshed the page before I had, like, saved it.

that you would expect your application into local storage or into your IndexedDB or local storage or literally anything.

Before I've saved it, right, before it's in in their database, if they even save that in their database, then it's it's gone.

to, IndexedDB you know, you have to do a little bit of setup, but it's more like working with things that your typical,

storing locally or something so that way specifically, a platform like XeroSync does this really well. It stores your information locally, and it handles that sync process.

the thing about that is if you refresh the page, they're gone. So you can stick those files and stick all the data related to that in the browser in in one of the places that we're gonna be talking about. In that way, when you come back and refresh the page or,

So it's nice to just persist it locally in the browser The only thing that that gets me sometimes is I'll

Or, TypeScript Playground is another good example.

before you even have decided to send that to to a server.

or even, like I was on JS bench the other day writing some benchmark tests for JavaScript,

You know what? The best part is too. You know, Courtney also uses Habit Path. And, just like most things, when other people use your stuff, it's really nice for them to be like, this is broken. And then I'll go check the Sentry, see what's going on, fix the problems. Head on over to sentry.i0forward/ Yeah.

storing data locally. There's lots of ways you can store data

user preferences

That's why whole bag. Yeah. Yeah. I don't keep them locally in my office usually because then I'll need to access them all the time.

who has what issues at any given point in my application, what's up, my man? How's it going? Hey. Let's talk about, And your users will delight be delighted for it. I'll tell you that. Another Node is auth tokens. You know, we don't often think about this, but anytime you log in pnpm anything, it's either storing it in a,

Welcome to Syntax. On this Monday Sanity treat, we're gonna be talking about local data. Locally, right now, Wes, I'm trying to get a snack in. I don't know if you've ever had these rice mochi balls from, Trader Joe's, but holy cow, they're my favorite thing If you are doing job application things Wes they ask you to fill out every single one of your experiences or any of that stuff, you gotta have that data, the user isn't losing that. That drives me absolutely crazy as either. Yeah.

and I know what's happening beyond the happy path. I it works for me. All my data's local. It works. Nothing but sad paths in the in your century. Just a sad place to be. But It takes great restraint for me not to eat them.

But if, you need to access your data locally, we're gonna be talking about that in this episode. But what happens if you need to access your data locally and that data's not there and, you're not checking to see if it's there correctly? Well, you're gonna get a bug in your site. And oftentimes, you know, with this stuff, especially if we're talking about syncing engines, making sure the data is where it's supposed to be, Yeah. We can often get in these situations where we think we're in the happy path all the time. And, oftentimes, we might not even realize what that happy path is. I experience that all the time on my HabitPath app because I use local data and data syncing. And, man, it is hard to solve those bugs, but not with Sentry because Sentry lets me know

in the browser, for but then it will also synchronize, and it can also persist

in the whole world.

Sentry saves my bacon all the time. Wes, there's plenty of application where you just save things locally, and that's it.

locally from the server

If it is like a to do app, you're physically saving all the mutations and doing everything and gave a really great explanation about how tiny base came to be. So that's well worth your time to listen to. But, yeah, tiny base is an option here.

So that way, you can access it faster. Because if the data's already on the client, you don't need to go to the server to get it. So it's a way of accessing faster data as well. Sometimes it can it can even just be a privacy thing. Yes. So I built a

in the local data. And in that same regard, you might be saving

background removal tool.

And the thing with people's photos is they don't want to upload them to some random sketchy service that they're they're gonna have. Right? So this background removal tool

or even choosing to sync it to a server somewhere. And that's just, like, straight up usage of your application.