Supper Club × How Descript Built A Next Gen Video Editor In The Browser With Andrew Lisowski

each panel itself has its own state machine.

And re really, the thing that drove x state home for me,

So, for example, with these extended panels, you can,

was, when I was interviewing David, he said,

in your React code, you can just be firing events whenever. So, like, when the mouse goes down, just Fire the mouse down. It doesn't really matter what state you're in. And then the state machine like, parts of the video into local storage, and we're able to, and it just makes it really nice to work with.

it simplifies your code immensely because, like, your react code just becomes, okay. When these things happen, fire off some events, and then the the machine can go, oh, I'm in this state. I know what to do with this event. And,

there there can be multiple of them on the screen, and then it was additionally introduced that you can dock them on certain parts of the screen. So now I have to have a thing that controls multiple panels at once and knows how to position them and update them. So what I ended up doing is, for for overdub, it's running in the cloud. And then our newest feature, which is kind of a video AI feature, we just came out with it. It's called eye contact.

you can have you don't have to load up the whole file. It just starts streaming down what you need, and then it uses web codex to render that into our canvas. That's so cool, man. Web tech. It's, like, always astounding how much you could do with it because I you know, just even recently, I was getting into, like,

And then when docking is enabled, So they have, like, a cool one called, like, presence, which helps with animations and won't unmount a component

consistent But I really want AI to enhance me and not to replace me. So, like, publicize that one. But, like, for for anybody listening, build your own apps to discover things like this Yeah. I did the same thing with music. Like, I have 2 music apps that I am literally the only user of, and they keep me up to date on music just because I built them. Yeah. That's the the coolest thing. I I definitely second what you know, the the, I follow a bunch of interesting people on GitHub, and There you would not believe just how many interesting projects pop up Mhmm. In that feed because somebody interesting has starred something somewhere along the way. And,

and that was a whole lot of code. You have there's different modes you're in. You have to announce When something changes, you have to announce

absolutely wild where I have to keep reminding myself that I can use them, where in the syntax side, I was Working on, you know, doing something with media queries, and I'm like, this would be so much better as a container query. Why am I like, why is my brain not there yet? I've been asking for this my entire career.

Yeah. In fact, we we just actually dove into that quite a bit on the new syntax site using, somewhere in the back end? So,

defined a drag and drop, who made this library and is very, very knowledgeable about the space to improve this. So, and we've done that in the open also. So all the improvements that we've paid for this guy to do to to this area of our app, anybody else could actually install this package and start building similar stuff. Wow. It does use FFmpeg, building consumer facing apps. Like, No matter We're moving to turbo repo, ever so slowly.

there's anchoring,

You talked your libraries, but also, like, build tools as well. I'm very curious about that. Yeah. So, it's it's all in TypeScript. The the front end is

Yeah. It's a it's a lot to handle. Like, as I said, I did the tree view, and the just implementing the tree view spec, it was A lot. And then it's like, oh, well, now you need drag and drop, and nobody's actually

ARIA Specs. So you I I literally went and looked at every single open source tree view, and I was like, how do they handle it? And real there was no no cohesion. Everybody did it differently. So I tried to go with, like, the best option and then provide, like, some screen reader text to be like, hey. You're on our tree view. Here's how you might use it. Yeah. I I did. I I was like, okay. I'm gonna go build a drag and drop from scratch. And I built it in, I don't know, probably 50 lines of code. It's not too difficult with the drag and drop API. Then I was like, okay. Now I'm gonna make it keyboardable, and Stitches,

you know, top layer API with Popover API, you have,

both writing, TypeScript, JavaScript?

which is like a whole thing because popover is implemented

effort right now. That is a it's a community group that's working on every Offline

in like, they're kind of required because you can't do the whole position relative business with a Top layer. I mean, it it's it's a wild world, and there's a lot of really cool stuff coming. But, yeah, right now, it is still a very rocky road for that. So much cool stuff coming.

You know, design documents in Figma or something. Yeah. We we keep those,

It used to be like a single repo with just a load of code in it. Now, over the past 2 years, one of my jobs was organizing and, like, Making it into a more functional monorepo so we could use a lot more of the tools and, like, separate our concerns.

It's a place where you you would hope the platform could have, like, brought some more primitives to help us out, and I I think there is an effort Going on right now for that to happen, like, I know there's been a lot of work around, like, custom styled selects, which are, like, the bane of everybody's existence. Yeah.

Like, why why ARIA, and, like, literally, that it has, like, But then some of my components just got so big that it was, like, hard hard to even reason about all the different states.

like, you would need to build your own components that I don't necessarily have documented. And You pretty much have to go with a library like this if you're building a complex app because I imagine doing all the accessibility stuff yourself would be

so much over the past year. Like, the, the evergreen browser effort by, like, all of them to, like, make sure that CSS is up to date. Like, We got con container queries. That's like Yeah.

Documenting all of these UI libraries? Because you're right. We should have it in this should just be part of the browser. Right? We should have the ability to have a drop down menu that's accessible, and we I'll have to,

What a so you guys also have, like, a full marketing site. Mhmm. Is the marketing site

cohesive in a way through, to the app or either of the apps in terms of design language, or do you just keep those is that it is truly

My whole career, I've I've been hearing no container queries ever. Too too bad for performance, and now we have them. It's It's crazy. It's it's dump it out all out for them, Send it to them, and they were able to send us back a video.

in Most browsers in a way that's easy to polyfill, but anchoring is not,

connected at all man, that's that's really cool. They call them AI balls. I would like that for every photo I take because I found out we I just went on vacation, and I just found out that when I take a selfie,

right, accessible,

stuff gets so much social traffic. It's wild. Like, I I I don't wanna spend any time there myself,

so valuable just to, like, keep your your hand on the pulse of what's happening in your community.

but every now and then better. Yeah. It could be so because it's always just, like, freeCodeCamp, some very cool dudes. Yeah. Totally.

you know, like, at the top of it because, Like, those are the ones that people are starting,

LinkedIn

it'll put that all at the top and group it all together. Whereas on the home page, you might See, oh, 2 people started this 2 hours ago. 4 people started 10 hours ago. It just kind of, like, groups it all in a much easier to process way, and I I find a lot of stuff like that. Oh, that's good. Is that publicly available, or is that just, and then the the built in tools for debugging your broken tests are like Of all the companies that contacted me, 40 different keyboard interactions. So, like, that one was, like, a it was a hue huge effort to do that. And then another one, which I'm actually a little more proud of because I didn't just follow a spec, was our extended panel. So, like, it's a pop over that you can reposition on the screen and then dock in different places.

Yeah.

Projects. Yeah. I I I feel like you could build, like, a a nice social network on top of GitHub that, like, All programmers would just flock to.

But the amount of people that do discover things on LinkedIn is kind of shocking. Yeah. It's a much different culture than, the social networks I'm I'm used to.

So if you don't already do this, I would say go follow all the people that you think are interesting. That'll make interesting things pop up on your feed. I built my own little app on top of the GitHub API that organizes all of the GitHub home information into a way that I can see, like, What's trending? So, like, it'll it's just a list where, like, if 10 people start a thing,

That could be our our Twitter replacement, but I don't think anybody wants another LinkedIn. It's it's funny that you said mentioned LinkedIn because

I definitely

and, they could do a much better

The home page kinda sucks, but the Are you hipster smoothie? Is that you? I am. Yeah. That is that is me across all platforms if you wanna find me. Gotcha. Man, that's we had,

I I love that, especially if you're, you know, the type of person who's inclined to look at code to learn things. Yeah. The GitHub trending is is okay,

it was hard. Like, at Descript, we use Draft JS, and we We jump through a lot of hoops

And your second one? And my second one is, So it's It's really just like a pretty standard React tech stack. Like, we use Jest.

look like it's a part of the script. Whereas in Lexical, it's all just built straight in. So and it's really easy to, like, extend it and, like, add your own custom nodes. So if you're building your own, like, text Editor thing and you're looking for a framework to do that with, I would highly recommend Lexical because, like, over a weekend, I was able to build a Collaborative script editor, and it was, like, dead simple. Wow. Sweet.

shad c n /uiorui.shadcn.com.

became evident. Like, it wasn't the most accessible thing If you wanted to add, like, more UI

This one is it's a,

text editor framework for re It's made adding features really easy because it's like, oh, we just wanna extend the state machine in this part. Like, 1 one I just Added was to our numeric field, which is also built on a state machine.

not as good. Like, Draft JS itself had to pick up on a lot of shortcomings that Content Edible had. And as they built it, they reported all of their findings to the various browser teams. So over the years, a lot of the the Draft JS code base kinda just

e things into the document,

amount of knowledge in this area. And what you can view Lexical as is his 2nd attempt at making a content edible framework. Okay. And it's the successor to Draft JS. Yeah. I was gonna ask how this fit in there. Yeah. Yeah. So when Draft JS was built, Content editable in the browser was We can actually with AI and, like, Convincingly with AI, is there really all that much demand for putting

well, not just for React. It's for everything.

So he has an immense

that kinda did what we wanted to do called lib a v j s.

in the web. There's some awesome versions of it, but,

It's an Emscriptum packages that make up Radix components Mhmm. To make my own components.

process the video? Like, I know you just not being able to set the bit rate. I don't know if I don't know if you're I don't wanna put you on the spot there, but, Okay. We'll make sure that's linked up. Awesome. Thank you so much for coming on. Really appreciate all your time.

And, what what it is is it hurts for me to look at. But with Playwright, it's, like, super simple.

we found a library that had, like, 30 stars,

based You could use Tailwind if you want. You could use CSS modules if you want. You could use style components.

Let's talk about,

It's doing a a lot of that stuff, So, you can use this to just, like, jump start a project. So you're going and you're like, oh, I need to select. You just run their CLI. It adds a select into your your code base. You can start using it. It looks beautiful. It functions how you want, And then you realize, oh, I I actually meant a drop down menu. Then you add that, and you can just, like, iteratively add all the different UI pieces to your code base and just Yeah. Get your the tool you're building off the ground, like, instantly, basically.

Like, how do you actually

thingy that, like, has a bunch of the, like, FFmpeg put them at different places on the screen.

but There is no FFmpeg,

that's I now have better upload, so it's not so much of an issue.

we you showed it in here. Alright. Yeah. The the 2 big ones in my mind are this and React ARIA. React ARIA is,

web stuff?

lower level Interviewed somebody from the company modules. No longer around anymore, but they produced, Radix

UI library, so it comes unstyled. So you can style it However you want.

from my view, when creating a design system,

is, I I've been described as coworkers as a machine.

Add to it because, I think you can try it in some use cases? Yeah. I'm also excited for that project. Like, What Tobias did with Webpack is is amazing.

it's like Tailwind plus CSS modules in the end, So all the letters are really thin, 80 characters tends to take up all that much, and you can fit a lot of columns of code on your screen at once.

a class and then apply it? CSS module style? Import

style utility library that's mostly just for layout, but it's

star styles,

we have, like, a tailwind Became not needed. And then over the years of having it exist, like, the shortcomings, flex, flex call. All those things that make it really easy to lay out your components, that's handled by, like, Our tailwind style library.

Like, around halfway through the project, I realized that we needed a tailwind style thing to, like, Sure, it can run on your machine, but can it run a 1000 times on CI without ever breaking?

What I've devised for our repo through communication.

to do our styling. So you're doing, like, the whole import, you know, a browser component to this, the video

one of, like, the the key rules when When creating components is a component doesn't care about its width, and it doesn't care about its margin. So anything that has to do with layout, a component should not care about. And that makes it really easy to go. Okay. I'm using this component, and now I'm the parent, and I'm controlling those things.

And so I, like, reported it to them. They're like, oh, this is top priority because it's data loss. Right?

I use the Apple, whole, like, trackpad.

rather than just like, oh, you can generate a video that's not really even usable. So, like Mhmm. For example, like, we tried to put DALL E 2 in the product, but, like, which, like,

over it.

So for my font, I use Iosevka, can only handle,

for

nonstop gouging you, over the course of years to release absolutely nothing. So, How does Descript handle, like,

topiary keyboard, Crazy hard to, like, understand

0, like progress in that app, and it's just, like,

So very very stoked about that. About the the polar opposite of Descript because,

I really hope that it works for me. Are you? Okay. Good. Because, like, I literally told me at Reactathon, and I went straight. Yeah. I I went straight to the the engineering team. They're like, yep. This is a problem. We've been working on it.

supper club questions, which is the things that we tend to ask Everybody, just,

Like, it's

It's amazing to have a person that, like, act like, is basically the number one user of the product and can give you, like, actual insights into, like, How how we should be

A text editor's theme and font.

The the main difference about it, it looks really ugly at first, but it optimizes That horizontal space is so much more valuable than the vertical space for me. Yep. And then for my editor theme, I use Versus code, and I wrote my own Theme.

The founder of Descript is the,

which feels real nice, but, I'm I'm not really all that tied to it. What about

former founder of Groupon.

progressing and, like, what the true north star is rather than just like, oh, this guy runs a product. It's like, no. This guy wants this product To be, like, the next thing for video editing. Sweet. I'm glad to hear that that's being worked on because I have I've been whining about ScreenFlow for probably 8 years now.

am I looking forward especially, like, all these AI editing features and whatnot. Like, these are things I need,

I'm working on a MacBook Pro, And, editor for,

in it, there's

No. So, like, we haven't really been pursuing that angle as much because it just doesn't make sense for our users. Cool. Well, I am pretty stoked about Descript. I've been I've been coming back to it they basically communicate up to a 2nd parent state machine that handles all of the docking and the re resizing and repositioning horizontal space.

couple months to see I have a a couple issues still outstanding with the export quality,

Mhmm. And freaking guy Really? Just I I was like we're tweeting about it, and he just DM's me. He's like, let's jump on a call. I was like, holy like, this guy cares a lot about making this a kickass product. You know? Like, it's Oh, yeah. He just jumped on a call immediately with me, and I explained to him the actual issue. So a really cool guy. Yeah. One one of the biggest surprises when I got to Descript coming from a 10,000 person company was how much I interact with the CEO

Yeah. I'm over it too.

Maybe they're longer than that. Yeah. They're making it into just like a glorified PowerPoint, you know, and it's

AI companies is like we're trying to make the AI features make sense for you which I then tamed by using x state and,

and how much he loves the product.

0 innovation.

I'm like it's it's still my editing thing. It's still the best thing for me, but, man,

that I could not get back, In there to be able to process and export the video, or is it literally using all web

You guys took 3 3 weeks or 3 months to fix this thing, so

we needed disruptors in this space, for sure. I found a a bug in ScreenFlow where

It it's it's been a a pleasure. Let's get into some

and like, load in a bunch of JavaScript just to make that that type work. But to give brow browser vendors a little credit, they have added

I, like,

like,

Still images that don't really look all that real into your videos? I don't know, but I'm Pretty sure it's a lot. It's either local storage or IndexedDB.

Well, thank you, Andrew.

which

Oh, good. Good. Because,

There is

or any of those, like, neat little tools they have for you?

You weren't used to not be able to drag the border to, like, drag the field while You were editing because the state machine just didn't allow that, so I just added a new event, mouse down on border, and the only state that reacts To that is while you're editing. So, like, I can be sure that I didn't really break anything else and easily add on new features. Nice. Oh, that that's a really good, like, explanation of of why you would want to use a because, like, I've been in so many apps before where, like, you hover over something, And then, like, you hover off a little bit weird, and then you open something else, and they're, like, overlapped. And you're like, I can't get out of this. Like, I can't click the x because This piece of UI is is in the way or or whatever. But, like, if you literally

Did did you end up using the, visualizer what makes Descript a really interesting app technically, we're gonna be talking about some of the redesign and design system effort

eliminates an enormous category of bugs.

Yeah. Radix. I would say that. It's rad. Yeah.

So you obviously have

does this look right? Are there extra things that this thing should be doing? And it's a it's an interesting way to collaborate. I'm trying to understand or Maybe just ask some more questions about the the radix. You

So but I'm just looking at this libavjs.

it's a it's a component library. Right? Like, it comes with a whole bunch of different a pop over, a progress, Scroll area, select separator, slider, switch tabs, toast, toggle. Right? And then, like, send that. If there's there's something in there that you don't have, then Then you have to go ahead and extend it and make your own version of that.

Wow.

For them, or how how do you write your CSS?

I haven't. They're really cool, though. It like, the The the two way updates are are a lot of fun to watch. Like, you can write some code. It updates your graph. You can you can update your graph. It updates your code. It's a a super cool feedback loop.

I think I think, in my case, those

And that state does not exist. There's no possible chance for it to open up. That's that's really well put. Yeah. So if you if you have, like, a bunch of state Variables like 4 different booleans, and then you're like, oh, man. I have to, like, set this one to that when this happens and vice versa.

radix? Radix? How do you say it?

CSS in each of these components

that tool is Best for sharing with non developers because Mhmm. Since it's just a a graph and a little bit of understanding can get you to understanding the graph, you can easily Share that with a PM or a product designer and be like, And with the

have to define every single state

are you writing

is a really good I d a really good place to start building videos. So with Descript and all the stuff we have, you can go in. You can start writing your video. You can chop it up into scenes. You can create the whole video without ever saying a word And then,

was is I just, like, looped over it and dumped the video tag into a Canvas element. But, like, with web codecs, it will, like, literally allow you to

you don't have to do that dumping. Yeah. I I believe that,

individual frames of the video, right, without having to dump it into a campus. It is a it is still a canvas Chunk, I believe, but, on on Twitter,

What what I did then Alright. Let's move into sick picks and, shameless plugs. Did you bring a sick pick for us today? I did. Yeah. So, the first one is lexical.dev.

and then have with our AI voices just to add the same type of plug in system because I feel like it just allows for such creativity.

A full video that you could publish. But if you like the video, you can just pop in with our recorder and start, like, recording over the parts where you think your voice would fit best. Nice. That's really cool. And, like, the the web codec stuff, I've dipped into a little bit here and there just when when playing with video. And, like, in the past, when I've Descript from what it used to look like, to what it currently looks like. And a lot of like, I was literally just given pages and pages of between all of them. And without x date, I think it would just been, like,

Be able to scrub through it and then take a, like, a screenshot of the one that I wanted and download that as a thing. And, Ace.

Worked with video in the browser.

That's that's pretty high. Yeah. Firefox allows 2 gigs, and Safari allows 1 gig.

80%. So if you've got a 100 gigs free, Chrome is allowed to have 80% of that, and every origin, so every domain name is allowed 75%

And then download it to the user's IndexedDB and then be able to, like, alright. I'm going on a plane. I'm gonna watch Wes's course. And then when you connect, it will It will sync up, and it looks like I'm probably able to do that. I think I'm I'm I'm moving away from Vimeo anyways, you're using web tech, but, like, is there any, like, WASM based stuff So it's nice to be able to like, it will learn your voice. Right? Do you know how do you work on any of that stuff? How does that work? I don't work work on it, but I I can tell you about it. So, like,

an app. You know? Yep. And only allows you to record into Chrome. Makes sense now. Yeah. That does make a lot of sense now. Wow. Wes, you just unlocked,

downloads for my video courses, Radix. Radix? Radix.

Float an element float an image and wrap the text around it. And it's another thing to build, like, a, like, a proper I like, I made, like, a little thing for my TikToks that I would import a video, and then I would

complex UI with lots of interaction and drag and drop and all that type of stuff. So you said you were involved in or or the lead in building a lot of this UI? Yeah.

and it's

So I'll probably be able to do that at the same time. Yeah. You know, is now I mean, that's probably what Netflix does or they offer, say, for offline watching.

I really enjoyed the, like, subcomponent pattern where you have, like, a root, a trigger, a content, a set of components that all go to go together under 1 object.

I I myself am a a React developer. And throughout all the design systems I've created, designs, and I took that and translated it into a design that worked really well together.

finally, because they they jacked their rates on me.

some paragraphs and an h two tag and the, like, the local first ones. But,

when you do that, you take yourself out of all of The niceties of, like, tracking how how far you are and if you want to bookmark something, like, all of the features of being able to track where you are in the video It's just here's a bunch of MP 4 files, and you can you can watch them on VLC. So what I would like to do is to be able to just say, like, make this available offline, Different parts of the video, it's it's astounding how much knowledge he has about this area. That just goes to show you if you, gain enough expertise in anything, you can be Desirable for for big time work. Kidding. It's it's true. Like, it's it's tricky stuff. Like, we so we use Riverside. I'm sure you do as well, and and we hit little half the people are like, Use a for loop. That's impossible to read, and then the other half are like, just learn reduce. You know? It's it's just, like, weird area where you when you learn something obtuse,

fails, you can see the part file that fails. We don't we don't try to hide it from you in any way. Cool. Yeah. I I'm just looking it up.

Yeah. Yeah. I wonder. Did they do that in the browser? Alright. Let's we'll dive into it. That'll be a whole show. Yeah. But Let let's talk about, the actual UI of the Descript app because

Chrome and Chromium based browsers allow 80% of the disk space to be used, And from that, 75% can be used by each origin.

it's one thing to design how they built, like like like heavy lifting tech. Like, not like like Notion is is pretty heavy, but, like, So 80%.

Over the course of 9 months, I I took,

Riverside has a And like you said, data losses, it can't happen. And it took them, like, 3 months to fix it, and I was, like, so frustrated with that. I was, like, literally, I lost a video

I was I was inspired by a lot of the tech that I found on my podcast.

of that.

because I've been thinking about it a lot is, like, I offer Looking at it, right, I need to cock my eyes up a little bit.

to to make sure that I didn't break all these different keyboard interactions. And by the time I was done testing the tree view, we had 50 different tests. And those Those tests have caught so many bugs.

for y'all? Personally, it's the design system test because, like, I'll go back to my tree view example. That was the that was the component where I was like, okay. I need to add Some tests

In this e to e, like,

it also has a trace in the failed test report. So you go to the failed test, you click on the trace, And then you can see a whole timeline of the the test that ran. You can hover over it, see how it changed. You can see all the network calls. You can see the console, And then you can even inspect the page at various points in time. So it's like almost

world that you found to be the most

a time travel debugger. Not quite it's not quite replay, Those are, like, the the easy ones and the ones that came with Radix, but, Radix doesn't have everything. So, like, as a part of that, I implemented

And I just added a little helper that, like, on every test, you can destructure storybook. You can tell storybook where to go, Then you get a page object that you can interact with. So it's just like the

You get an HTML report with all the failed tests. It shows you the steps it was taking, what it waited on. It'll put a video in there. And then the thing that blows everybody's mind when I show them too is You can resize them and make them bigger and smaller.

You know, 1 out of every 8 runs I'm gonna do, something odd is going to fail.

and Cypress does a lot of these things too, like being able to,

not reliable in the sense that,

the time to test is very, very quick. Did you have, like, any specific types of tests And when you have very when you have very complex things,

important we pick a few different ones here. But first and foremost, what kind of computer mouse and keyboard are you working in?

that's good to hear that you you're

the system in Cypress failing. So,

dive into,

You know, the dom at any given point, like a a failed test. But the one thing that I found with Cypress

on this to be really reliable. We have added this to our new repo. I haven't ridden any tests with it just yet, but,

or the code failing and not just

that it was my test failing

definitely in my future here. So glad to hear you had such a great with it. Yeah. And they make it really easy to extend. Like, we, the 1st place we started using it for was testing our storybook,

Cool. Yeah. I I have quite a bit of experience with Cypress,

they started breaking. And I was like, oh, man. My test broke immediately. And they actually were finding issues because, like, 1 out of every 10 times this weird little time out thing happened, and it was something we would have never caught on our own. But since we were running tests in CI nightly, we found it really quickly.

And I never felt totally confident utterly important, Especially with design systems. I was gonna say you mentioned monorepo. Do you what what tech did you use for your monorepo?

they're they're so good. It's like I can't I can't even explain, like, how good they are. Like, a a test fails in CI.

it does a lot of waiting,

Like, Descript was was early on. Like, Descript was before all of the chat g p t and all this stuff, like, sort of blew up.

Oh, it's it's all running in the cloud. So,

to holy crap,

Are Those models running in the cloud, or is it or is it running on your computer?

in, like, 6 months. And,

Move suddenly, but it's a really hard problem because there's like Yeah. There could be so much going on in the picture. Like, say this plant back here is, like, swaying.

commercials where, like, The Rock is Eating something Yeah. Is terrifying. It's it's funny it's funny how quickly we went from LOLAI Yep.

is the Syntax supper club.

This

so buckle up and grab that old handle because this was going to get wide eyed as old.

Today, we have a really Awesome guest from Descript,

Welcome to Syntax.

I sure hope you're hungry.

Wash those hands, pull up a chair, and secure that feed bag, because it's time to listen to Scott Tolinski and Wes Bos attempt to use human language to converse with and pick the brains of other developers. I thought there was gonna be food,

Hoo. I'm starving.

I So,

So Is it available? Yeah. Sure. Sure. I've been a programmer for, like, 10, 12 years now. I've always had a passion for

So

so I made my own.

JavaScript on microcontrollers,

and I have to have large text so that you can see it, right, if you're watching it on your phone or you're you're half screen watching it.

GitHub dark high contrast. So I I I saw the GitHub dark theme that they came out with a few years ago, and I was on, like, the built in high contrast theme. And it was just like, I couldn't See anything, so I just made everything as contrasty as possible. I want, like, the code to jump out at me, so fire and forget events.

GitHub is a a great tool, but, like, I don't think many people use it like this.

now we've got his I've I've logged in to him a few times. Oh, really? Yeah. He he's a cool guy. Really cool guy. That's what I love about conferences too. It's just, like, you just go like, I think we went out for lunch, and And just like me, all kinds of really neat people. There was, like, 6 people out there, and I was like, these are all Well, that's good.

So

Nick Hare, when we talked about,

What do you do to stay up to date on web stuff?

to talk about,

Yeah. I have a podcast.

It's called Dev Tools FM. We talk to people who make developer tools, ask them why they built them, things they learned along the way, And then usually, like, some future facing questions and WebDriver, Well, running a podcast where I look at all the new things happening really helps with that.

Welcome to Syntax. How's it going, my man? It's going pretty good. I'm excited to be here. Yeah.

about the field they're in. Right on. Right on. So why don't you give us a a quick rundown of

I actually started it when I started Descript.

a better rundown of the actual product.

somebody on the podcast

the big 2 are Figma and Descript, which is graphics editor and video editor, and they're all built with with web tech. And How that's even possible blows my mind, so I'm pretty excited to have you on. So thanks a lot. Yeah. When I when I joined Descript,

What Descript is. I know I gave a sort of a it's a freaking web editor, but, like, you can give us a better

Figma was one of the things that I looked at, and I was like, okay. That's amazing.

who you are, what you do, as well as

I I think we should mention just right off the bat, for anybody listening, I know people sorta dip their toes in. Descript is a freaking video editor built with browser tech.

Blows my mind. I'm really excited to have this interview. I've been trying to get

Is Descript gonna be that for video? And over the past 2 years, we've taken a lot of steps to get to that vision. Nice. Oh, that's awesome. And and we should also say, like, you have a podcast as well. You wanna shout that out right off the bat? Yeah.

like you just get the code and then like you're not npm installing. You're not it's not a high order component. It's literally just some code, and you edit that thing, and that's it. You don't have to update it or anything. That's your code now, you know, but just like a base. That's a template back from back in the day. My code now. Yeah. The only thing it adds to your project is, like, some dependencies, like, on Radix. But past that, you can choose to update with them, or you can just Take that as a starting place and, like, build your own your own components off of it. Beautiful.

Head on over to syntax.fm

Yeah. Thanks for having me. Yeah. You're welcome. Yeah. Awesome. Alright.

And don't forget to subscribe in your podcast player

That's and it just you just download,

drag, drop, put it in your code base? You just run npx shad c n add component name, and then adds it straight to your your code base. Scaffold Very simple. Yeah. Been been hearing about that a lot lately, which is just

Alright. Shameless plugs. My shameless is an easy one. Go to go listen to dev tools f m. We we try to put out, a lot of content. Like, as as Scott asked me, how do I keep up to date? I keep up to date by listening to experts in the field talk about things they're passionate about. So,

So what this is is,

into a directory, But then, to make it make sense within our product, and I think that's really kinda what sets Descript apart from other, like, And then as more and more requirements were added to that, it just became, like, this beast of a component,

for a full archive of all of our shows.

already prestyled with Tailwind.

Ace.

if you wanna hear some good conversations with some cool creators, come on by.

it's a library, but it really isn't. It's more of a generator framework, and what it generates is design system components.

timeline based editing like you might be familiar with in,

at first glance, you might just think it's like a transcription A year and a half ago, we

around 2 years ago, I wanted to make a change and, I am a a very like, there's a lot of people have a lot of things to say about Webpack, but I still think of the bundlers Dominic built, both React DOM and Draft JS.

they were doing something very interesting, and they had that, marketing video by Sandwich, which Any marketing video by sandwich

editing a timeline to edit your video, You transcribe everything up front, and then you edit the script. And then we try to lean really hard into And for a few years, I basically just worked completely in the open.

And we have a whole host of cool AI things that we do. And now that we're moving to web, we have to do a lot of very interesting things to make that work in a browser because video is It's quite a big thing. It's like So that I've stuck with it. I've tried to switch to other ones, but every time I put another one in my editor, everything gets really wide. And I'm like, I can't See his bar. Sorry. I just go I go straight back. Yeah. Oh, I should try that because, like, that that's a problem with me as well because I do coding tutorials, and I have to have a Small screen, I tried to move to a mouse, and, like, I instantly started getting wrist and elbow problems. Mhmm. And then I moved back to a trackpad and everything was okay. I I don't know what the I like that too. Yeah. Yeah. I don't know if there's ergonomic science behind it, but I've never had problems with it. And then I use A real force some m model.

go away from more platform y things to, like, being able to get my hands in the product. And so I made the transition to Descript because,

So what Descript is

the script based interaction rather than the time line based interactions.

is, webpack is so flexible, and you can do so much with it. And, that's why I'm really excited about Projects like RS pack that are saying, like, okay, web pack,

whatever media that you have. But, what what we really are is we're trying to redefine the video editing experience to go from I was actually talking to someone on my podcast about the 2nd choice, which is, like, Run Docker containers on the cloud and do, like, heavy lifting in a Docker container somewhere and just stream that to the browser. That was an option, but the option that really got all of our engineers excited

which is a lot more natural. So instead of, Like, how do you make that look good too? So, like, that's something that we've been trying to solve, but it's like, video AI is is harder even though that it's like a booming space right now. Oh, yeah. Yeah. Yeah. Everybody's seen the AI

gets me.

the tools on your Mac or Premiere, things like that, to a script based editing,

What about, like, turbo pack? Have you dove into that at all?

that's when I eventually built my own automation

heavily on Lerna

I think Vite and, by extension, Rollup are much more user friendly,

Ours is, like, I'd say hodgepodge right now. Like, it was on yarn yarn workspaces.

Is is that out yet? The last time I looked, it wasn't quite out. Yeah. Yeah. I have no idea. I would imagine that is

not ready, but I think you can

and,

I haven't yet.

those tools. And, to make a UI but it's good enough to get you to the point where it's like, okay. We can actually see the issue that happened. And, like, literally, right as I implemented something you got? It it is publicly available, but I didn't set up the auth Right. So now when I go to it, my browser thinks I'm trying to hack myself. So Yeah. I might not I might not,

or My my own release automation tool, is the most powerful. Whether you wanna say it's the most user friendly is a much different argument.

When I was at Intuit, we were, like, heavy in the monorepos. We were, like, the 1st JavaScript team at Intuit to start doing monorepos.

And back then, we relied

Just pretty much the standard tools. What about sorry. Did you say what your build tool was? Oh, well, we use a A lot of Webpack.

But the control you have in Webpack is is still unparalleled.

it it is good in some respects. We just wanna make it really fast. Yeah.

on top of Learner called auto.

I don't look at the camera. I look at I look at, like, the button

So say if you're, like, recording a video and you're reading a script on your screen, your eyes are going back and forth reading the script.

Miedeross,

link up a video.

he has a Process video and audio and get the Almost impossible.

pushing to take the picture or I'm looking at the screen. So, like, my wife's eyes are always looking at the camera, and mine are always, like, looking slightly to the left. Like yeah. That's a a really fascinating tool. Yeah. The Those are the AI features that get me excited. Like, runway

make it look like your eyes are looking at the camera for most of the time. And, like, now every video that comes out for me or my coworkers, I'm just Staring at the eyes trying to sit figure out, are those AI eyes? Because they're they're staring straight at the camera right now.

Matt

tools Tools that allow me to, like, utilize it in ways like this are are super cool. So, like, we have some stuff coming out with GPT also. Like, everybody had the race to put GPT in their product. We've been taking a very slow calculated approach trying Yeah. So with, Radix, it it's a headless

and he posted a video this morning being like, I'm using the new Descript feature. And I watched the whole thing being like, what is the feature?

And and then at the end, he yeah. And at the end, he showed himself looking down and reading his laptop, and I was like, oh, damn. Wow. And that that really helps me because I have this issue not here, but I I have the issue where

AI, it's cool. Like, stable diffusion, it's cool.

So,

It's it's honestly amazing. Yeah. I will

If you misspoke, I do it all the time where I I'll say, and the boolean will be true, and then I edit the thing, and I realize, shoot. No. It's false. It's literally the exact opposite thing, or I misspeak or I I say something like, at home where I have the I have the camera a bit too high, and I'm not necessarily a bunch of podcasts,

So a lot some of those things are, like, our AI effects. Like, the events that the current state is for. So, say, if you're editing

You have a little bit more access to the system.

it's not blockchain.

my my favorite miss misspoke

But if you're moving to the web and you want it to be accessible to everything everybody, you have Start moving more and more of those things into the back end.

like, there's this, like, thing you could take called branch chain amino acids, And I called them blockchain amino acids, like, without That's very funny. Without noticing it at all. It's like,

Of all the companies doing AI things at the time, and I think even now, like, Descript focuses on making AI useful.

of syntax Was we were talking about supplements on one of the fitness shows.

there there's a couple things to it. Like, at the at the very beginning, Descript sort of came out where

That's That's Bitcoin.

I called it,

Andrew Lizowski. We're gonna be talking all about Descript. We're gonna be talking about,

Along with just in general, the cool stuff they have going on over at Descript. So, Andrew,

You're just like, well, I I know it now. You know?

anytime I post, like, a reduce you are in the editing state. If you hit the escape button, you come out of the escape button. You had to hit man, it's Like, shout out to people that work on on these libraries because that stuff is certainly tricky, and you literally have to have it. Like, legally, you have to have it in in your application. Right? Yep.

it's really just amazing how much that you can manipulate with it.

Yeah. Even if it's ugly or hard to read or or, you know, any of those things, once you know it, it it Suddenly makes a lot more sense. Even, like, for me, like, Redux

the The plug in system in Webpack, while a lot of people say is confusing, Like, if you look at people like, Zach Johnson of Module Federation, like, Module Federation is a thing because

Yeah. But I I do agree with all the critics. Like, configuring webpack and knowing how to configure it are not Not fun bits of knowledge I have, but I do have them. Yeah. Yeah. And if you've if you've gained them, I've had, And,

The package they use for that tappable, I've actually used in, like, 6 other projects

and you have them, it probably feels like a superpower at this point. Yeah. Yeah. I was Thinking about that earlier with,

with Vercel behind him trying to make, Like, Webpack 2 essentially a thing. Mhmm. I I I think it's it's destined for good things at least. I I like the power, like, Yeah. That's really not a yes most of the time. And what I found with Playwright is it really helps you get To, like, perfect CI because, Like,

This is a compilation

it's at one point, I'm like, come on, Riverside. But at the other side of it, I know that this

little weird things that pop up, and job at surfacing kind of these niche little

And I was, like, I didn't know you could put 6 gigs In local storage, I was curious where they were putting that. You do I know this is not your thing, but do you know, like, what the limit is?

Oh, I was always wondering about that because, like, right now, I'm I'm uploading mine, and I'm only 34% uploaded because I'm on a a slow upload. And

Compilation of libraries associated with handling audio and video in FFmpeg.

This kind of stuff is probably just riddled with edge case,

Libbey so he he figured out just parts of FFmpeg? I think so. The the the man is very smart, and this is the probably the most technical part of our web video stack. And there there's a reason we hired him. I don't for sure. I've been following the Slack threads and just, like, the the minutia that they get into about, like, libraries, and then we use that to Do a lot of that stuff, and we actually contracted the guy who, Like, one of the big things with automated tests is flakiness.

it will sometimes be, like, Like, 6 gigs of video that it needs to to upload, and it takes a couple hours if I'm on a slow connection.

Bumps and issues all the time, and it and

You gotta find these really smart people to figure it out. Yeah.

We've been using Riverside for our podcast too, and We've only had a few hiccups, but 1 hiccup, testing the core parts of your system, Yeah. X8 solves that problem. It makes it makes those impossible states impossible, one of the things that, like, got me really excited about Descript when I joined the company was

we we basically lost our recording. And,

but in the end, they're just drawing vectors in the browser. Descript is trying to And do whole videos from all the way up to 4 k, so that that takes a lot of work. And I guess that that's a a good lead into a question I had even getting into some of the stuff about video. Given that there is, So, we we use Canvas for that.

impressive, but, like,

with browser APIs like get user media, or is that being done,

Figma's

Webcam interaction stuff, is that being done

really easily. That's something I would absolutely love because I feel like I,

pause sometimes, you know, I don't give proper

So then I do end up having situations where I cut really harshly into

There's a lot of tools that are like, we're a random number generator. Come give us your stuff. We'll generate stuff for you, but it doesn't really, like, get The core of creativity are putting the user in the seat of power, and a lot of Descript's AI features, I flipped that. And you're a creator, and you have AI tools that help you. So, like, in your, like, audio, component Dominic from the React team.

Especially if I goof up, I wanna take that take immediately again.

past that. So, like, the 2nd really cool feature built on overdub is so say you're editing some audio and you look at the wave forms, and there's, like, this, like, sharp cliff. You can hear it, and it's kind of like a pop.

Wait a day for it to train, but we recently came out with updates that make it, like, seamless and a lot faster, Information on your home page is, like, really, all it is is a generator that will put out, Radix components and you fire a drag event, the drag event does nothing because the person's editing. So you you make impossible states Impossible.

Into my text. Interesting. Wow. That's cool. And then, like, the the obvious next question after that is but there's not just only audio, Andrew.

like, that goes all the way from transcription, which is AI based, to the voices that you were talking about. The voice clones have actually gotten a lot better recently.

We have smooth edits now where we can use your AI voice, And you can say, oh, this cut right here, use my AI voice to make it seamlessly transition, and you can get rid of all of those, like, Descript was the most exciting because, Like, in the electron app, we are running, like, Multiple servers to do things.

key value text. Right? Mhmm. Yeah. And then IndexedDB allows you to go a little bit deeper

I can't I can't remember off the top of my head. We at Descript, we actually use IndexedDB quite a lot too, and a bunch of A bunch of third party scripts do too, so Yeah. I I think it's called there's 2 ways you could go to add those things. You could add them all as props, which is kind of heavy.

you could

in the thing. I remember looking deep into, like, what the limit of I I think it was IndexedDB

was, and there was like, every device was a little bit different, and it was also depending on how much

can you have to rely on the browser, but you also can't rely on it too much. So we have to do some things in the back end. I have I have a question about, like, a a practical, how everything connects.

Free space was available on the person's computer as well. Yeah. So so that's another thing you have to, like, contend with when making a video editor for web is, like, It's running in a browser, so you only have so much storage space. Like, our goal with, Descript for web is that it runs on a Chromebook. Like, Anybody can run it on anything. So, So we have that for everything. So we have popovers,

your mileage may vary there. Yeah. Interesting. I think I think it is IndexedDB because local storage is just, Yeah. That must be why, What about like, what does your stack look like for yeah. Mocha choco latte.

the tree view Speck from,

Was this the 1st project you dove into State machines with? Yeah. I hadn't really done state machines all that much before. I had looked at them and gone, oh, that looks confusing. I don't really need that in my code.

that's something that it's one of those, tools that I've always wanted to get into, but I've never really, Like, committed to it. You said over over after using it, you became a fan.

drop downs, context menus.

Became a big fan. Oh, cool. So you are you're using Xstate. And,

on every show for the past, like, 3 or 4 months. We did a whole show on what streams were, hipster brown,

Yeah. Yeah. That's that's tricky. I, like, I famously or not famously, but I've Remember, I during the pandemic, we came up to our cottage, and I had this, like, bootleg LTE Internet. And I tried switching to Figma so many times,

And I just could not do it because I was kept choking out the my upload, and and it wouldn't upload fast enough for me to be able to, like,

frame buffer stuff in Rust, grabbing individual for and it was just such a Giant pain. And at the end of the day, I ended up going with Webtech,

And, like, that was sort of a trade off of

efficiently work on it.

it was taking so long to do Anything. And granted you would have more control or whatever, but it's just always astounding the amount of work you can can get done in Descript. Wow. What a, what a testament to that. I feel like we've been talking about web streams

just because

and then they just keep they keep popping up. Like, we have Ryan Florence on, and they're streaming you can stream from the server to the client, and then then here you were on talking about it's It's all streams. Right? Like Yeah. If there's one thing to sorta get good at in JavaScript right now, it's probably understanding what streams are. Yeah. That that and another theme I've I've found is Local first, but they're those 2 ideas are really kind of at odds. Like, in in a perfect world, you really do want, like, a local first d script, but, also, like, do you want Hundreds of gigabytes of videos just sitting around in your browser's local storage?

situation here because I I've used, You know, I've been doing video for YouTube since, like, 2012, and I've used, like, every editor on the under the sun. And one of the things that always bugs me a lot is, like, ScreenFlow. Right? You if you record a little bit or even there's some others. You record a little bit, maybe perhaps something crashes or, you know, there's an issue in the recording,

And you end up losing that whole recording because what they do is they throw it into some weird container format that you can then never recover.

is that, like, a consideration or, like, a major concern? Oh, definitely.

That situation as it's recording, if something were to happen or interrupt, I'm bad at saying things. I think gonna call it. I would've said Radix, Wes. Radix? Radix.

What I found was the easiest way is just using class names and tailwind style stuff. So, like, we have, like,

And then, the components themselves, just respect that and don't don't set margin or width. Sounds kinda neat. Yeah. That that's really cool. I like that it's it's all headless as well. We did a whole show on a few of these different headless UIs, and this is not one that had popped up, so I'm glad that you,

I really think Playwright got it right. With Cypress, I used it, Like, once or twice, and it has a an interesting API that doesn't really meld with my brain.

And it's all it is is a way to automate controlling a browser,

Playwright is really like, it helps you in the fact that all the assertions that you do or, like, the actions you try to take, It doesn't just do them. It goes, okay. Can I do it? And then I'll have time outs for, like, trying for 30 seconds to do something. So that helps.

what's the other one? if you try to export a video with the same name, It prompts you to say, hey. Would you like to overwrite this file? And I go because I always export a TikTok version and a wide version for Twitter, and then it'll be, sometimes I'll forget to rename it, and I found a bug where it would just delete the file before it asked you.

and there's been attempt after attempt to make, a nice Automation tool for JavaScript, Yeah. So, in the in the Tron version, we have like, the core of it is basically FFmpeg.

API, and it's, like, almost English, but not quite. And it just which I found from a guy on Twitter.

One thing that I was instrumental in is I brought in Playwright, which Huge fan of Playwright. Such a cool tool. So well done.

Yeah. It's, like, a chaining Oh, wow. That's the real kicker.

and,

Cypress? Cypress. Yeah. So, yeah, Cypress and Playwright are the 2 big ones in, like, automated browser testing if, you've been around A while, there's also there is Selenium

The coolest thing about Playwright is, like, in with all the other automation things I've ever worked with, you have to manually wait for things.

And adding more more of that. Rundown of what that is in comparison to,

Like, they definitely handle all the cases more. Like, I think I could've probably built a tree view with them a lot faster, I can't wait to get rid of yeah. Open UI is the community, The engineering doesn't own that website at all. I think it's a Webflow website, probably. Okay. Yeah. Yeah. Keep it easy. Yeah. Yeah. Cool.

I u like, I use probably, like, a dozen or so of the undocumented One thing you can't be certain of is that a user has a GPU, and the GPU can do things. So a lot of the cool AI stuff that has to happen can't necessarily happen on a user's machine. So we've had to do a lot of work to put all that in the cloud and to make it reliable too. And The AI stuff is actually really cool because was on, like, a platform y team, and,

And Radix is really nice, and, like, I've learned a lot from the repo.

Like, I I encourage any developer. If they're using a library, go read what the library is doing, and that's gonna that's gonna help you improve a lot too.

but I chose to go with Radix.

until all of its animations have run. So there's, like, Lots of really nice little things in there that, Video is also a problem. So, like, when I was first editing my podcast with Descript, the unfortunately, the biggest comment that I got back from everybody watching it was like, Why are there so many cuts? Why are the heads moving around so much? And that's because I was, like, I was making the audio perfect, but I wasn't making the video perfect. So, like, we're we're definitely researching and trying to solve that problem of, like, video overdub where you, like, take out a filler word and you don't, like, our Playwright test for Webb, So, like, it used to be you had to, like, submit a voice, So, we we figured out How they were storing the high quality,

Yeah. Slam dunk the thunk, monk. Yeah. Because whatever happened to a a regular regular old redux, a mocha choco

I want to do this in Redux, and people are like, oh, you gotta

was one of those things that I looked at

use the Redux duck thunk. You gotta use all these different ones. That's like thunk. Yeah.

So earlier, you said that a bunch of stuff had to be moved to the back end on Descript. Can you talk a bit about that? Yeah. So going from an electron app to a web app, there's a lot of things you can't do anymore.

a long time ago and was like, Yeah. This just feels like a lot of stuff, and it's hard for me to know what's connected to what, what's connected to what. But then once or drop a review if you like this show.

you got the hang of it, you know where everything goes. You know what you're looking for. It all makes perfect sense. But Yeah. When you throw a reduct saga in there. Oh, yeah. Or any of those extra Oh, every time that you ask a question on Reddit, It'd be like,

was, leaning more into web tech in, like, it actually running in the browsers. Cool. So, what we ended up with is,

We have a recorder. It's kind of like a Loom style recorder that integrates, yeah. When when we were building this, we had 2 choices.

it relies heavily upon web codex, which is like a new standard that's coming to browsers. It's not available in everything, so that's why for now, Descript for web is only in Chromium based browsers. But by the end of the year, we should have Safari too because they plan to ship web codec support. Is the, native app or the native app that which is built with Webtech, right, is that an electron based, or or what is that based on? Yeah. It's It's an electron app.

with the editor really well. So that's That's part that for the web transition, we don't have fully baked out yet. But by the end of the year, we plan to bring that same experience into the browser because, and we leaned heavily into Radix.

While, like, most people use Descript for, like, editing files they already have,

It's it's a big electron app. It's actually 2 electron apps right now because, And we've added things,

we've found that Descript is also since it's just a document,

over

my team took that as an opportunity to go, okay. Everything we can do can be open source. So everything was open source.

the, like, the first half of my career, I was very much into open source, and my role at Intuit

what of those 12 years you choose, there's probably a side project that I have kicking around that I'm I'm fiddling with.

because we had to, like, be better in some regards. So, like, currently, opening up a project on in the Electron app takes a bit longer than it does on web because on web, we just stream things. So, like,

like, when you open up a project. One of the things that was kind of Surprising when we made the transition to web was, like, a lot of things were actually faster With the recorder product, any any perceived data loss at all is like that that is a huge ding against the product. So we try to take every step that we can that there's a backup of a backup of a backup. So, like, if your recording But what's cool about This project is it comes from,

And then, also, we don't actually load up all the files,