Supper Club × Dylan Jhaveri - Video for the Web and MUX

You're like, all right, you skip this, like, 8 minute intro of somebody telling you, like, Welcome. What's up,

niche interesting topics I think Wes and I will wanna dive into there. Yeah. Totally. We'd we'd love to help you,

Like, you can hook

You need, like, a full featured audio player with, like, every feature you could imagine, AirPlay, you know, like, all of those things. Yeah. We should totally use this. We would love to help you. We would love to help you. And and we wanna get we wanna get into some really interesting stuff too with it too. We've been talking about

I was, like, fixing a snowblower the other day,

adding reactions or Just fun UI things to it. There's a lot of, like, really kind of

Anyways, I'll let you talk. That that's what that's for, the Muxed Data? Yeah. So Muxed Data was actually Muxed's first

where are people scrubbing back. You know, on YouTube, you can see the little blue, but where people are scrubbing back to. Is that what it's that in the show? in that regard.

space heads? Today, we're with the other day. Yeah. Usually, like, I'm like,

Laying in the snow minus 20, and I'm, like, watching a YouTube video on how to thread a belt. And the guy has, like, an 8 minute intro, but, like, what's up, belt heads? screen screen recording. And then we use media recorder to pipe the stream

What's it called? Oh, that's data. Mux data. You could say you hook that up to a video and you could see

like, stats.

Get past this.

Yeah. So that that's really where the The bulk of the work for the for the player stuff is going into. And and I'm just looking at the mux,

still, like, the the thing in this space, or is there Fancier things? Yep. Yep. It's it's FFmpeg. There's there's, the other thing is g streamer, but we we use a lot of FFmpeg. Wow. Somebody somebody at Mux knows all the dashes, And what that does is engagement. So,

like, freaking YouTube. Sometimes I'll upload a video to YouTube, and I'm sitting there for, like, an hour waiting for that thing to be HD ready

Like ScreenFlow is just CPU export.

Yeah.

For sure.

We we would love Yeah. Slack always thinks first of all, our our opinion is that we're we're we're not of the opinion at least, I'm not of the opinion that, like, kind of web components are the solution to everything. It's going to kinda, like, replace React or anything like that. Like, I I think web component's just another Another tool, and it's good for specific

And then, like, 6 minutes, not even, like, 2 minutes later, it was it was ready. And I was wondering, like, What are they doing differently? Like, couldn't they do that on demand as well? Maybe it'd be too expensive. Yeah. It's a good question. I'm not I'm not sure the internals of how of how YouTube YouTube manages that. Yeah. They should use mugs.

You know, it feels like this might be a a case where Some of these heavier tools, I know, like FFmpeg.

Is it special servers that have

I don't actually know that. Oh, okay. I'm always it would have to be. Yeah. Like, right? I don't know. Maybe I'm yeah. I think it's I think it's big CPUs, honestly, but I I'd have to check. Yeah. When I export my video, there's like a couple options. And like you can do like hardware accelerated, or you can just use CPU. And and by default,

And then the

I I don't know what the status of FFmpeg

one one thing that was always, like, kind of shocking to me within that was the fact that

that you can pass All the flags? into like a stream could come from a webcam, come from your Yep. From audio, and then you pipe that into a meter recorder. And then when you're you got all these chunks

is the is that just done on the CPU?

has Mux done any anything with web assembly at all?

when you put something into a bucket, it's it kicks off that transcoding process immediately and spits out, like, 8 versions of the video

GPUs in them or, like,

Oh, for sure. That's amazing because,

So

By default, in in that workflow,

it. Oh, that's super interesting. And

Hoo. I'm starving.

On this supper club, we're gonna be talking to Dylan Javeri,

I sure hope you're hungry.

so buckle up and grab that old handle because this ride is going to get wild.

This

Yeah.

Welcome to Syntax.

for kind of 10 plus years, I've been a software engineer building products,

is the Syntax supper club.

I'm and, So MediaChrome is kinda like one level lower, that that transition and And move that to Tailwind as well. Nice. We will, so nice and fast. Link to your blog as well. Cool. Well, anything else From, you at Bucks? No. That's it. Super super stoked to talk to you guys today.

so I work at Mux, mux, m u x, .com.

about mux and video for the web, the challenges of video streaming, modern video streaming, future video streaming, and all Kinds of things in between.

Wash those hands, pull up a chair, and secure that feed bag, because it's time to listen to Scott Tolinski and Wes Bos attempt to use human language to converse with and pick the brains of other developers. I thought there was gonna be food,

and it's basically A media company, and you're streaming your videos over the Internet, and they're they're streaming on your website, they're streaming on your Apple TV, and they're going out to, Product.

What does Mux do? Why can't I just put video source equals my video dot m p 4? And, like, where does it sit amongst Something like a Vimeo or a YouTube. Yeah. Totally. So Mux is broadly just video for developers. Our mission is, like, we wanna power

With using that is that users have all kinds of various Internet connection speeds. They might be on Yeah. Mobile devices. They might be in different parts of the world. They just have different Internet connection speeds. And they're not gonna have enough bandwidth to download playlist into a video element, and it just works.

M 3 u eight Into a video element source, and it should just work. And that and that's really what it feels like using this element. If you need to customize it, you can. So, like, Bravo on on finding what I consider to be like a really good use case for

doing video. So tell us,

So the way the way that on demand video and the way live stream video works

You have live video, which is kind of a live stream that usually has a little bit of a delay, so it can have anywhere from, like, 5 to 30 seconds of delay. That's like if you're watching if you're watching YouTube TV,

But that is only there because Mux is an entire

and you're talking in real time. Now segments.

this this is how we kinda think about the 3 segments of video. Now that's a bit of our secret sauce. That's a little special to Mux, which is called dress just in time transcoding.

Like what Mux is as well. So, like, obviously, you have the player and it looks like you have stats and stuff to actually consume it.

that large video to watch it.

give, you give the player the dotm3u8url, So the solution is Media, push it into a buffer that and then have the browser,

Video probably the first thing to understand about video, there's kind of 3 main buckets We would definitely like to Plugged into those tools so that when someone

and gave to you, and that's gonna be the the biggest bandwidth version of that video.

all video on the Internet for developers.

And

platform for

is with what Scott was talking about with the m three u eight. So that's, like, HLS protocol. It's a standard for delivering on demand and and live streams. And what HLS said and the reason the reason why is you can't just kind of, like, Put an MP 4 in a video element. You can do that. Right? But, typically, the MP 4 that you have is gonna be the kind of, like, highest quality, highest definition MP 4 that your Your editor exported streaming technology. So that's what we use is HLS. It it's the the URL end ends with dotm3u8.

you was our very first product back in, like, 2015

then you throw that all on on CDNs, and you and you distribute that.

Speculation. I could only come with spectacles. Like, do you think HLS will ever be natively supported in Chromium or Firefox? You know, I haven't heard much about that yet. I I don't really know one way or the other, to be honest.

over,

and then break into into different quality levels and then streaming having that adaptive streaming Based on based on the bandwidth. And the 1 browser that does support HLS natively is Safari. So Safari is the only one where you can, like, throw an HLS our stance is, like, send us any video, and we will process it and kind of prepare it for the standard HLS output. So We're completely agnostic to any video format that you send us.

HLS natively, it Desktop Safari also supports media source extensions, so you don't have to use Apple's Native HLS on Safari. You can use your own you can use HLS JS, or you can use your own and and use that media source extension. Wild. Yeah. It's so funny because we went from, like, a world where,

on iOS browsers is with Apple's browsers. So it's like the Safari based iOS browsers. They don't source support media source extensions. So the only way to play HLS kind of switched off between working at companies, starting my own companies. But for the past almost 4 years now, I've been at Mux. Mux's video infrastructure for developers.

Amazon has, like, some services you can piece together to do that.

because I was gonna ask, like, can you take something that's larger and clip down to a a smaller value? And Scott starts writing underneath it. You sound fine. I was like, no. I'm gonna ask about clipping video, not I'm not asking my audios. Okay?

HLS

like, render it. So that's what HLS JS does. That's what Dash JS does. It's kinda like Interesting. Pareses all video processes in it and then subs it shoves it into a source buffer. And, So that that project is called Mediacrome,

streaming techniques like Dash. Is Dash still exist? Do you know if that's something that is still around? Because it looks like it was in Chromium at some point,

and I know you you might not have any inside information, but just like Purely

All the other browsers don't. Is that why the Apple announcements used to only work on Yes. On Safari?

stuff, though, I I do wanna, like I I wanna dive into a little bit more about HLS,

Yeah. I I thought that was what we were talking about because we we have our show notes open all the time, and we'll, like if we need to, like,

the 1 browser that doesn't support

like, the and,

and

because when you when you get into video streaming, there's there's other,

anything that's going on in the internals of the player. You just have to say, play this HLS video, and Apple just that, that I work on that that, you must have, like, a, Like a browser running somewhere that just captures the output? Yep. It's basically a headless Chrome that it's basically a headless Chromium browser that spins up Wow. And different technology.

HLS, which means you have no control over the bandwidth switching. You have no control

you we have APIs for for doing all that kind of stuff. What what about something like, So you have on demand video, which is, You know, you go watch a video on YouTube or Netflix, and that's just a static video that's that's what was recorded before and put on the Internet. So you have on demand video.

It's nothing but red. Oh, yeah. Dash is still around. I mean, you need to use, similarly, it's not supported by browsers natively, so you need to use Dash. Js or something similar too. And it it's a very similar concept To HLS, it takes a video, breaks it up into segments. It's just the

MSE, media source extension, which is how you can take

communicate with each other. We'll do we'll write write little notes. Well, you wrote clipping. I was like, no. I think it sounds okay.

What we're actually outputting and what we're what we're delivering so it's so that you're always trying to make this trade off of having

Optimally, like, transcode versions of your video that makes sense based on, like, the content of the video or based on the audience of who's who's who's viewing that video. So If your video, for example, is mostly viewed by by users who are on, you know, Yeah. Where I know, I know that one of the main developers who worked on the on a lot of the mikes.com stuff. I know he listens to the pod, and he'll be he'll be really excited to hear hear hear what you have to say. Sweet. Yeah. We're we're happy to have you on because,

low power devices that are watching on their phone, you you There's probably a different bit rate ladder that makes sense for your videos that would that would minimize the amount of bandwidth and and

We have We have tried stuff in the past, and and we do have some stuff, that we've experimented with And then we use Mediacrome to build MuxPlayer. So, everything everything that we built with MuxPlayer

like, you're like I I almost see it as as, like, electricity. You know? Like, you're not,

the

And, maximize the quality that your users are getting. So there's a lot of kinda, like, machine learning and things that can go into into optimizing

the least amount of bandwidth with the highest quality video, can be used for, you know, for the next 10 years by by by developers and and not have to be constantly chasing the the latest trends. And When we evaluated the options, just plays it. Interesting. Desktop Safari, even though it supports

with being able to Do video. A lot of developers learn through video. I'm a self taught developer, so video was a big part of how I learned to code. So, yeah, super stoked to be on and talk about it today. Nice.

again,

We wanna power the actual video when you have to distribute that video and actually have to send it to people and want to stream it or Turn it into a livestream and be that. So, we we're not really using AI what we do is you send that video to us. There's kind of 2 paths that goes down. We have, like, a we determine whether it's, like, a standard input or a nonstandard input. If it's a standard input, it just means, like, It's, it's using codecs that that we recognize. It's it's,

today. Yeah. It feels like something you gotta ask everybody now, though. It's like,

And that there's a lot of factors that go into that. It can depend on the actual content that's in your video. If it's like a sports game and there's a lot of action and moving back and forth versus, like, a static talking head versus and then depending on, like, where users are and what kinds of devices they're they're viewing on, there's a lot of, like, kind of, yeah, machine learning and things that can go into To optimizing that that, that we've kind of explored in the past and done some stuff with. But yeah. So we don't have, like, a a direct kind of, like, how are we using AI AI to power to power video, Cruise is allowed to take passengers, and there's an app. It's just like it's just like Uber, Lyft. You just, you know, put in your destination. There and

as directly.

have done some some version of, and we'll just record it. And Wow. Broadcast broadcast as live stream, or you could just save the recording.

with live streaming or even, like, when we did our live episode,

record the screen they have some They have some pro they they own web developer.com.

it feels like there's a lot here for us In terms of potential APIs. That's really cool. I'm excited about that. And how does the how does that work? Do you have, like, a before I can post it. Because otherwise, you post it, and everyone's like, the the quality is awful. It's like, yeah, I know. But the other day, I posted 1 to YouTube, And it's like, it'll be 55 minutes to get 4 k.

like, real time video. So Oh, yeah. If you're so so we so we have a, API now. It's in, like, private beta where Anything you can just anything you can compose in a browser. So you're just using your JavaScript and CSS you need a way to make sure that your users your customers are having a good time watching your video. Like, are they seeing, like, buffering spinners all the time? Are they, Great. And Yeah. So that's that's what that's what we have to to play around with. Wow. When did you guys redo your website? Because I,

If anything you could create in a browser, Yeah. We we did we rooted the docs site.

You could just clip it, bring in the audio and the transcript,

we were Trying to broadcast, and we ended up having to jump on Zoom to do a video call. And, like,

and piecing different tools together, and then we and then you just send us the URL to that to that page, Cool. Shameless Plug anything you would like to plug. Just the new mux.com, the new the new mux the new mux brand site. Check it out. Yeah. Definitely check it out. Alright. Thank you so much coming on. We appreciate all your time and,

that I think I'm really excited to see people get kind of creative with that. That's Yeah. Because you you could bake, like, a OBS have you have you seen this stuff? What what are you doing with it? Yeah. Yeah. Yeah. Our bet is kinda like, as long as developers are building stuff and putting video in their applications, then then that's, then that then that's where Mux fits in, and I don't think That trend is is going away. Yeah. You guys are lucky in that, As a web component,

we just wrote a blog post about this recently, but it's called a web broadcast that we're experimenting with, which is

as it plays it back, And then kick that out to MP 4 or some Yeah. So we actually have something,

so, yeah, this is a this is a, like, API. I'm happy to happy to let you guys try it out. Yeah. There's a lot there for us, Especially because we've been experimenting related to the web components and the player, that I wanna touch on, which is So you're familiar with Mux Player, just the web component that you drop in.

behind the scenes if you inspect the source. So, like, you you guys are really you're not faking it

And,

And the the doc site we rooted on Next, 13 with serving components as well. Darius I'm, And Then and then we just cache them over over the CDNs, like, as so if you if you go to a video and no one watches it, then we never have to actually and you could just broadcast that as a live stream or, you know, you get the recording after a live stream. So if you were able to create Just a page that has you know, it could have video coming in over WebRTC.

And, you can keep thousandth of a Bitcoin.

Said they're they're discontinuing their custom domain product because it's it, like, stopped working. Yeah. I think the ad blockers are trying to get around the custom domains. Yeah. I think they just resolve the DNS and, like, figure out that, where that domain your that domain is pointing to, and then they just block it. Yeah. I even saw like a Google Analytics, like a hot potato service that you could run. You take your data, you throw it to your server, and your server will Then hot potato throw it like form it to like other servers.

So if you scroll down from, like, the square video, there's all these little video screens. You have, like, a fake screen, a fake Zoom chat,

I had a call with them just to see, like, what it what it was all about, and I could not Get on the Zoom for the longest time. I could not open up the Zoom link they sent me. And finally, I realized, oh, My ad blocker was blocking

Oh, I don't think anything of it.

And, web components here. Let's talk a little bit more about,

Yeah.

grid squares And used it like a typical web grid where you're placing things on the web grid rather than we often think of, like, alright. You have these big squares and you're moving big things around. Right. Like the Twelve column or whatever. You guys made an actual grid and then placed things on that grid like Oh. The web of yesteryear in in, like, a really modern way. It's so good. Love that. What do you think about all the Tailwind on the site, Scott?

hear it all on the website, just tossing in a a GIF or something that looks like a movie. Y'all are really consuming the video in a

like, another fake mobile screen. The like,

But I would imagine

thing where, like, you can write articles Kick that out to like an MP4 that that is then shareable. But who would do that on demand for people that are like, oh, this Scott says something really cool. Can I share that little bit?

So,

like, really fine

like, tip people with crypto, like, you you get, like, Satoshis, which is like a

that the ad blockers were gonna say, well, that looks like Tracking data, especially now with AI, you know, like, then They start expecting the payloads, and then they can tell from the payload. Mhmm. That's that will definitely mess some stuff up, though. Some stuff might look like tracking it. Yeah. It actually might. Well, it already does. I had a I had a call with, I sell ads a couple weeks ago,

I think the website does a great job of showing video used in non typical ways.

the little, like, One platform video for everything section has a CSS grid section where they did a, Before we get off of this,

I don't have no thoughts there. It's a Sweet sight. So Tailwind, Next. Js, I'm just poking around as well. Some really cool stuff on there. The Next 13 with the, you know, server components and stuff. Oh, man.

ad blocker.

Even the CSS grid here is pretty neat, West. The

They have this new, like,

that video is an actual video in the In the mobile screen using mux player,

Yeah. Yeah. The domain was I sell ads? What was it? Buy buy. Buy, sell ads.

It's actually it's 2 separate sites, mux.com and docs.mux.com.

fraction of a Bitcoin.

Inspect the dom and, like, see what's going on. So we we didn't wanna cut any corners.

unless you have anything else, there is one thing I I do wanna shout out once more about the new website because I'm obsessed with it.

Most ad blockers in the browser? Yeah. There's just no way to get around that. No. That's everything we do. Yeah. We don't collect any, like, Personal information or whatever, and it's like, we we tried to we tried to make the case for ourself, but the all the all those blockers are, like, very, very aggressive. Yeah. We have, like, custom domains and stuff, so you can use custom domains to, you know, get around that. But, I think actually custom domains I saw Fathom recently

so So you can kind of take all the data out of Mux Data and process it

engagement

Awesome. Is it I I think I probably know the answer. Is it blocked by

Their video infrastructure. And that was Mux's first product, and,

Insights are, like, granular level level,

And Muxed Data was first built as it's a tool for,

the playback

Experience, and it's constantly sending sending beacons to the Muxed Data back end and that and Muxed Data's processing that and then giving you all these metrics. So you see,

exit before video start, all these metrics. And you can break down these metrics by region, where they are in what part of the world, what What their Internet provider is, is it like is Comcast, like, slow in the northeast right now, or is your CDN video in the browser, you're trying to, like, do a live stream from the browser.

Mux Data has done really well. And now since then, you know, we've added on Mux Video where it's like, okay. Instead having your own video engineering team, why don't you can just we could be your video engineering team, and you can, you know, give us your video. And now we've sort of sort of bridged the gap, and we started to add, what you're referring to, Wes, is more like engagement metrics. So beyond just the, But then it came time to,

you know, Samsung TV and everywhere ever it every everywhere you're distributing your video, maybe it's like sports live sports games or on demand video, whatever it may be, Like, I tried something like the auto g p t stuff and tried to do it, and it's like, I can never get it to work as well as what people are saying it can do.

put it in your own own data warehouse to to process that to to come up with those kinds of and then capturing it with media recorder? We use get display media to capture

quality of experience. So you can imagine if you are and it really it's it's kind of originally started as kind of more of an enterprise 2 of them are very similar, and one of them is completely and then it just drops you off. And then you're like you turn around. You're like, bye. And then you're like, but no one's even in Car. And then did it feel weird, or was it like If I swear it felt weird for, like, 3 minutes. And then it's like and then you just kinda, like, you know, whatever. Look at your phone and just, like, kinda forget, Like, that that no one's driving. It's it's it's completely wild, and it feels like

in

in addition to I guess, we can talk more about the web component stuff. But in addition to a player, You guys also have, like, uploaders now, cameras and sensors and all this stuff. I gotta I gotta get in one of these at some point, but I'm also very scared. You gotta get that update. Somebody's somebody who's listening, slide Sky USB with the firmware for a thing. Here's a robo.

A challenging one, or was it an obvious one? It was was

around here. What was the was the choice to go with web components Yeah. There there's a lot of really interesting stuff here that I I can't wait to dive into. So I've been using,

Quarter no. Media recorder is a screen recorder. Yeah. Media recorder, you can capture camera Oh, buy, sell ads. Gotcha. Yeah. Probably their whole domain name is is busted. Yeah. Right. Yeah. Cool. Well, I think that,

of data, and then you you could send those chunks to mux, I assume. Right? And were you sending those to

So then I pipe that into some JavaScript and based

You're not gonna be out AI'd

another tool to do the thing with the audio and the text? startup time, rebuffering,

I'm showing the transcript and highlighting the current word. And then I was like, well, could we record this

somehow

The transcription as well as, like, timestamps for every single word.

the user's,

And so we're I was playing around with, And one of those is

and kick that out to

No. No. We were Right now, it just manually took one of our like transcript outputs and and manually import it. But ideally, that's the end game is that you could

And you're almost like a utility

builds a builds an application video, it's using Mux behind the scenes. They put in their Mux API keys or whatever, and it, like, does the API calls and sets up the webhooks and just, like, Handles all that for them. That's actually something that we were just playing with on Friday is, I built and it's an SDK that gets installed at the player level. So you just install the SDK in the player, and it monitors

because somebody uses AI to build a better platform or something like that as people still need to get video to the end user.

Media recorder API in the browser and

podcasts into, like, a transcription thing, and they gave us back

or screen. That's awesome. So we're trying to figure out, like, Now can we do that headless? So you're doing, like, get user media

the screen

on where you're playing the audio,

you know, those, like, like, podcast clips where people put text Over top of it, and they highlight the current word and whatnot. So we we piped 1 of our

Uh-huh. Screen Tacking on. So something I have to do before I launch a course. I have to go through every video,

or the APIs.

it is gorgeous. The new one, it it reminds me of That was

And, like,

You know, we we maybe talked a little bit about I've been using just Straight up only the the video stuff. But we also use the live streaming at some point for one of our I think our 300th episode we broadcast

on Friday. It was kinda, like, soft launch on Friday. Oh, wow. People started, like, kinda talking about it over the weekend Ken, before before we kind of, like, fully announce it. Yeah. My gosh. It's great. The colors are great. It reminds me very much of, like, Ableton Live kind of Design, in a good way. There's so many cool things here. What are, like, teenage engineering style design in a a very positive way? Yeah. The the team really, really crushed the new, like, brand brand design.

Yeah. The playback ID and and just stream away, Tossed into,

I I wonder a lot about, like, video format changes. Like, there's WebM and things like this. Are the the code that what is the Playback ID?

Captured the video and audio and piece it together, and it is,

Clip. You're essentially you're loading. Right? Is that dependent on the actual video file itself? Like, do those have to be dotmp fours behind the scenes?

an HLS video player, and next thing you know, you're you're up and running. Yeah. I gotta wonder about, like, video formats a little bit. I know I know we keep going I I keep going back to that stuff. But when we talked about, like, HLS, you said it's, like, kind of like a a map for, like, which video Do you wanna give maybe just a a little background about who you are and what you're doing?

does does that Work at all? Does that fit into play here at all, or or or is it always dotmp4 components can be done. Right? Because too oftentimes, we are, like, getting into arguments about web components,

5. 5. Okay. Yep. Yep. How long does that take? Like like what's happening? Do you have, like, a Someone's PC sitting on their desk at Mux that has a sweet graphics card in it. Like, are you using some sort of, like, GPU

And and after that, after we tell you it's ready, like, It's ready. You can play that video.

I when I send you a video, let's say I send you a video that's in 4 k, and you guys you guys will then process it into

where Where it's like, okay. Load a video player. Give it a URL. It doesn't really have easy ways to, in a performant way Yeah. Preload, you know, preload just small segments of video and and do Stuff like that. When

That video and start watching that video, that's when we actually transcode it and actually make the HLS and then use that player

How many versions of that video? the hardware accelerated version is really not any faster. I don't know if it's wrong with it, but, yeah, I was surprised that I was like, oh, it's for sure a GPU because I was looking into, like, building, like, a custom hackintosh with, like, a $8,000 Yeah. Those are the kinds of things. Oh, yeah. Yeah. That would be pretty cool. I it's funny. I my then my next point I have here is clipping question mark,

like, as of right now, iOS and Android and stuff like you, that's all very custom. Like, they don't really have APIs available for just kind of off the shelf, especially if you're using React Native because React Native is even, like, 1 layer of abstraction higher, builds that that video player and the tech behind that video player. Oh, nice. So as a as a fan of mediadashchrome.org,

Transcode it and save those segments of video. We just save it in this, like,

specific format that's, like, ready to be just in time transcoded. And that makes so much sense. Yeah. Because I was getting into before I went with Mux, I did a bit of a tour on a lot of the different, like, video hosting things. And one of things that I would really looked at was the AWS, like, VOD workflow that they have. And Mux customers in a different way. Yeah. So does this work with audio too? Is that

heavy server?

you know, pixel density, and all and all these factors make it standard. And if it's standard, Then we're able to process it really fast. So a, you know, hour long video, it could be ready in, like, a matter of seconds to using. There's some other open source ones out there, but Video. Js was kind of the most common. And a lot of our customers, they'd they'd use Video JS, and they'd try to use some of the some of the plugins out there. And, they'd usually find a way to get it to work, but we what we found and Since Mux is just video infrastructure, like, from the beginning, we The 2 that are similar is on demand and livestream.

and not bounce. And, these are 3 If you've and maybe you've seen RTMP, like, if you've ever used OBS and you put in the RTMP. So what what that happens is you'll be sending your video and audio to an RTMP server. The RTMP server will ingest that video,

the grid the grid buttons, the how they expand to be from and an audio element, and a viewer count element. So it seems like Y'all have, like, a a focus on building UI elements for video as well,

muxing slash media chrome media dash chrome. And media source extension is iOS

Oh, yeah. That's funny. Yeah. The TypeScript kinda stole that one. I guess that's why we have MIME types though, not just extensions. I'm having so much fun playing around on your new website. I love the,

Different containers. So, like, with the HLS, that mux that mux outputs.

That, again, that kinda like the end users end developers, you know, using mux never have to really, you know, think about or worry about. Yeah. Totally. And who wants to worry about that stuff anyways. Right? The TS files reminds me of my days of on our team wrote wrote a blog post about that, we don't we don't want your your customers to know you're using Mux. That's kind of a you know, that's between us and you that that you're using, Mike. And we're and we're just we're just the infrastructure. So you you're you we wanna make sure you can build whatever experience you want process it, and create these little 6 second chunks of video, A surprising amount of this is just text files that are, like, not not formatted like JSON. They're just, like, text files with line breaks that you have to that you have understand and parse, and that's what the player understands and parses. So the player will look at m 3 u 8 and see, like, okay. I have 6 different quality levels, and each one requires a certain Amount of bandwidth to stream. So it'll start it'll pick, like, the middle one, for example, and it'll just start trying to play the middle one. And it'll start downloading 6 seconds of video at a time. And while it's downloading build the audio player, make it make it awesome, make it look like syntax. We can make it a theme, called Crowdcast and did, video streaming.

and But now the kind of video world is is moving to to fragmented MP fours, which is f m p 4, which is just, like, It's basically the same m p 4 container format, but in a in a fragment. So it's not the full it's not the full m p 4, but it's just a way of breaking it up. So, right now, if you do mux If you do a mux, low latency livestream, you'll get fragmented MP 4 out on the other side. But that's sort of that's sort of just a detail and that was kind of built almost just feels like for a little bit of a different era. It's, like, kinda jQuery based,

and it's on our GitHub, And then Mux gives you other things, like if you wanna add Captions files to that video. If you wanna pull out thumbnails from that video,

Right now, we do dotts files is what you'll see. So those are those are transport streams. Dot ts is the actual, like, video Kinda got involved with the folks here and then joined joined here for the past 4 years. And right now at Mux, I, end users can download those

Border radius tonight. Sorry about that. It's it's, like, not not relevant at all. But, man, this new website's great, so shout out to the Mux designers. That's awesome. I'm super I'm super happy to to To hear that. Our our team worked they they worked really hard to get that out,

So MediaChrome is just a general project for anyone to build video players, And what it does is it gives you all the things that are missing from the browser when it comes to building a video player.

So you're familiar with, like, building a form. Right? You have a select element. You have, like, Radio, Like, we make your videos ready really fast. So even if you send us an hour long video, It was really it It was really practical, you know, you're kinda like cable replacement,

burning DVDs.

that TS TS files are TypeScript.

is is where kind of All of our efforts going into when it comes to comes to the player work.

what that that project is really kind of, in the browser if you wanted to. Right? Exactly. Yeah. Yeah. And that's what a number of people, you know, building these live streaming platforms quite it it it's sounds simple, and it's one of those things you dead dive into. And then it's You can never get the audio and and video in sync. You can you know, there's all kinds of problems you run into as soon as you start start trying that.

Yeah. Right. And, that's also why VLC always wants to open my TypeScript files. Yep. Totally. Totally. You're poor developers who have to use both.

last week, and everyone's everyone's super stoked about it. I'm just looking what fonts, these are using. I know. The fonts are

format,

good. There's 1 more thing I could talk about,

using a lot of bandwidth on my servers? Like, it's it's not something that really crosses your mind in, like, Modern web development. But as soon as you start doing something with video, it, like, becomes a serious expense of the amount of bandwidth that you're using. And So, you know, even a product like TikTok, like, the they're they're very finely optimizing how much they're preloading to make sure, you know, they're not paying too much of the bandwidth, but also that users are able to Kinda see that constant content instantly mostly well earned. But there's a miracle that's happening in San Francisco every single night, which is from 10 PM to 5 AM. They're self driving cars taking passengers all around the city. And, like, normal people, like, instead of pulling out an app and order an Uber, you order a self driving car and just comes and picks you up, Takes you to where you need to go, drops you off.

Do do you know if, like,

like, It's and it it starts up in, like, maybe maybe half a second, which is, like,

Like, companies like TikTok or Instagram, are they streaming all of that video? Is it is it using

is how aggressively they kind of, like, preload videos.

but just, like, the beginning of each video for, like, the next 10 or 20 videos in the feed. Because They didn't wanna put the dash in there. They didn't they didn't wanna put HLS JS or whatever. Oh, yeah. Yeah. They didn't wanna load, like, a third party HLS library to make it work in the other browsers. So, yeah, they you had to go to Safari and use Yeah. Yeah. So that's interesting. I wonder, like, do you think so So you can so it can be and when it's a theme, it can be kinda portable. So you can put it on different if you have different websites you wanna put it on, you can use it in other places.

HLS, do you think? Oh, yeah. Yeah. Yeah. I haven't checked for sure, but they're definitely doing some something like that. They're probably using either HLS or Dash, and they have that integrated with the player. The thing that TikTok does really well and those apps do really well that Everyone is trying to do now, which is is quite hard to replicate, a shameless plug? I do. I I have 2 sick picks. Cool. Those are, like, You know, recent technological advancements that I'm stoked about. What first is obvious is the obvious one, AI.

you know, scrolling. It's, like, loading everything everything fresh every time it scrolls, and it's, like, You know, it's kind of expected. Maybe, like, 100 milliseconds or 400 milliseconds. It's like sort of they're like, no. I wanna be fast like TikTok. And it's like, Well, in order to do that, you have to like, TikTok does some really advanced stuff where they preload not the entire video,

you know, you did every video in Flash and then you have HTML 5 video comes out. But it it feels like This this kind of streaming based video is really the only way to do video nowadays.

the thing about video and the thing that for me was a learning was, like, video is expensive. Like, if you're if you're just building a SaaS app that's just doing, like, basic CRUD or whatever, it's like, you never really need to think about Bandwidth. Right? You deploy it. You deploy your your your app somewhere. You you're never thinking about, oh, are users Like transcoding and stuff like that. We we've kind of looked into it and and played around with it and kinda like hackathon kind of kind of ideas, but, we're not doing anything with it yet. But I could see it being handy, And they feel and our goal with that project is they feel like you're just building them into the you're you're just using the browser, Now on the other side of that, you have, data, like, per video.

Mhmm. So it's like, We we get users all the time where they're, you know, they're building an iOS app or they're they're using, like, React Native or something, and they're, like, you know, scrolling through a feed of videos, and they're like a volume range, A mute button, a full screen element, a picture in picture element. It gives you all these, like, kind of low level controls, that at the end of the day weren't weren't necessarily exactly our fault, and we're like, okay. We can document this better. You know? We can We can try to make it easier Yeah. So that's kind of like a general purpose work of your application. You should feel comfortable with it. You should like using it, and, probably the most the biggest video player that you know and heard about is Video JS, and that was started by, actually, one of the founders of Mux back in, you know, 2010,

Yeah. Like, saving me so many steps of, from start to finish. And the craziest thing is you start riding and after in, like, 3 minutes, you completely forget that no one's driving. It just feels it feels so normal,

I mean, you you guys are seem like the the forefront of modern video on the web in a lot of ways. So it's, really, really excited to to hear from you. Do you have a sick pick or

You know, Stack Overflow documentation, this, that. Like, I I actually find it very handy to, like, post and paste in error messages, and I pretty much have it open all day. And it's kind of, like, has now become kind of the 1st place I go to, and it's and it's definitely helped me definitely feels like it's it's better my my day to day workflow. And I'm just super excited about to see kinda, like, how it still just feels so so young, so I'm just super excited to see how how things develop in this space,

closed. I guess it's beta. So, like, it's not just public yet, but a lot of people have access to it. I've done a bunch of times. It's, like, Blowing my mind, and it feels like no one's talking about it that, like, after after 10 PM Yeah. Like, there's a lot of people just around self driving cars, like, with passengers going all over the place, and, like, no one's talking about it. And it's, like, it's kinda blowing my mind that This is happening. Like, the future of self the future is, like, here. Yeah. Yeah. That's a it's, like, honestly amazing that you think about that that Literally, a car drove somebody We're really excited to get it out. Yeah. It it feels it feels great.

is self driving cars. So everyone in the everyone, you know, loves to shit on San Francisco, Cisco, and it's mostly And then what we do on our end is when users start requesting

over the next, I don't know, few years. The second one is, we're just trying to be video infrastructure. Right. So if you're an AI company that's doing something like generative video and you and you build all the stuff where you can, like, type in a prompt and generate And and get this, like, video that's generated.

glad to to pick the brains of

I think in some ways, The stuff I see on Twitter, AI is, like, overhyped.

it's so crazy, and I feel like no one's, Like, acknowledging how how crazy it is.

Honestly, it's a service that I've personally been really enjoying for a long time and, the audience.

It's, like,

version of FFmpeg that you can run-in the browser, and you could do things like Mux has a has a WebRTC product now I can talk about. That was kind of the 3rd bucket of video, is WebRTC,

So we're like Using a an adaptive bit rate, How does that work? It is a bunch of big servers, and we do something that is very

Transcripts and subtitle stuff with AI. Is that something that you all have hit, or have you found that it's not necessary? Yeah. We've we

changing how you're thinking about some things? I know we've been looking into a lot of

it's like a WebAssembly behind the scenes? Yeah. So as far as The original format of the video, transcode it just in time.

in the browser is, but is this something that y'all have looked into at all? Yeah. There's a really interesting project called FFmpeg WASM. It gives you, like, the f,

Our kind of exposure to AI is more like through our customers.

Exploring some of that stuff where if you're if you're actually, like, recording your video in the browser or you're trying to, like, compose one of our main products is Mux Player, and that's open source. So that's, like, a team

sustain yourself in all sorts of ways of interesting things to work on. What about, like,

Hit you guys at all in terms of Sent the syntax website and did it live, and it really functioned just like you do do a live stream anywhere else. You you pass in that The

So we are we are starting to play with that with that kind of stuff to see to see what's possible. Yeah. No. It feels like a a space. Just in general, video for the web feels like a space that there's enough exciting things graphics card to go faster. I did that. Yeah. And I talked to the folks at ScreenFlow, and they're like, I won't make it any faster. It's it's really didn't really? Yeah. Yeah. Yeah. That's what I ran for a long time was a beefed up Hackintosh, and it was a Yeah. In the butt. It did not get me that fast. And it didn't help? Yeah. It was like not it was not very much fat or not fasted. Like, you'd expect something like that to be enough faster. You'd be like, woah. But it just it never really was.

available in the future or potential future that You could probably

has the AI stuff

And then,

you have a few options. You can build all this stuff yourself.

each segment of video, it's measuring how long is it taking to download that video. And if it's taking too long, then it'll it'll downgrade to a lower quality. If it's if it's downloading fast, then it'll scale up to the higher quality. So you might have experienced this if you're watching Netflix at home or you're watching a YouTube video. And if your Internet connection just suddenly slows down, you'll still be able to watch the video, but it'll just start to look visually like a lower quality. And then if your Internet catches up, then it'll it'll ramp back up to the to the higher quality level. So so that's the way kind of HLS delivery works. And you can do that with an on demand video Because you have kind of that MP 4, you could process it. You can do that with a live stream. So with a live stream, typically, you'll you'll ingest that video over RTMP.

and then push them out over CDNs.

dumb query query string, and it, you know, kinda builds it Throws throws in a bunch of dibs, builds out the player, and, The way we get video into those browsers is through media source extension. So that that's kind of the important API, use cases. Yeah. I think a video player is a really good use case for for a web component.

So coming over from that, it didn't really make sense for us to have a video player first. Right? We're just video infrastructure. There's plenty of players out there people can use.

around the edges for these players. We can we can put in pull requests to fix issues and other players out there that aren't even ours, and and we were doing all that kind of stuff.

running into problems with video players out there. They wanted an easier solution. They were, they're like, I I love everything I'm doing with mux, but why am I having a hard time with the player? We'd have kinda support issues, and then we're we're debugging other players out there. And it's like, you know, we're we're dealing with a bunch of the issues ourself, that matters like, Muck kind of abstracts that away. So it's like whether you send us an MP 4 or a dot MOB or any any like,

hey. What you know, what have we gave our customers an yeah. A player that they could drop in and just use and get and get all the functionality

fully custom.

that was kind of the player that most people kinda defaulted

if you're kinda familiar, you you you just recognize that player when you when you see it in the wild, and and Mux very much tries to be transparent. Like, If you're if you're a Mux customer, and, that's just a container for what's Actually, inside the video, which is h 264,

honestly. Like, and that that's what we've we've kinda seen peep people kind of repeat building that kind of thing. And, radio buttons. You have checkboxes.

we use open protocols for delivering video. Our our stance is, like, You could bring your own player. Like, we're not trying to be a closed ecosystem of you have to use our player. Like, you know, some other platforms, it's it's sort of proprietary, and they kinda force you to use your that that player. And then and ready ready for playback. So we'll do, like, 1 processing pass on it and basically Put it into a format where we can, does it is it the right resolution for their screen? There's certain sections of the city it's allowed to be in, and it's it's quite a big area now. Oh my gosh.

kinda feels like you, you know, you have a constructor. You pass in, like, a Australia slowed. You need to fall back to another CDN. So video engineering teams use Mux Data in order to monitor quality of experience, thing we have as we do more of that, I could see us

and that and that's how we approached it. And I can I can talk more about the the kind of web components angle and stuff like that? Happy to happy to talk all about that. Yeah. I'd love to get into to some of that stuff. It it does feel like instead of supporting 1 video player, you were having to support all of the Video players out there do support for them. So but,

But what we found is that customers were, Is it is it skipping? Are you having all these kinds of quality of experience issues. So Mux Data,

because

just couldn't be any more seamless. I often use it as, like,

And,

is video streaming. Right? It feels like video you should just be able to pass in an h l or m three eight u whatever. M m three u eight. Is that it? Yep. You got it. M 3 u eight. Wow.

of the way that web

people are pretty happy just using web components Out of the box and and and all the other frameworks. So it's been it's been working out well for us. Yeah. We use the video element for level up tutorials, The one thing that

An example

we use just the straight up web component in a SvelteKit instance, and it was,

It replaces something that feels like it should just work in the browser, which

in terms of having to take over your entire stack or something like that, would the the the mux video player is the perfect web component for me For everything you upload all the time, even if no one's watching those. And, like, that always felt, like, extremely wasteful to me. So it it Makes perfect sense as long as that the transcoding process can be fast enough. The is is FFmpeg

web components

every single framework. And as As you as you all know,

of you 1, and, and and we have to build, you know, a specific player for every single codec, and AAC, kind of templating language that just feels like HTML that you can just write HTML and build a theme And Mux tries to be kind of the the simple answer where You 2 who have used Mux, you probably you you never had to think about how many quality levels am I doing. Am I am I choosing the right bit rates for each video? So Mux tries to be the simple answer where you just Throw your video at us. We'll process it. We integrate with multiple CDNs. We give you a playback URL. You just drop that into a player, either our player or any player you want That works with HLS, and you can just get that get that adaptive beat bit rate streaming experience.

frameworks come and go. They they get popular, then they then they, you know, wax and wane in popularity.

feels native to the web.

are using one of our s one of our elements in your application. It should feel like the rest of your application. It should feel like it should fit in with all your application code. It shouldn't feel like this weird How quick is the video starting up when they hit play?

You know, we Could've we could've spent a bunch of time building a React specific player And what, adaptive bit rate and what HLS does is you take that high quality definition video, like the MP 4, And you break it up into little small chunks. So you break it up into, like, 6 second chunks, for example. Mhmm. And then in each chunk, you process it into maybe 5 or 6 different quality levels all the way from the from the highest quality level all the way down to the lowest.

web components met that for us. And it it it plays really nice in in all the frameworks, And we have to have this special React wrapper to do to to make it work to make it work in React, and that's just kind of the state of things now. I think React will get Better at web components support. It remains to be seen if the community will kind of embrace it. It it kinda feels like maybe not or maybe maybe at least not yet. So so we're happy to, you know, do what we have to do for React to make React developers happy, but it seems like so far, every other framework, out of that that they want, and, Well, we actually built Mux Player. Like, we we kinda built a web component framework for building players.

and doing that, but then we'd also have to build

And we wanted to build something that Modern way, and and it's really, really, really sweet. That makes me so so happy to hear. We, we were definitely expecting people like you to inspect

it was kind of obvious when we kinda weighed all the All the things we had to consider. So the files look a little different. Like, HLS is basically represented in just text files with line breaks. Dash the the manifest files for Dash look more like HTML. So just the the format of it is a little different for how the player understands what to do. But Underlying, it's the same the same idea of taking a video, breaking it up into small bits,

Like, kind of the principles we approached it with is that when you especially when we as you kind of alluded to, we're you know, we have the player. We have the uploader. We have, like, other elements we're building for the kinda application layer. And,

Yeah. It's, like, media. Right? So it's general video and audio. Yeah. You can build an audio player. Exactly. Yeah. Oh, man. We need to we need to build the news. We're so we're rebuilding Syntax website right now.

to look like Mux Player and swaps out the icons and stuff. But but, really, what we wanna do is enable any developer building a video player to just use that project, build a video player, And design it the way they want and just, you know, be able to ship that and run and run with that. Even if they're not using Mux. Right? They're today, they just have some MP fours they wanna play. They could build a player with that, and then, you know, maybe later down the road, they they decide they need mux for live streaming or real time or they wanna do more video, then they can start they can start using mux, and they can, you know, then then become, of video on the Internet.

change this config, and they all have different syntaxes. Or take the CSS file and copy and paste it into your project and then just, like, Edit the crap out of it and, like, override override all the styles. Massive CSS file. Yeah. Exactly. It didn't really feel like you were building anything custom. It felt like you were just kind of constrained to what the player gave you.

we and the latest iteration of MediaChrome is that this idea of themes where we have a very simple

for with Mediacrome and have it work with any of those video sources.

in various places. So you can you can put a play button, then let's say you have your own button for, like, for, like, a a syntax FM, like, call to action button. Button. You could just put that in your player right next to the other buttons, and it just you're just writing HTML. So it just feels it feels natural, and we have things we handle things like she uses Scott? Like break points so you can you know, when a when it's in a small container, you know, you can lay out the buttons a particular way. When the container gets bigger, you can Change change change the layout of it the same way you would design design a web page. So that's really kind of the the secret sauce behind the MuxPlayer work. MuxPlayer is actually Doesn't do a whole lot beyond Metochrome. It it uses it just use it uses the Metochrome theme that we've built to be

was is is built with MediaChrome, You know, you can use all the FFmpeg, use all open source source stuff, and Figure out how to do that, integrate with CDNs, and and and do all that stuff.

when if just, like, the individual components for building building a video player, and That project is meant to be compatible with with anywhere your video is hosted. It could be an MP 4. It could be it could be on Vimeo. It could be on wherever it is, and you could build a a video player So in some in some ways, I feel like it's maybe I'm just not prompting it right, but in some ways, I feel like, a, it's been overhyped. And then in other ways, I feel like it's completely underhyped. And now I'm finding that, like, you know, Basically, throughout my day, I have chat g p t open, and I found it really helpful with debugging.

just bill using HTML to kind of The same way you would build and style a form, you can just you do that to build a video player. What we found is that all the kind of all the video players out there, when it comes to customizing them, It's it's like, okay. Change the colors or, like, and you're watching that, that's You do or if you're on Twitch, you're watching Twitch, that's like live video. And then you have a 3rd bucket, which is real time video. So real time video is, like, You're on a Zoom call. You're on a Google Meet, product where you can imagine if you're

That's significant. There,

Who, what app, or anything like that, but they look so cool,

Head on over to syntax.fm

They're they're you see them all around. You see during the day, you see the Waymo and the cruise, like, both of them cruising around With no passengers was actually finally at a point where we could it had brought enough support across the evergreen browsers, and we could actually build something and I have to put a little 3 second bumper in front of every single one. Is that something you could Take a sec. Alright. I've Bumper Dime before. Slap that in right before you start playing it. That's stitching. We don't we don't have that yet, but I'm gonna put in that feature request We have the opposite of that, which is clipping. So we have the opposite of it. If you have a video and you wanna clip part of it, we don't have it yet where you can kind of, like, add parts onto the video. But,

It still scares the life out of me for some reason. I saw a video the other day of, Somebody was in 1, and a cop is like, alright, pull over. Like, there was an accident, and the cops, like, pull over. And the guy's like, I'm not driving. There's nobody in the driving scene, and it didn't know what to do. Oh my gosh. That's wild, though. I can't I wanna try what's the company? Is it Waymo that you you use? It's I don't think Waymo's allowed to take passengers yet. The one that can take passengers is Cruise. Okay.

That's right up my alley.

Yeah. That's the one that GM bought.

with no passengers, no driver, no passengers, empty cars, like, cruising around during the day, but after 10 PM is when And the problem So what MediaChrome does is it gives you a play button, those chunks and integrate it with a player and just be able to watch Watch that video and consume it. So if you are a developer and you're wanting to put video in your application, I don't mean that in a bad way. I mean that in an awesome way. You know, like, you are the backbone Bone to getting the video to your user. Exactly. And, one thing for sure is, like, as these AI tools develop and as it's like you can build applications with AI and people are you know, describe try, but, you know, you've seen the demos where people, like, describe an application and kinda, like, build it for them where they can build a back end.

insight. I thought this was a fantastic episode. Nice chatting with you guys. Thanks. Yeah. Thanks, Dylan.

and don't forget to subscribe in your podcast player speculation.

Yeah. Robo taxi.

you know, I have the Ioniq five, and there's there apparently is the Ioniq five robo taxis in Las Vegas that are the same, but I don't know anything about

Or drop a review if you like this show.

for a full archive of all of our shows,

And, Wes, if you haven't used this thing, it's a web Components

for video that make working in video feel like a way more modern modern thing

When you when you set out to to build, like, a web component video player, was it like was the intention that all of the other video players out there are just Not meeting the kind of modern needs that we have, or,

was it just like Mux needs something in this space? Let's let's get something out. Yeah.

the director of open source and community,

The last startup I had before this was building a product.

Mux and a not fan of most video players on the web. I was so excited to see you guys put this out.

That Video was a component of that. It was a SaaS product,

So I came into that kind of with no background as With video in particular, I just was building products on the web, building with a lot of the same stuff you guys talk about, and, I had to kinda learn about video for that start up. And that's how I started learning about the space, Learned about Mux.