Creator of Home Assistant: Web Components, Self Hosting and Home Hacking

And

like, home screens, like a like a old TV they hang on the warp, and it shows what's coming up. And I I'm always curious about, like, how do you how do you do that in a way that you give people full control, but it still looks good at the same time?

is actually,

light toggle on and off to to maps

cards, they don't. They just have their own and it's actually an analog phone. So But just browsing a website and hitting print, there's a lot of really good ideas that are built into Matter. For example, every device can be connected up to five smart homes at the same time software stack, everything.

simple

Shadow DOM, Wes really allows us to encapsulate the CSS because there's a lot of people out there that And so then, you know, it gets into Home Assistant through our Bluetooth proxies, and then boom, you have your own data.

by leveraging they based on IP six, it can work into Matter. So your washing machine can now go into Matter. It's also the first standard that is basically developed where HP Node wants to talk to home assistant.

this because this is before, like, this is really old as well. I know we were using nuclear. Js.

a, like, a Python application, and the UI to interact with that is

is to you can you can put a calendar in there. Like, literally anything you could possibly think of.

And

instead, I basically created, like, the idea of Redux, right, where you just have one object with properties and the properties only change customizable so you can people can define their own cards from their dashboard, and we just push the state in as like a property, and then they can render whatever card they want.

Right? So that's old.

everything from

they don't you know, they learn to program. Right? Like, they learn programming or web development as part of their Node assistant journey in building out a dashboard, is an open source smart home platform with a focus on local control and privacy.

is built in the browser. Right? which makes putting these things online

that includes know, or goo even worse, Google. And they eventually will just

Custom

to graphs of where your electric car

on the customizable Vercel spec. And I was like, what about combo box? And they're like, it's coming after this one.

I have

some matter oh, I have matter over threat blinds, Somebody had to do the work. That's what we do. Right? To bring all this stuff to Scott just our community, but any anyone that is just tinkering with this. You ever talk to anybody from either Safari or

Matter,

every time I, you know, I I get a lot of Vercel. Like, either people Bos send me devices to try and these kind of things. And every time I'm trying to automate my house or put things in my house, I see kind of bugs or things that can be improved, and then I just start coding and fixing that stuff. Right? I need to, like, scratch that itch. I do have a lot of stuff, though. I have I have a Reolink doorbell. I have Yield Logs. I have, put a different approach. And it's great. Like, everything is just possible. It's just the foundation of your smart home, really. Yeah. It's it's honestly awesome because

They just suck. Now try convincing, style what's it called? Style groups. We have reusable style templates, if, like, a value changes. Mhmm. And we just pass this object around. Wes just push it in. And if it doesn't

So true. So true. That's I actually did we had, the folks on that were working on the

the ideas are great. The execution is not so great yet.

guess what? Most Wi Fi routers just suck at that even though these standards have been around forever. Right? and you can actually over the air, so with lit, they changed this. They changed this to way to work more like you would do react. Like you said, it binds properties, and then at the end of the micro task, it rerenders.

which There's this, So we did build a bridge. We call it the external bus. And basically, it's an abstraction

I know. I think what browsers on mobile phones do great is that you hit a a native combo Bos. You just have a native UI. It's just not the drop down. It's just like, hey. Here's just all the results like you're used to from any list on your phone, So that we build that one too. That's that's amazing.

a Z Wave remote.

And so we build this thing we call the world's most private voice assistant, just go there. Our open API also like these things like Node RED, self hosted

So

Scott gonna happen. It's not gonna work. No. Long term, though. It's the same thing with browser standards as Wes. You know? Like, we've been talking about the customizable

an analog phone, you connect it to a voice over IP box, which then the voice over IP box that we use, Grandstream, has this feature called off hook auto dial. It means that the moment you pick it up, it dials a number. Now this number can be an IP address because it's voice over IP. So it calls Node assistant. Home assistant picks up, and now you're talking to a large language model. So you pick up, and then, like, my kids have these whole conversations again Node about, like, whatever.

which is I know this is forever. Right? But for example, it it relies on IPv6 and Wes heavily.

like, everyone in the world to buy a new Wi Fi router because of your fancy new smart home standard.

what does your smart home look like? Like, are you are you nuts? Like, do you have everything?

No. So the thing is

And now in the last release, we did, like, last week, Node assistant can call back. So for example, sometimes the laundry is done, but you don't wanna send notifications to phones because you don't know who's there.

select drop down for for many, many years, and we're almost there. Yeah. And, like like, part of me wants it right now, but part of me wants them to get it right so that Yeah. The the next twenty years of web development have have amazing primitive.

in React. You can use them

Yes. And they created Web Awesome, and it was like a Kickstarter. And so they are making open source components with, like, a paid component to it. We're we're gonna use the open source parts. We cannot use paid stuff because we're distributing everything open source.

You've done it, and this is this is a testament to it. And it's open source, so you can go in and see how all of this is written and yeah. It's super cool. Yeah. I'm curious if it's, like, one of the the

because And then Google released material design with Polymer. Right? Polymer Vercel, like, OG web components before they were part of browsers and web standards.

people update their apps in different, like, frequencies. Right? So maybe They don't like fun stuff that they cannot control. Mhmm. I think well, I mean, And they do wild stuff. They do very wild stuff. But being able to just say, okay. Write whatever CSS you want. Like, make the background of every div red. Great. Because in Shadow DOM, it doesn't touch the rest of the application.

Plex has been able to solve, but they do it in a weird way where But the for the device to test out the Wi Fi username and password, they have to tear down their access points so you don't get feedback if it fails. And we're like, Bluetooth can just do this. Like, there's all every native application of every product out there has it. It's just no open standard.

So that's gonna be

I always wonder how they do that. There's there's a couple

maintainers need to live and eat as well. And if you don't pay them and somebody else is paying them for something else, they'll they're gonna do something else. Right? At some point, like, there's not enough attention or burnout or they get kids or they move to a different city and get

certain features are being offered by the Wes view. So if you launch the UI basically, we don't wanna build our dashboards in native because It it feels a little wonky, to be honest. Like, there's definitely, like you know, we access this idea of portals. Right? Where, like, very deep in, you just create a dialogue and boom, like, somewhere else in the page it pops up. We've recreated that by just creating an element and adding it to the body. You know? We just that's our portal. And then we just render in there. And at the end of the day, we have at the end of the dialogue, we remove that from the from the element.

web components. Right? But then Google just you know, they are kind of reorganizing. And let and Angular kind of I know what they're doing, but like the the material design components

it's going to be a whole big set of UI components

are lit, but then they they're betting a lot on Angular because they have this other internal web framework that's gotta be merged into Angular or something complicated. And they're just shuffling all these resource around. And I think it's time that we, you know, we bet on open source for these kind of things Wes with the company behind it, though, because I think that is important so that the longevity The a lot of stuff from LitElement is now in the standards. Right? Like, so there's

if the app cannot connect to the UI, Really? Yeah.

All the cool stuff the web has to offer, at least part of it, is limited. Right? So we've been building those things natively. So QR code scanner, we have natively. We have, our voice assistant natively now as Wes. Because, yeah, that way it works. And look, I've been trying, like, talking I think there's a somewhere internal proposal at Mozilla about, like, custom built operating system with the smallest amount of footprint just to run Node, really. That's the key. It has two partitions. So if you do an operating system upgrade and it fails, it will automatically roll back. We run everything in Docker. So we have the supervisor. The supervisor is in Python application that manages the system. So it talks to the operating system and say, hey. Can you update this? Can you start this? Can you stop that? There's a I know there's a microphone plugged in. It will forward the events to Node Assistant. Then Home Assistant runs as a container.

somebody updates their Android apps once every six months or

somebody updates every day. So we need to support kind of, like, incrementally all these features.

how to make this possible. But, you know, Google and Apple, they never care about the secure context as it's called in browsers to allow that for local because it's just not a problem they have. Right? It's a problem that we have. It's a problem that CarPlay, And so that means you don't have microphone, you don't have camera.

that are built in in Wes components, meaning that meaning that you can use them

just an open source project

on our core, not on our front end. That's a actually don't know the exact number there. But so anytime we do something, we're often the biggest. Yeah. So, like, the biggest cell phones and

Everybody's really excited about this project because and it just works so well. My last question I have for you is, Yeah. Yeah. I I think that would would be a a good use case. Maybe we'll explain real quick, Lee, for for anyone listening JS, like, Node Assistant is

it will instead of looking locally in the local storage for the refresh token and the access token, it will reach out to the mobile app because the mobile app has its own access and refresh token, and that one will then take on the whole auth lifecycle. So whenever the access token expires, the native app will make the request to get a refresh token. And when we had this communication in place, we then now have the mobile app expose certain things in, like, a configuration object like, hey. What else are you capable of? Because and we we're at the same level.

right now Wes have a big community building custom cards. And so if you have a cool custom card in your dashboard, you would open the app on your phone. It's like, it doesn't work. Like, that's lame.

different hobbies. Anything can happen. Right? So then see, okay, this dialogue is in this file. So we dynamically import the file, because we don't put everything in the main bundle, of course. Then we instantiate this element. We add it to the DOM. Web components are nice in that you can just add things to the DOM before they are loaded. And the moment they're loaded, they just pop up. Right? They instantiate like, they read the HTML. They parse it. And so that's how our dialogues work. That's why sometimes if it's like, if the stars don't line up and Homosuggest, you would click a button and you can wait, like, a second, it generally should be cached by the the service worker, of course. But I want my combo boxes. Like, that's been the biggest pain point. Like, a combo box on mobile Wes you can type, the keyboard pops up, then you start dragging the results, the keyboard goes down, but then your touch point hits outside of the list item and your list closes again.

I mean, I think a lot of things that we do, we like, we are the most active open source project in the world, and we are huge. Right? Like, 21,000 people contributed in the last year And

We have the QR code scanner. So you wanna that you have locally, and it points at your local IP. So only internally you can go to it, but it still requires Yeah. I mean, this is for us the reason, like, why do we build ESP Wes tools? Why did we, like, make it as a standalone project and not build it into ESPHome? Is that exactly this. There's a lot of enthusiasts, and they

not Jellyfin. What is this? Streamio, I've been has something similar to that. But, yeah, I'm always curious how they are able to do that because

And the the the Node is super readable. Like, the I I could I see it as almost like a learning resource. Like, how can you so many people talking about web components Sanity, like, well, it doesn't scale or it doesn't work when you have such a large application.

they have a public DNS that with a certificate the state management that was the inspiration for Redux.

biggest implementations of, like, like, open source project of using web components. You know? Everyone always points to, like, well, YouTube and Google uses it, but that's not that's not open source.

The native app, is that

Now the reason we always recommend to run Node Assistant operating system is because not everything in the world is written in Python, sadly. So the Z Wave driver Thankfully. The Z Wave driver is written in TypeScript, for example. So but it's the best Node, and it's the one that we're funding now. So if people wanna run Z Wave, which is a standard for the Scott home, you have like a radio dongle, you plug into your smart home hub, into Home Assistant,

what are you gonna do with it? And at some point, you learn about home assistant, and then, oh, I'm gonna install it. To go back to our installation method, so we have the operating system, which you could run as a virtual machine, which is what the Wes is doing. If you installed on a Raspberry Pi like CJ did, then it's just that. Right? Like, it just up it can update itself everything to the UI. There's no command line.

Actually, you know, fifty fifty one or 52% of our community runs on Raspberry Pi devices. So Really? Yeah. Because it's just a it's such a gateway drug. Right? Like, oh, you have this Raspberry Pi. A lot of people buy these computers because it's like, oh, this is 35, 40 5, 50 5. They're getting slightly more expensive nowadays. But you buy this device, they sit in a drawer, like, yeah. And then

because WyzeCams are super cheap, but it's because they want you to

Now if you don't want any of that, you can just run Node Assistant container individually.

And instead of me running that Docker container manually,

We we always say, like, computers are so affordable today. Like, get, like, a Raspberry Pi. Get, like or get a place that runs VMs, run Node assistant. And then the last way you can run home assistant is just as Python application. You can even today, just pnpm install home Assistant. Boom.

And it just works. I've never successfully pip installed anything,

and then I can pipe those into my Node Assistant.

USB stick, connect Node Assistant to it over WebSocket connection, and now Home Assistant can speak Z Wave. Now this is great, but this in Node Assistant OS because we control the whole stack, because we control the docker host, we can do all this magic. If you run Node Assistant in a container, which is totally valid way to run it, you would have to set up Z Wave JS yourself in your own container, and then you have to update it yourself.

on that.

No. But that that was one of the use cases that I had too because I have, like, all these devices from various things. And so I I set up Node Assistant on a Raspberry Pi. I use the the Home OS. Nice. I would recommend, like, if you're a dev that bought a Raspberry Pi and didn't know what to do with it, it's just sitting in a drawer, you can throw Home Assistant on it and start connecting devices go read the source code. Like, I I just started diving in this morning, and I've never first of all, I've never seen so many web components.

when you're running Home Assistant OS, which is like a like a VM, it can then go and and start many other Docker containers for you, that plug in being one of them. Yeah. No. Exactly. Well, I've I've I've tried setting that up too, Wes. And I think if you're on your local network, it actually sends the the video signal over local. But if you're accessing remotely, it goes through WISE servers. Oh, okay. Yeah. Oh, that's good. So I'm not not racking up their server bill.

There is this

to it. And if you want to see all of the errors in your application,

you'll want to check out Sentry at century.i0/syntax.

So I installed this, like,

subscribe to their their monthly plan. Right? That's that's the the thing. I'm I'm sure you've got opinions

So I was like, well, screw that. I'm just gonna stream these suckers twenty four seven.

Wyze Bridge, which essentially just streams twenty four seven, I think, off of their Amazon server. I don't wanna see the bill. And then it it pipes it into, like, a RTSP stream,

Again, we've been using this tool for a long time, and it totally rules. Alright.

You don't want a production application out there that, well, you have no visibility into in case something is blowing up, and you might not even know it. So head on to century.i0/syntax.

So supervised, Plus the app is, of course, used on the desktop, so we have a fallback anyway. But for example, features that we have is open the app settings. So we didn't wanna have a toolbar to open the app settings. So

but also people can host themselves is is certainly a challenge.

projects everywhere, and and making something that is

like, I've I've run it for probably five or six years now.

self hosted

even further behind. Like, I'm running it as a VM, not just as a Docker container, as an actual VM. And it's awesome because, like, I have

And, also, Home Assistant is a I I know CJ is gonna be excited about this as well. It's like Home Assistant JS probably one of the one of the largest

I'm currently running it on the the Node Assistant operating system, which is, in Angular, but or you can also just use them straight away in in some straight up HTML.

And I think that was a

And these kind of things is like now having these projects secured, we still Namukast is the commercial partner and still funding development And this piece of state management actually has survived, like, I don't know, eight years, which is kinda wild, And so by the end of this year last year, we actually released this hardware. It's the the voice preview edition, which is just a tiny box that you can use to control your smart home. And inside Node assistant, you can either I mean, that's really, like you don't see that in other in other platforms. Right? Like, even Node Assistant is not that stable.

Home Assistant runs on web components.

but we also allow it to send it to Apple Node, send it to Google Home, to Amazon. If you like those ecosystems, Scott natively Android or iOS are the Android Auto, hacky in between path, which is called home assistant supervise, which I do not recommend. And we see the users also that are using this lowering and lowering. And this is where you basically run the supervisor history, Vercel, So that is just in if you run Docker.

UI here and and turn on my recording scene.

I can't architect it as as what I want. You Node? And the Apple stuff, you gotta, like, be in Apple's good blessings to ESLint that environment, but, like, you can connect anything to anything on Node Assistant. And I just have, and it's just it's rock solid too, man. Like, I've been running it for forever,

it's not a use case of Google Home to be like studio automation, but, like, it's kind of these things we end up doing all the time, like, as content creators, for example. Mhmm. Great. With Home Assistant, you can do it. I have dipped into it, like Node not dipped into it. I've been running it quite a bit. And even, like like, all the lights behind me, let me see if I can Firefox have said Wes don't wanna do this. And their concern is that if you give this permission by accident, which can happen,

what happens there is that there's just because anyone can build anything. Right? We don't limit. Like, anyone can build any integration. We don't need partnerships. So so for example, you know, I have Elgato Lights. Now And he has whole discussions with GPT before about, like, Pokemon or Minecraft or the fact that Shoelace or Web Awesome has, like, a company behind it that you can give money, right, to keep maintaining this stuff Mhmm. I think JS awesome.

Which one I or do what do I have here? It's so funny because I almost never use the Node assistant UI.

and it just works. I think do something bad.

yeah.

somebody had Elgato Lights and built an integration. And now I have a local Elgato integration to control my lights. So for example, I can have the lights turn on when I start streaming.

like, one step and I was I was flashing it. And I was amazed at how far easy,

I walk into my office at 9AM, and they're all on. And I walk out at 05:00, and they they turn themselves on and and lamp o'clock. And it's it's pretty cool. But and as much of, like, a home automation geek as as I am, I also today, we wanna talk about, like, a couple things. Specifically,

automation engine. It just fully plugs into Home Assistant, can do anything you can do inside Home Assistant automation, I think we should see the benefits of Matter in, like, five years,

It's just automated.

go to my Home Assistant I've got so the anyone watching the video, I've got a Wes 32 here. And I was talking to Scott. I was like I was like, you gotta get one of these, man. I've I've got so many little projects running on ESP 30 twos. And I was like, and you can flash it in the freaking browser.

But in it are just modern web components with the newer frameworks and all these things, and it just keeps working. The beautiful thing about

but it works. And today, custom cards from back in the days actually still work. Like, if you had a card to with a I think it's called external auth equals one, The the the I just I've been holding up a picture frame, for the people that are listening only. This is a picture frame that fits, Nintendo Switch games. Oh, wow. And so I use this for my son. He puts in his games, and we hang him on his wall so that he has, like, a whole, like, Mario, Minecraft like, wall. Yeah. That's great. Oh, I were really quickly, I wanna shout out Andrew I don't know how to say it. Schlemnullen? But we are, we still have some Polymer. Like, this stuff, even though it's not maintained, it still works. And it's still support us. Right? So you can do that by subscribe to Home Assistant cloud or getting, like, a Home Assistant green on Amazon, getting a voice PE. That's the the voice preview boxes that we have. All these sales of those things support the development of Home Assistant, ESP Node, ESP Wes tools,

those cards can still function.

render a card or a switch or whatever, as long as the things you're into like, the if the services, they still work as well. So all that stuff remains stable, send it to AI, have, like, chat GPT or GPT four control your home, I donated Node assistant to it. We donated ESP Node to it. These are the two user facing projects we have. And ESP Home is a

I see all web components all the way down. You know? Home assistant tag, home assistant main tag, home assistant drawer, partial panel resolver,

because it interacts at the browser level with DOM elements. So the sidebar element in Node system with gestures, that's all still Polymer.

I think this is great. Like, I I saw your blog post back from 2016

I don't know, eight six, eight years ago, something like that, and it's Yeah. I I hit that. I'll tell my example is I was running a I have all these WyzeCams,

So what are you using now? If I inspect element on my Home Assistant dashboard, I see a whole bunch of, like, pretty much I'm very glad that you will not one day wake up and turns out Xiaomi or someone has bought the all of Node Assistant,

still a % valid.

about how you chose Polymer and web components. And the fact that you made a technical decision, well, like, nine years ago at this point, and it's lasted, which is hard to say about like, because, I mean, if if you would have chosen React, you would have had, like Three or three. And you wanted to say relevant and update. You'd have to you'd have to, like, rewrite three times. So that's awesome to hear that it that it's working. That's something that I actually already realized when we were working at React, at a company I was working, is that a new version would come out. And back then, we were making a lot all these changes. Right? Because Wes twenty fifteen was just coined and, like, classes and everything was new and everything had to be old. Like, you have browser refine, all that stuff. It's way less settled back then than it is now. But then a new component which would require a new React, and you would have to upgrade everything and kind of go along. And I realized that with today, with web components, we don't have that problem. Like, we still have some Polymer in Node Assistant. Right? Like, we've been there's been a new material

browser standards is you they're not gonna go away.

like you said, is

no HTML elements. You know? I see a a web view? Is it native with bridges? Or how does that work? It's all web view. So we Wes

it it self contains the styles, meaning that there's no way for CSS that's written inside of a shadow DOM to leak outside and accidentally goof something up, or you share a class name between two elements.

I feel that way even in my courses as Wes. So if I do a CSS I have a Flexbox course from,

a couple of months ago, and I hadn't updated the thing in in two years. I took a backup. I installed a brand new version and reinstated it. Everything just worked. And I was like, oh, man. Like Nice. That's beautiful.

It's amazing that, like, you build something in in browser tech, you know that it will it will stand the test of time. And that that's really important because, like Yeah. I I logged in to my Node assistant

Wes component package Yeah. So this yeah. Yeah. I have I have a few here as Wes, ESP 30 twos. Yeah. So we don't just run Node Assistant. Right? So we are last year, I created a foundation, OpenHome Foundation.

And the other way around too. Right? The other way around too. So CSS from outside doesn't leak into the Shadow DOM except for CSS properties. Oh, yes. It That's Is that ever a problem? Actually, I'll I'll ask that question now. Is that ever a problem where thanks to my Apple Node GitHub that I use a strap border router works 50% of the time. That's part of the whole threat matter stuff that isn't fully working yet. Oh, and the one thing that I like a lot is that, you know, we've been doing a lot of stuff with voice, with home assistant.

which they also now put in maintenance mode. So I'm we're not gonna bet on Google UI toolkits anymore.

and then there's shadow shadow roots all the way down, which anyone who's listening, web components a shadow DOM,

do, like, a low level connect to to something like this, that's that's a missed opportunity, I think.

with this website. If you give notification permission, you get annoying notification, you disable it. If you give away little world shadow DOM. They can mess with it. They can write all the CSS, the JavaScript.

and, like, they are he's like, well, internally, the road map, like, maybe it's an option. There's actually I don't know. Can I just read out a URL? Because he just asked me literally today. He's like, if you all just go to this page and tell Firefox that you like Okay. Wanting to have this web serial, I I it is too long of a URL. But I'm gonna try Put it in the show notes? Can we put it in the show we'll put it in the show notes. Yeah. We'll we'll put it in the show notes. But, anyway, it's just like place where you can tell Mozilla called Mozilla Connect, where they connect with the community and where you can tell them, but they haven't done so far.

They are growing up on on iPads.

much easier.

Firefox about the Wes serial and web Bluetooth APIs? Because I know they're still being developed for Bluetooth LE in Google.

space has become. Both Node assistant as well as this whole, like, microcontroller Awesome. Are are you Dutch? Yeah.

then sends back a URL to follow-up for the user to finish onboarding, which can be the IP address that it just received, for example. And then you're on the web server and you get started. And all of a sudden,

now it's just two minutes. Right? You plug it in, installation, implements Wes serial and web Bluetooth? Right? Safari and voices, better speech recognition, And e s p Node just say, hey. This has a microphone and speaker. Make it a voice assistant. And now it's a voice assistant for home assistant. And it has wake word detection, and you can play music on it. It's not loud.

So

you don't

I have been talking to somebody at Firefox,

I would love for it because you think about, like, a new generation of, like, makers and stuff.

That's that's amazing. Yeah. I, I recently did a project where I controlled my Roomba the I don't know. How would you can you give us the pitch of what Home Assistant is and and who we are? I was doing I was doing another one. I forgot what it was, but it was, like, this Wi Fi improv. And I was like, that was a project. I was like, oh, I was like, oh, this is a a standard project,

web USB, web Bluetooth, web serial, So the second step is that we created this open protocol called improv Wi Fi, which, by the way, it works over web Bluetooth or web Vercel, both. I have we build SDKs for both. And then we just say, which Wi Fi access ESLint do you see? It sends it back over web serial. We show a UI. Users enter the password.

know what has happened to you. If you give away your location permission by accident,

okay, I've accidentally shared my location

Wi Fi. You're on a web server running on a $5 device. Like, you don't you don't even have to, like, download and install, like, an installer and click through it. It's you literally visit a website, has it's crazy. Like, I've got Google Homes in our house for for the voice stuff for the exact reasons that you just said.

they build stuff for other enthusiasts. Right? And we wanna make sure that we unlock all these cool features like WLED. Right? How easy can we make it? So how can these web based tools, Scott everybody can use WLED. Node everybody can install this Node or all these other crazy projects Shoelace got bought by the awesome Wes Awesome group of it's called I know the company behind Font Awesome. Web Awesome.

as babies. Like, just don't press these buttons, and that's why you end up with, like, tablets where you cannot program on. Right? Like, you can only go to the App Store.

It sends the password over to the device. Device connects at my work, while I was working, we were very early adopter of React. I went to the first React conference. This was back when you had React create class, right, before the components even, before the hooks. Right? Like, this is way back. Oh, man. Yeah. And I was like, I wanna do something else at home. Right? And I was like, homosysm was my playground.

is filled with enthusiasts. You know? It's not it's not a lot of full time developers and and and nerds. It's just enthusiasts clicking stuff together and making it work, and it's amazing.

I can I can see their perspective, but I also don't like it JS that by the way, I don't know if we said, but maybe two steps back, only Chromium

but you don't know what. Like, yeah, you can maybe see the log of what was written, but it's just bits and bytes. Right? Like, there's no you need to be a programmer to understand that. And that's, like, their angle saying that's why we don't want

like, robotics microcontroller

32,

because that just that last bit, we're like, okay. We have to create a new standard, create a bunch of new libraries.

they both said, which

You know? And and you could probably load a custom firmware on there that would

the old way used to be that you have an access ESLint, and then your phone connects to the access point. And through DNS hijacking, they serve you a place to enter the Wi Fi username and password.

Safari,

projects, and,

to the network I think the the rollout of Meta is all these dependencies, but everything get imported on the fly only as you use it. So if you, for example, Nuclear. Js was the

And if they're not able to

the thing that tomorrow,

the whole,

and it pops up, and you can flash your thing. I've I've got lots of WLED

And then we're like, oh, we can do normal Vercel, and then we did that.

I think, is a hard no.

and how accessible But then you also wanna allow people to skin it and theme it and have, like, a, like, a flexibility and control over making it. Like, there's this there's a a subreddit that's just dedicated to people who make, like,

we realized, like, okay. From Python installation and super difficult,

that not secure

the watches, kind of rough in the beginning, but,

if you want to install

the ESLint. It's it's such and their application layer of, like, what is a light together. So you couldn't use it separately. That meant that, for example, your washing machine using Wi Fi

because the parts that are UI facing are all Wes view. Right? So the things that are not cool anymore because of, like, some stuff they recently did. But, like, I got this Bamboo three d printer, and it just it's so easy to three d print. Like, I used to have, like, other three d printers, and it is really a hobby.

So there's four options. The the easiest one, home assistant operating system that is using Buildroot. It JS a And as we were building this, all of a sudden, OpenAI drops whisper. Right? And all of a sudden, TPG 3.5 comes out, and all of a sudden, meta drops llama. And just there's this thing called reactive element, which is literally just a custom element that can respond to properties and has some life cycles like did update, So disclosure, just I'm on the Z Wave alliance board.

bang is is kind of scary for people, and and it kinda sucks that Oh, that's great. That's that's really impressive. Do you do you have to be really picky about what gets added, both in terms of, like, bundle size, but also in terms of, like like, memory consumption? Because you're running on these

Home Assistant on your

JS it it has a MDNS on my local network,

about the self hosting aspect of of everything. Because

maintenance because we get under control because we would get all these bugs and, like, oh, this is a bug broken if I run Node system Docker in, like, a jail on FreeBSD or something. We're like, yeah. But we don't really care about it.

but it's not a secure instance. Local host is, and you can run, like, a camera on all those limited APIs, but not not on the other ones. Are you doing anything for code sharing between Android and iOS? Because, like, typically, if I were building something like this, I'd use, like, capacitor, but I'm seeing there's, like, two separate repos for Android and iOS. Are you using anything? Node so we we decided not to do it

We have an audio container, a Wes container. So we have some bit of, like, helper functions around it, all running in separate Docker containers.

Because events are very powerful as well. Oh, so you have like a like a pub sub or or what? So we have, for example, a start dialogue ESLint. So you can register like, every dialogue is really just a custom element. Right? Like a web component. So we have very serious. Right? They even going as far as, like, I remember the whole discussion about, like, flattening an array.

at the root, we have a bunch of listeners like a dialogue manager. And so you can Sanity, open this dialogue, and it will

sometimes you do want CSS to make its way into the card?

But we also use a lot of events to kind of hack around that. Mhmm.

And so We realized that the first step is so hard because you need to install the software on this device. And so what people had to do

Sanity, so that made this possible.

I'll I'll link link up his, his Twitter, but he does a lot of hardware hacking as well.

trying to

figure out how to communicate with a old VoIP box so you can you can connect to a like, a old school telephone, so much gear out there, and there I don't think there's one I have some stuff. I have Google Home and, like, not everything integrates with it, and, like, it's it's it doesn't

Awesome. Alright. Peace.

a sick pick for us today? I have the sick pick, but it's kind of And from there, you can do whatever you want. Right? We believe that

Yeah. That's cool. Do you have a a printer, CJ?

But you can send them three d models, and they'll print them for you. But I was like, man, I should probably just get a three d printer. Yeah. That's great.

But recently, I wanted to start building

And he also has been

I couldn't believe when we met you at the conference, and I was like, biggest homosysic fan ever. And, and then I found out you're using web components, and I was like, my whole worlds are overlapping here. So I appreciate you giving us a hour of your time to to come chat. That was really fun.

and there's just a thing. Right? Like, I'd make this, like, Mario and this Node awesome. You know? Node. And it's like it just it just exists. And I just hit print. Like, I'm not I tried some tree modeling. I just don't have the patience for it, the measurement, all that stuff, the iterations.

And so I was looking into, like, you can there are services online. You can oh, nice.

That's amazing.

We got that. So Oh. That one is like the the cherry on the top of my smart home. That's great. I love that. Alright. So the last two sections of the podcast we have are a sick pick and a shameless plug. Sick pick is where you you pick a product or an item or a service or a movie or anything that you you love. Doesn't have to be programming related or not. And then a shameless plug is, of course, anything you'd like to plug. Do you have a

all that stuff. That's awesome.

it's my birthday next week, and I my wife is like, you gotta get a three d printer. And I've been looking at I've got the tab open right now for that. I I think I'm gonna go with that one. It's very nice. If you get the they have this thing called AMS, which I didn't know I wanted, but it's really cool. It's like an automatic material system, and so it can change between the different plastics that it use. So you could do four colors, but you can actually chain them. So now you can go up to 16 colors that it can automatically mix The state, that one has to stay fixed. Right? So the way we pass around state and Home Assistant

The with the latest bamboo that I have, it's like an x one carbon.

and I thought that was really interesting. Alright. So shameless plug. What would you like to plug away to our audience?

What if you could make a sound in a room and somebody has to pick up to, you know, show that they've received a notification to get it? Yeah. And so, actually, improv Wi Fi was interesting because we initially started doing it with Bluetooth. Yeah. Because it. I come from the world of, like, just give people treat them like adults. Right? Like, we always try to treat everybody as, like, with these So Wes just build one.

better camera support through WebRTC routing. Now last year, I actually took Node Assistant, created a foundation. I donated Home Assistant to it. I donated ESPHome to it. It means that these projects, they can never be bought. They can never be sold.

you know, we raise money. That money, you get a remote connection, remote access into Node Assistant, better voice assistant, the matter standard

Like, make so I want I have, like, old iPads. I don't use anymore, old tablets, but I wanna make it look like like a CRT monitor. So I wanna three d print, like, the bezel.

No. I'm in the same boat. Like, I've I'd for like, I'm being a hobbyist and maker, like, this three printers are the one thing I haven't explored.

Wow. So, I mean, if you Yarn using Home Assistant or you are going to use Home Assistant,

you

of the 52 people that work today on Node Assistant and the whole broader ecosystem.

dragon skeletons small pieces that code sharing there doesn't really help that much. I see. And we try to push all the logic into Node Assistant. Right? That's where we you know, a QR code is just a bunch of bytes and let the back end figure out what to do with it, for example. That makes sense. Let's talk

shut her down Yeah. In six months from Node. So I appreciate that. Cool. Alright. Well, thank you so much for coming on. This was, over, Wes But that's that's gonna be our next iteration that we are gonna rewrite a bunch of our stuff. That's great. Yeah. We had, it's Connor Rogers and Corey Levisco. We had them on the podcast, Syntax. Defam forward slash

Yeah. Happy to be here. It was great.

custom cases for old screens.

into your prints.

if it's not, then, like, it can wait a second, but then the dialogue pops up. And it's because of this system where we just add the dialogue and we import it at the same time. That's great. And I'm gonna ask you about the service worker in just a second, but I'm seeing little comments where it looks like there's lit in in here. Is that what you're using for the web components now?

Node thing I wanna mention is for anybody listening,

reroute, and then we are deeper in the stack. It's all lit element.

and then they came up with a Polymer had a lot of things that didn't make it into the web component standard,

Yeah. Yeah. Yeah. So we used to use Polymer, quite of unstable.

and it also relied a lot instead of where React would, like, batch changes, Polymer would not batch changes. So if you make set one property first, it would re render for that, and then you set the next property, it will re render for that, which is craziness. Like, it's all of a sudden, every race condition had to be caught because if this order was different change, like, the actual data you're interested, you don't rerender. Otherwise, you render.

obviously, set of DOM custom element. And then the way lit kind of they have the lit renderer, and then they have the lit elements.

And so we use LitElement.

And lit elements actually

Like, those events, they just don't really add up to a lot of CPU. Now if you start doing video streaming, video transcoding, these kind of things, it starts to add up, but the normal Yeah. The the trick there really is

cameras can be normal, but, like, if you don't have the transcode, which is the normal operation, then there's just not a lot of CPU. Oh, man.

And And instead of using objects for Redux ESLint like we're doing just objects, nuclear was using immutable JS. Okay. So you had to pull in immutable JS, but immutable JS Wes, like, maintained by this guy at Facebook. And he left Facebook, and immutable JS kind of died, and nuclear JS died. And then I kinda realized, like, it's too much magic. Like, if you do all this magic, if you wanna have people create custom cards, you can't have them pull in reducers or whatever, selectors, all that stuff. So

And what about all the AI and and voice assistant stuff? You guys are in that space. Right? Oh, yes. I love it. It's so cool. Two years ago, and just really before the AI hype started, we realized that if you look at, like, the big smart home platforms like our competition, it will be Google Home, Apple Home, Amazon Alexa, and what they do better than us. Right? Because they they don't do such well on dashboards or integrations or history or automations, it will show a button like, hey. Do you wanna open the native app settings and do some I see. Checking with your Vercel? But we inject inside the if you go to your settings page at the bottom somewhere, there's like app companion. You click on it, a native page opens with settings.

it has four gigabytes of RAM. We're just like, we're just gonna put four gigs in. Nobody will ever have issues with RAM. CPU usage is like 5%. Like, your smart home, not that much is happening. Right? We all think we're super cool, but, like, you know, you get, like, a temperature report is, like, every five or ten seconds. You know? How how often do you turn on your light bulb? Right? Like, eight times a day maybe? So I'm I think I wanna get this. This seems awesome. This is I mean, my kids love it. So my I have one here that is, like, looks like Pokemon. Like, I just had a somebody three d printed this with, like, all the case files. Everything is open source about this. The hardware, the the QR code scanner just works a lot better natively, speech for home control fully local.

Beautiful. I'm Dutch as well. I'm obviously Canadian, but, Dutch heritage. So always love love when you have it on. So

creator of Home Assistant, which is like, I run mine at homeassistant.local And then, like like, half an hour later, I was just, like, flashing it in the browser, and I was like I was like I was like, I'll look into this project a little bit more. And it was you. You made this project.

I'm sorry. I didn't ask you how you pronounce it. How do you pronounce your name? Paulus Schoutzen.

Paulus is the

Welcome to Syntax. And,

per per device, I'm really excited for this project. And Scott's Scott's been working with it as well. Yeah. And it's it's gonna be neat. I think that we have been relying a lot on the Google users that buy Vercel, that collect data in your house, you own that data. You can do with it what you want. So from Node Assistant, you can have dashboarding, automation, ZigBee lights. I have

BT Node, which is a custom Bluetooth protocol that we build to share IoT data.

all the big tech got together. They were like, we need a standard that needs to be good. Here's all the things we learned, and they contributed everything. Right? So amazing. I have two young kids, and they go to birthday parties. And I'm like, the coolest dad. Right? Like, I have this But what we realized when we were building this is that the first step to get once you have it up and running, Home Assistant Sanity ESPN can do over the air updates. It's awesome.

with their own encrypted connection. So of Python packages got so fast that we save hundred eighty five days a month with Node Assistant CI just getting it. Like he has all these questions about games, and then he's reading the Mario encyclopedia, and then he's, like, today, I wouldn't wanna pick any other technology for Node assistant because of what it enables us.

That's crazy. What? Can I ask your thoughts on this whole matter as well? Because there's there's been, like, Wes? Like, two standards. There's ZigBee and Z Wave.

adding more device types. And Wes the specification is released, then, of course, smart home platforms have to implement it. And then devices have to be built. Device have to be certified. This is hardware. This moves very slow. Right? Where I think in the JavaScript world, the way we work is that

that works over Wi Fi. So ZigBee and Z Wave both connected their radio standard the Internet to exist and to be accessible.

Just I don't I don't think it matters, but, like, just disclosure. So there's

couldn't be integrated with a standard.

a little bit overhyped, I would say. I,

through web Bluetooth,

and they will work with both. Now this is, by the way, theory. It's temperature humidity sensors from Xiaomi. They cost, like, $56

while we're tinkering on a standard, somebody builds a transpiler into, like, into our tooling so we can play with the standard and everybody is already using it, then it gets standardized. And once it's in browsers, in three years, we put it in the modern build of our application.

Matter actually solves this by focusing on IP. Right? So anything that has an IP, That's ridiculous.

I think that people expect everything from it on day one, but they are slowly will update, update it. And we actually use that part in some of our elements or we use lit elements if we just actually have to render them. But if we do a lot of magic, like our we made our own page router, which is the partial panel resolver, for example, that you saw. It it has, like, a definition of, like, these are the panels that we know. We don't know the internal routing. Every every element that is can contain multiple pages is responsible for its own routing. But at the partial panel resolver, again, really like a dialogue. We just import the file. We create the custom elements. We can cache custom elements if you wanna maintain the state in the DOM. Right? Like, which page was selected or these kind of things. Yeah. Basically, the underlying layer of lit reactive element helps us just manage the life cycle of the elements, wait. But this is Vercel, so we can just use Wes serial.

Z Wave. Yeah.

in a way is great because it's the first smart home standard Yeah. So I'm Paulus Schoutze. Home Assistant Nabokaze never had any investors. It was always fully funded by the And environment around it changes, like the CSS that's being used or the components that you rely on, then you end up, like, breaking a card. But if a card is fully encapsulated with its own stuff, which, yes, it brings in more CSS,

if you use Apple Home and you're like, I wanna try Node Assistant, you don't actually have to tear down your Apple Home. You can use Home Assistant in parallel to it with your meta devices, Yeah. I I think browser vendors, they are the people that make the standards. They take backwards compatibility things can happen. Right? Because these are real world APIs that can do cool stuff, I just have this app on my phone, and I see something cool, and I hit ESLint. And then two, three hours later, I go to my office, there's parts of HomeKit in there. Apple just said you can use anything from HomeKit. We don't care. Google said you can use anything from Google Node, and Amazon said you can use anything from Amazon Alexa. And so And so for example,

everybody was transitioning with Babel to, like, e s six or, like, five to six JS it was still called back then. And, like, started using classes in Polymer, couldn't do that yet, etcetera. And so it Wes

So the reason I chose web components initially, right, so not the reason why I stuck with it, but the reason I chose it is because my son, you know, the press the middle button, he can talk to it. And it has conversation support in that, like, if if you follow-up, it knows the context of the previous things you've said because we just Scott the whole history to the large language model.

And I don't know. Do you want me to just go down what what it enables? Like

And I was like, oh, yeah. Let me try this. I like material design. And so I added that to Node Assistant, and this was this was rough. Like, this was with non standardized HTML imports. So they had to they had their own bundlers that didn't fully work. They whenever user on a Debian Linux installation, and then it kind of basically, you have full access to the host operating system to SSH to run other applications and all that stuff, and the home system does its thing. And eventually, things will always Scott conflict. Right? Like, people install Portainer to do kind of cool stuff with Docker, but then Node system expects exclusive Docker access and Portainer starts messing with Node system containers and everybody's angry at each other.

And for these different chips, here are the builds. And so what it does is that it connects it also guarantees kind of stability. And so the cards inside Node assistant that are built in, they share a lot of the stuff together.

And I Impossible.

It's not the best voice capture, but it works. Right? And this is the same the way HP Node is just you write a YAML file, it generates software, it installs it on the device, the biggest web component user. But it also means that the technologies we choose, they have to be rock solid. Right? It means that we are we can't afford a rewrite with the amount of people that work on the project. You can't, like, do something. So for example, we are considering adopting shoelace.

and then it, configures the Wi Fi. It pops up on the Internet on your local network, I mean. Home Assistant finds it. ESLint integrated.

Then we automatically select the right build. We install that on the microcontroller.

this is a microcontroller that has a microphone speaker. People Yarn listening. I'm holding up an, an m five Stack Atom Ego. It's a $13 especially this is this is actually a big problem that we run into. If you locally go to a server, you don't have HPS.

was install Python.

So I work with Espressif, which is the company behind the ESP chips. And they already had, like, kind of a prototype Wes we build it out, but they just had, like, a raw library. And so I built this tool called ESP Wes tools, which you basically define a manifest saying, hey. This is, for example, the project WLED, which is one of the projects that uses our tool.

device with a microphone and a speaker, which is crazy cheap, and it has a case around it, USB c powered. Wow.

Impossible. Yeah. You can't you can't go there.

So then I realized, like, Shadow DOM. With Node Assistant, we have made sure every layer is is guaranteed, like,

project to use YAML to generate software to run on these microcontrollers.

Oh, you should so of

depending on how many device you have and how many different integrations get pulled in, you need more memory. We sell Node Assistant green, which is just a because people were like, oh, VM. I don't know what it is. Raspberry Pi. I don't wanna, like, do that. So we we started just selling a straight up Linux Bos, with Node Assistant preinstalled called Home Assistant green.

tiny little computers rather

And, like, the installation

Just because of that. Sanity to a device over web serial. It asked them, okay, what kind of expressive device are you in? Wes 32 and ESP 32 JS three. There's this is different models with different architecture, different requirements.

environments, all that stuff. So that way, you can actually install Home Assistant successfully.

but seven fifty eight.

the the Python world is rust getting their rustification going, and there's this startup called Astral. They build UV.

Not really. I think the way because the way Node Assistant works is that it's set up very modular. So we can preinstall local computer or you have, like, maybe a home server, I have a Synology that I run it on. What are my options, and what's the process for getting Home Assistant running? So there's different options. And, like, this is open source. Right? So there's always a limited options, but we had to actually scope down the options a little bit to just get Yeah. So Home Assistant is world's largest open source project. It is a lot of work, and of Home Assistant. And by subscribing to Home Assistant Cloud by Nabu Casa, you support, it detects it. It will actually install Docker container with Z Wave TS for you transparently right behind the scenes. You're not doing anything. It will install

$70 computer.

is because it has this XMOS chip. And XMOS is this British company that's making chips for audio cleanup for speakerphones forever. Right? They're just really good in capturing voice.

like, voice locally. So, yeah, that that sounds super exciting.

this is amazing

And so we set off a journey in the beginning of 02/2023

to this sucks.

last week when after we had booked this, I was just doing some hacking on some some LED stuff, and

Like like, it's But I can also see Yeah. Not accidentally like like, I can I can connect to my headphones from web Bluetooth?

called speech to phrase that So the way you do that is that you create a slot. Right? So you create a slotted element, and then you can transform an HTML from the parent element inside the child element.

the with Wes serial, the ability to flash

like, this this voice kit that I think Google released it a while back. Yeah. Yeah. Yeah. But it has, like, a Raspberry Pi in it. I think it's defunct now, but they had, like, this I I don't know. They branched off to another company. But this is one of the first of, like, trying to do,

So my mind was blown

I guess I don't know sure when this airs, but tomorrow, Wednesday, February 12, we're doing a year of the voice chapter nine stream, which we've been keep doing them basically for two years now Wes we're launching a new speech to text engine Right? And where it Wes, like, I think MooTools had used one function that they wanted to use, and so they didn't wanna break it. And so they have now an array dot flatten function.

like and it Wes like, I don't know, but here's a website. It'll just, like, throw it up on you. And

questioning things. And, like, the AI has an answer to everything. If it's correct or not, it doesn't matter. It's like we're talking Minecraft. Right? So it's really good. That's great. For me, that's one of the killer apps too. Like, I've like, I'm very privacy

So Yeah. You built

runs on the Raspberry Pi four within one second, which was like speed was the most important thing that Whisper could not offer us.

so now we're building this voice tech, which everything is always pluggable in Node Assistant. So different speech to text engine, text to speech, the the brains itself.

saying we're gonna do our Yarn of the voice, which was a bit optimistic. It took two Yarn. But the goal was we want users to allow to control them to using their own language, control their smart home. And so not any of the AI stuff was just control your lights, really.

And it's amazing how it's gone from

And then if you wanna do AI, you either need bigger hardware or you need, like, a cloud API to do, like, the speech to text part. Yeah. And then you can do all the AI stuff, and it's so much fun. It honestly I cannot find, but there's this Bluetooth, install custom firmware on them. So just you fake an over the air update, and they're like, oh, yeah. A new firmware. And then it actually speaks, without corporate backing. Like, there needs to be a way to fund maintainers you know, it stuck around. And I think

I didn't even realize.

a, what, $5 microcontroller?

And it's super accurate, but it's limited. So it knows turn on the lights. It's trained on your areas and device names, and we are able to do It also allows us to because there's two parts to it. One is it should work today, but, actually, it should work in a year from now. Right? Like, if your we don't do that all in our spare time. In fact, six years ago, I started a company, Nabu Casa, which supports the development if you have a Node too many smart home systems, you could run Home Assistant on a Raspberry Pi with one gigabyte of RAM. But my colleague tried the other day, and he has, like, 15 Sonos devices and Wes cache Vercel, and then he blew through his memory. Right? So it's

changing my commands, 12¢ probably even even a lot if you're considering.

or you can do everything fully local. We're actually I mean, it's this stuff is very hard to get it in a way that it can be customizable for every use case yet remains accessible yet but they did voice. And our users were like, well, now I still need to go to Apple or Google to do voice.

it's not good at all. And, like, you ask it a question,

conscious, and just the idea that, like, every voice app you're interacting with is going to some cloud, to some company. So to be able to self host it and and have it in your home Wes pretty cool. I actually have,

it's so funny how our my perception of it has changed, being like, man, if this just if I could just pipe this thing into my OpenAI API and spend 12¢ a day

And so you put that in this voice assistant, you basically get all the r and d where Google and Amazon has spent billions on. Well, you just get it for, like, a dollar and a half or something. You put it in your device,

Now the so the the the reason, by the way, why this one is so much better and, like, whatever. I've

Then we restart it so it starts the software up. Then we have the second step feature because the second feat second step that you need once you have these devices installed is that it has to connect to your Wi Fi so you can control it. Right? Like, WLED has, like, an embedded web server, for example.

You can run it fully at home the way you want it. At its core, it's a Python application, but we prefer you use a Docker container or, like, our operating system. And it's the brains of your smart Node. So it pulls in all the data from wherever you host it. If you have Wes thermostat, we pull it from the cloud. If you have Philips Hue light bulbs, we talk to it locally. We bring everything together.