Ex-npm Employee Making a New Package Manager?! Vlt with Darcy Clarke

But, You know, the security companies, I think, that are reading from, Totally. But I wouldn't wanna get the bill for that. It's reasonable. It fits in a room. Right? You can you can sort of conceptualize how big 20 terabytes is of of, all these noisy configs in if you're surfing GitHub.

I forget what the title is, but some scary title about, like, the massive The massive bug at the heart of the npm ecosystem.

That's right. So Click clickbaity

and they haven't really been looking at the contents of the packages.

the, author information, or even dependency information that is completely inaccurate.

And I coined essentially this term, manifest confusion, npm is is Pretty, I think what Yeah. What's kind of, And when you think think about remote caching, it doesn't sound that unique. It doesn't sound like it's this Crazy idea of having, the best part of that that project was actually how do we refresh the application.

that was written, like, 10 years ago We'll start.

have done a bit of a disservice to the ecosystem because they've actually just Read mostly the metadata, just like 3 other bands, No Pressure, all the all the packages. So that's That strategy is inherently gonna be slower than doing a SIM link approach,

as if it is accurate.

And so there's actually a difference between those 2 things.

1000000, hits to the website, like, that week after I I wrote that blog post, which is Wow. Is awesome.

is is accurate and that it's doing the right thing. And unfortunately, a lot of The security companies legacy security companies like Snyk's, Oh, so good. The specifically, like and then everybody else gets the money. Yeah. Yeah. There's some knock doors that were knocked on, I think, by our our support and sales and and executive teams to The big companies that were definitely abusing,

Today,

the NPM registry

title for sure.

So smart guy. Frost is the best. Yeah. We we work very closely together,

Behaviors and and the actual contents of packages versus just packages that,

query in the Volt dependency graph.

Over the last I think he well, I I think I can say this. He tried to poach me away from GitHub, He actually took that position as CTO at NPM, and he said, oh, like, if we need to start staffing up again,

and then, like, he just figured out how to do, like, networking and File concatenation.

relying on metadata to try to infer, or or using legacy sort of CVE data, which It's also been, manager

The metadata that Volt or,

one of the key innovations that I I sort of came up with in the last year or so is the idea of, dependency selectors.

is wild. That

And they're taking a a new approach, I think, in this space of actually,

We should have Faras on, actually. Yeah. Totally.

Is providing is information that we want you to

An app you might use, he built WebTorrent, like and The ability to download a torrent in the browser

Socket which is a Rust based,

This is a a pretty exciting episode for me because

when I was doing

Which was bought by Shopify.

I sure hope you're hungry.

I

will probably

he taught me much of what I know today, so I'm really excited to talk to him about

Welcome to Syntax, the podcast with the tastiest web development treats out there. We've got Darcy Clark on today.

Build's new package manager or something silly like that. But welcome, Darcy. Thanks for coming on. Thanks, guys. Yeah. It's, So it's great to be on. And, I don't think I've taught you that much in the last decade or so. So you've been on your own once.

New package manager,

which was

This

so buckle up and grab that old handle because this ride is going to get wild.

and I got brought into that that was a question before you mentioned Socket is where does Socket

among other things, he was in my wedding party, and, Package ecosystem.

WordPress dev, I was still in, like, university,

I think package authors that are trying to do the right thing. They're trying to give you, And you can, GitHub. When GitHub talks about packages

is the Syntax supper club.

building websites. I remember we we, the source, TypeScript types, And I'm sure they have some, like, Copy paste library that needs to make sure the copy paste is consistent across

Wash those hands, pull up a chair, and secure that feed bag, because it's time to listen to Scott Tolinski and Wes Bos attempt to use human language to converse with and pick the brains of other developers. I thought there was gonna be food,

Darcy was, like, the literally the only dev there, and this guy was pulling, like like, 28 hour days,

Yeah. It was hard to find that list, and then it was even harder to be like, Okay. Does this have everything we need in it? So you had to go through, like, this list of really

all exciting stuff that that he's working on. He's working on a

it was and we were just to ship Wow.

on, like, Like, home assistant uses it. Like, it's the package manager for so much more than just JavaScript. Right? Yeah. It's it's interesting. Like, you you have to try to

bait the title as, like, ex npm employee,

a, like, Holt Renfrew or something like that, and it would, like, pull tweets And I remember we had to find a list of swear words and filter them out. Oh, nice.

Oh, I'm starving.

If anyone's curious about the story, Dogs. Yeah. Dogs are wicked. Gold goldendoodles are awesome. They are

Darcy,

And

bundling or or Hosting a a cash version of of your the packages you need for your your projects.

that also has has, I think you have to always, like,

And it can be really dangerous if,

And so, yeah, companies Today do essentially host their own caches of of packages.

MPM, So this is a bit different than what current package manager clients do, which is that they,

NPM

because that tool actually just proxies through that request. So you expect to get one thing and you actually get another.

And this is sort of one of those key capabilities that NPM never built into its Platform that I think allowed for for it to sort of, so I did a lot of work. I think, initially, and net new capabilities on on top of the existing package Ecosystem.

The packages they need for their their projects in those instances, and those will be firewalled.

I think the last number I heard and and I haven't checked my own, There's a lot of stuff in there you don't use. Right? Yeah. And Almost all of it. Yeah. Yeah. Right. And so if there was a way for you to essentially

because I have a running,

locking

or Sonotypes

folks,

Awesome.

installing.

sort of backwards compatible, we, like, to to just have all this logic business logic baked into, like, the the view rendering. And and I I feel like the industry has, like,

essentially, to install your dependency, but then they'll throw away the metadata

offerings like JFrog.

One is exactly what you're saying. They host, like, an internal registry, whether it's For Datchio, which is an open source, have looked at sort of metadata defined, which is really scary. We call that, like, phantom dependencies.

And a package and then let it,

clone ish That you can run yourself,

for a market to grow around it.

The one dangerous thing about what those tools have done though Is oftentimes they will, And, a variant of a package.

override what is in the public upstream registry. And so you've heard you might have heard of this, Mhmm. Other things where it's like there's these hard deadlines. And so when I came into MPM, it seemed like, oh, we've got a lot of Hard deadlines because we're a VC backed company.

A follower right now, so I'm I'm basically indexing the entire I got it as a gift, so I I'm not saying a project by Microsoft.

And do you know how big is if you downloaded every single NPM package, do you have any idea how big that is? I'm just So cute.

He put on a USB stick.

JFrog's Artifactory notably, other companies had come in and sort of eaten a lot of the enterprise reloading entire application remotely, because we didn't have access to the store.

One, they can speed up build times. And and again, going sort of back to some of the Quote, unquote innovations team. And and the team that was supporting the CLI

Yeah. You can't put it I think depending on how big USB sticks have gotten, you might be able to do it. It's,

Values as well or or the bundled version or does it do the bundling? So that's what you're you're barking up the right tree. Essentially Okay. If you're able to control, infrastructure on the other end. So they may leverage,

sometimes The code is there. Sometimes it's not. We had a whole show on why is my node module so big, and the the answer the answer at the end of the day is is is text. There's lots of text in there.

So, like, that's one thing that always got me is

like Turborepo vulnerability in the ecosystem called dependency confusion.

the one innovation, I guess, I would say I've seen in this space, way back in the day. And and MPM in 2018,

is maybe an email every morning

But

And and that one feature is remote caching.

so, like, are you saying that, like, you could possibly

like, it's going to ship the authored

the infrastructure, back our time and and helping build up the community here in Toronto, around node school.

NPM. Right? What is Volt? And and why do you you need to make another both package manager and registry? Yeah. That's that's interesting. So Volt VLT,

All that stuff, it felt like magic, Though, when you when you figured it out, I remember the 1st time I did anything with WebSockets. I was like, this is magic.

But, yeah, amongst other things, we worked on, we often we I think we even have a show on that about, like, configs. It's like, why is there not a single config out there? Like, why has nobody Yeah. Fan. Oh, yeah. It's It's good stuff. Sweet. I'm that Monine, that is gonna be a good show. I I was at history a couple weeks ago for the fit for a king,

I remember in high school having to use, like Oh, yeah. A flash server to, like, do anything kinda socket based stuff. And, yeah, when Socket. IO came out, it just, like, blew my mind. Yeah. Yeah. That's wild.

config, and it will start nesting

npm

like, what like, we should delete this, or what happens if you delete those? Or that that was a That was the 1 person who was like, what happens if you delete those? And I was like, well, if you delete code from your code base, it will probably break. Yeah. You know? Like, it's part of your it's For your application, you kinda need that. As much as I I don't like those config files, it's

And something that I've done personally is I just have all of those config files nested underneath package dot JSON. So package dot Jason looks like a folder. You could pop it open. Hides it from your your route. That's cool. Yeah. We should let's check that out right now because people always say, oh, what was that thing you talked about?

or settings in your your login. You know? And and also, here's a little sidebar. If you're using Versus Code, there is a feature,

better than having to remember what buttons you clicked

It's not a plug in. It's simply it's built into Versus Code, but there's, like, a big config out there. I think it's from Ant Fu. He he does a lot of stuff with that. I'll I'll tweet the 1 or I'll at least have in the show notes the, the line that I have for,

specifically for package dot JSON stuff. Yeah. It's If you Google Antfu, a n t f u, Versus code file nesting config, or we'll have it in the show notes as well. And you basically Copy paste that sucker into your Versus code, I I didn't really do much work on the that one, but, like, you built, like, a tweet wall Where But,

I wanna ask real quick about the on prem registry. I just this is just out of sheer curiosity. Is that, like, sometimes companies need

File nesting patterns.

in their company. Right? Like, I've I've had workshops before where we could not figure out why npm install wasn't working, and it's because They have their own private registry of NPM because they can't you can't necessarily trust that one or they don't trust it. Right? cranking we're just Cranking beers,

in their firewall? Yeah. So there there's a couple ways that companies do this.

them for you.

essentially have, build artifacts available to your teams.

Allow you to do requires an imports alongside each other, which I think is a Mhmm. Kind of dangerous, because there's a lot of magic that's happening there.

an NPM install sometimes takes forever? And and and you can PMPM, and it's it's way faster.

then you're gonna be able to, oh my gosh. Like, over a definitely over a100. I think We had,

Yeah. I would say, like, it's always apples and oranges when you're comparing these package managers. What I noticed a lot, while I was managing MPM was people would Throw benchmarks at me, and I would say, that's that's great. Like, but,

how come Not necessarily

This is a bit different. That strategy is a bit different than the one that Von has taken, where they've

the other thing is and and maybe you can explain how this works. Is that like the, like, hosting the registry,

consistent packages and and consumption sort of strategy. That's awesome. And certain folks that can't even sort of,

Chrome web applications and and some cool stuff there. So I was there for about a year before I moved back to Toronto.

after I left Jack Cooper.

you know, it it was like almost like we would get the r and d budgets from Google, installing from caches. And it's because of this This bug of and this inconsistency in the actual APIs.

Red Bull, like all these big big brands, and we got to experiment with, As authored, and we could require as authored? Yeah. Totally. So I I've listened to that podcast, and I've actually had these discussions with many different, And to be basically ready to take down those types of packages. So that's just like one example of, like, you know, the team worked really hard, and I I everybody I worked with at MPM was was amazing.

built Steamify

web experiences. Stuff that, like,

So I actually knew, Isaac Schluter from, like, back in the day. I'm sure you folks have bumped into him at conferences. I think One of the earliest times we had bumped into each other was, Any access to the public registry. Like, they they wanna ensure that the way that your company is getting its packages is through its its third party

And they had a bunch of amazing folks, leave,

doing some contracting, but also had worked with him, And I watched it on the plane, so that's why it's fresh in my memory.

taken? Yeah. Wild west photos, Weekend of the Bull.

20 tens. Yeah. I only know because I have the, Well, I I hope one day you'll be selling the, I have the

at, FPM registry is really looked at through like a singular lens that the API,

Yeah. Those those are some good conferences. So, yeah, I had met Isaac,

the a new CTO was appointed. So our friend, Ahmed Nasiri, who had worked with, Yeah. I I'll I'll say it's it's It's super early. You can go to vault vlt.sh.

and and their open source, you know, ask for the production version of,

they had, like, wild west photos And, there's,

who's the founder of MPM and creator of the MPM CLI, When I travel, I usually pick up a copy of Scientific American.

actually, are 2 things that I think can be solved by package variants,

Jess Koff.

features Of and and we should also say is that, like, this is very early days for you. Right? Like, you are in, what, sort of planning stages?

And the there's still a ton NX and Loga,

Left to sort of add to the infrastructure and the registry, So, without Yarn, we wouldn't have Yeah.

The clients are sort of limited to whatever the server and the APIs are gonna give them.

takes on on how to install All your dependencies, PMPM, like you said, also on subsequent installations.

Awesome. So,

take on how to reify and install your dependencies.

what what kind of stuff is is it missing? Like, maybe go into the the the main

You can sign up to get early access once we we, and sort of keep, like, an index of of known packages or known good packages or Known maintain packages that they have access to. So when I when I say that NPM is the largest or when you hear that NPM is the largest, Yeah. Registry.

had an innovative,

articles and all kinds of things. And the solution we went with was get a Google box, then we had to hire an expert with that Google box to configure it.

is just actually a fraction of what I want to index.

Yeah. So it's 28 terabytes worth of, data. Big, From the registry You know, some some sentiment from GitHub as far as I know that that some of the things that we were doing were, you know, trending in the right direction.

this massive library

That's sort of the hope Really? Yeah. And so it was, like, So what we're calling distributions,

index all of the content and then send it to Google, but it would do it, like, on the thing

vulnerability in the ecosystem today.

dot dev and Snyk security, where does that fit into this whole

so that you could if you had, like,

but not as big as yeah. That that is a lot. I could fit that on my NAS my NAS drive. Yeah. Yeah. Like, Oh, that's exactly where my mind went. I was like, wow. That's big. I was like, you could, like, you could walk to a store and buy 28 terabytes as well.

And it worked out for them, but, yeah, it's a a wild solution.

today, I do have a link to a blog, and there's only 1 post on it from a few months ago.

of

server. Yeah. Like, it was a it was a blade server, and you you throw it in your rack, And it it would

We had a a client at one of the agencies I worked at who ended up buying 1 of the boxes and and keeping it in their rack. Really? Because what they did is they had It was like a the site was

on the hardware locally. That, Wes.

So Vault's, On on what we should do with config. And I I think note having just supported, a scope to be looking at your private registry, but he actually goes and and actually fetches Upstream

vault box where Oh, yeah. Feeds. Like, I saw on Marketplace the other day, Somebody was selling a Google search indexing T shirt from the one in Phoenix, and that was 2009. And I base everything off of that. That's all my memories is based on that t shirt because I still have it. That was wild. Is that the one that had,

So imagine that Volt becomes like the Google of packages.

But, my good friends at socket socket dot dev, their software in whatever way you are going to allow you to fork I got a dog last year. His name's Charlie.

it was a foundation. They had a lot of grants, but they had just

We're actually talk about a security

like, Shopify might have a 1,000,000,000 products. Right? Like, you could you could index them

So the fact that I'm right now following the NPM registry and and indexing all of that, yeah. The the re just reason I think that we Need a new package manager, and and more so, we need a new, infrastructure locally.

it's supposed to have 5,000,000

I use NPM for a lot of my

But yeah. I was I was really excited. And so just to summarize, I guess, that time, it it was definitely chaotic.

I was like, at some point, they're gonna change it, and it's not gonna work, and, like, it's gonna get out of I was like, I'm sticking with NPM.

But I I looked at it as like, Here's an opportunity to add all these features to the this product that I I use every day and that other JavaScript developers use every day. I I thought, Oh my gosh. Like, this is a dream job. Right? So Save me. Yeah. I'm I I remember around that time, I had lots to say about just the vomit in 80, You know, using 80 20 rules here, like, 80% of the use cases, package registry as we speak.

warnings and 80 security issues and and whatnot,

of someone being like, I npm installed, and this is what I got.

Stuff that would like, you npm install something, and, like, your screen like, I I would get the support request probably every day For real.

And it would be a 1,000 warnings and all this funding, and then and I'd be like, no. That's It's fine. That's how it works. Be. Yeah. You know,

maybe that's a good good

segue there is

Personal projects, but never for my course. I gotta stick to NPM. So,

And then I was hired within, you know, a week or so.

Until I I came on board in in July 2019, You know, I would say that competition breeds it comes really, I think in the JavaScript,

of reposition the the the product to as, You know, one that people could trust again and and care about.

like, many hundreds of open source projects, and I, over time, sort of was, became a janitor and cleaned things up.

The sort of community CLI like, I think the last app,

Blazing ahead in in the node ecosystem early on, and and Isaac, it was a big part of that.

and sort

And so we had over So I don't know if it's on streaming yet, but, yeah. It was it was just I have a I have a Letterboxd

For basically all the open source projects at MPM.

let let's go through them all. We have time. What's your Yeah. You got an animal? Is do they literally have, like, a server with every single NPM package on it in their

Interesting for how I think developers can can start to introspect and and take actions on the fancy graph if they they write these selectors. So Let's move into sick pics here because you said you had a lot of them, so I wanna make sure we have enough time to chitchat about that.

it it it makes it And that company I ran for about 2 years before I moved to New York and worked for a company called Fantasy Interactive.

ways of of traversing your dependency graph just like CSS.

An animal. So yeah.

What what do you got for Sick Picks today?

about the music as well. We'll save that one for for the end, but

Well, I have to ask. Like, I have to get limited here. Because I have, like, an animal, a book, A thing, a movie, and a whole bunch of music that because I haven't talked to you in so long, West, I I just need to share everything.

And treating basically the dependency graph just like the dom.

Oh, yeah. I was I was actually gonna ask you about,

almost a decade, I had, like, a a side gig that I would always lean back on if I wanted to pick up extra work.

Wes kinda noted I've been all over the place, but I I did do consulting as well,

Sort of laggards in this ecosystem, package, .com, 7, 8 years, like, we had we'd been giving I became a JK master very early on in my career.

know, time,

was hired, GitHub had just now get announced GitHub packages. So you could see that there was, like, this encroachment on on sort of, But the the hope here is that actually by including a specific variant of the the software that you want, That specifically has the the imports or exports that you need,

More opportunity in the platform. And so when I came in there, it was it was definitely like a lot of alarm bells were ringing in terms of, Okay. We need to quickly find, you had to like, I had done a lot of contracting.

product market fit again to show that, you know, we aren't it breaks the React Native Ecosystem. So there's certain 2019 was sort of going through a really tough tough time.

Really pressing, you So that's something that I didn't mention before, but,

even GitHub. When I I first,

yeah.

When I got there, and and also some of the goodwill had left with some of the the great people that left. And so, that was another thing I had to quickly try to To show people that, you know, I really cared about the ecosystem, and I was there to to hopefully rebuild trust and, don't necessarily have

and what I learned through that time of my career was, you know, how to pair a troop and and come into situations where, You know, your you know, the project is on fire. How do we get it out the door? I did some work with, like, digital agencies for, like,

time to to handle.

for for

So, essentially, it'll come with a client and a

registry implementation.

I can't imagine the the nights he spent thinking about how to approach this type of thing. Configuration is tough. That's a it's really interesting, like, the all in one solutions.

into these tools if it's all

And they were working on an internal project to to, I think, bring together all The JavaScript projects into a single monorepo, a way to contribute or they use one of the enterprise,

do it all for you. You know? Like, I will buy

you know, I'm I'm very, Oh, it's it's it's crazy. Like, there's so many Canadian are, you know, in Volt, we will be likely, I've always looked at

More seamless and Yeah. Easier? That's good. I'm I'm very much on board with a lot of these tools that Pay for, like, A 1,000 different streaming services that can never watch what I want. It kills me.

What I will say about Volt is that

aware of the current workflows that teams have set up.

I don't have to put together 8 different things. Even Even the ESLint thing, they they are rewriting their new config it has no there's there nowhere in the

and so there was, like, something like

because of the

you can't have a dependency of an ESLint config, and I had to install some crazy hack that Microsoft came up with to Jeez. To do that with mine, but it's coming. We had Nicholas Zakas on the on the podcast, and

it it was Not hundreds. It was almost thousands of repo repos they were trying to bring into a single monorepo.

You you might use Ferdaccio in your CI to To scaffold, like, a private,

like, thousands of engineers writing JavaScript every single day. Like, you think, oh, Microsoft Office. Like, What are they doing with JavaScript? Oh, yeah. The like, every single product has a web version. Right? And it's super complicated.

registry to stage packages so that you can test in your CI those packages as if they were in the registry. Like, it's it's a lot of work just to even publish a package, And so I wrote a blog post recently called man at the, you know, the And they're basically lifted CSS selectors that allow you to write, like, really expressive

Definitely, and on repeat has been,

And you'll see actually, like, people have to use 2 or 3 tools today to even, of of some of these tools now, like remote caches are are no different than, monorepo, Unbelievable. Before you had to use a flash bridge, really Tool like like Artifactory.

scaffold and build out their packages.

can help you stage and test packages all in one. Yeah. Yeah. That's a whole another world,

there's specific tools that they they use today to sort of

These companies that have, out there. Right? Like, does anything else come close? Because It's front end, back end.

Gone in and out of that trend a few different times, like putting these things together and then splitting them apart.

Yeah. Yeah. There was a time where everything we built was XPression Engine. That was, like,

I did actually a few Nexus, or even there's a new entrant Cloudsmith, Anthem part 3 and then More Than You Know, like, Those just slap. It's like

our stack of choice. And I I just went to their website, and

And a couple of companies like that that I actually did work for late into, I would say, my my PHP days. Yeah. Yeah. Man, you have, Like, you've been all over the place. Like, we haven't even talked about like, you worked on PHP 7

View controllers are, like, Interesting, I say quick, but it was it really flowed really well, with Isaac, and and we saw whether or not we'd be able to get get along together.

which is sort of like their enterprise. They had a few enterprise clients like Facebook Hour long call.

It it looks like they've modernized it a little bit. I have no idea. I didn't know expression engine was still being used at all, Some people are using ExpressionEngine out there. Yeah. And the WordPress,

cloud of Nike and Pizza Hut and Disney and whoever. So, hey,

ecosystem is still thriving. That was another one that I wrote for a long time.

but they have a you know, they have the classic client client's

Yeah. No. I I I like I like the Nespresso. I can't do hot coffee. Too acidic for me, but The the vibes are good. I often talk about the Nespresso as and I I know coffee people are gonna get mad at at this type of thing, and I appreciate really good coffee. But, like, Yeah. Like, it was

and that was a whole thing.

You know? So it's not just, like,

It's like a it's a much fancier Keurig type of deal, though. You you have

It's not the the best coffee in the world, but it's really good. And it's consistent, and it's

like, I think the last number he gave me was 28 terabytes.

And, And then I was building a new company, Themify, which was commercial WordPress steam company.

the pods, but the pods are recyclable.

I still make pots of coffee here and there because I I'm just a caffeine addict. But, went viral again next week. I'm going to the show here in Toronto, relied on the behavior of expecting Stream registry. And so you've heard you might have heard of this,

You know? Like, it's, you can just pop a pod in and And go for it. So I've often wanted one of these. I I keep looking out at like a I gotta find 1 on the side of the road and fix it before I get one. But You're saying your daddy's instant coffee. Right, Wes? He he's he's built many cool things in the day.

Committing trash to the world here.

Yeah. I did hear there's some hockey stuff in this movie though, so I'm down for that. Yeah. Jim Vasili,

There's something to be said about how technology figured out how to make it consistently good.

history lesson, it's really good. It's it's funny too. So Yeah. I so I follow a YouTuber, Mathias Wandel, and,

Yeah. Yeah. Yeah. It looks nice. I I think you would really enjoy it. Totally. And and you get everybody has, like, their own taste. Right? So you could just get different pods. And and,

it's just you know, this this turf of of package management,

Do you ever foresee a future where we literally don't

node module, we can essentially create an ESM if you only care about performance, that's a race to the bottom. Like, rip out all your code, And it's gonna be super you know, your package manager is gonna run super fast if it has no features.

Install it or import it. And Yeah. The the huge problem with that is that it doesn't back port to all of the existing So one of the great, thrown away if you had a a client that actually Yeah.

pre bundle or precompile,

are out there. So every common JS package out there that was published, you know, 5 years ago, that might still be good Software that might not have bugs, that might not be, you know, needing to have any updates to it. It may not be able to be compatible with your Your run time or or the way that you're, which is a play on Dependency confusion,

package or modules. So the module syntax problem that we have right now, like, the latest and greatest,

all of the packages around Redux, and he just went through this, like, year long trying to get everything thing published to ESM, and other configs. You know? Like, if you have a a config and you're like, alright. Well, I wanna override this Level that's 8 levels deep,

everything is so tricky because you're shipping

and by having a smarter, band that I need to shout out, which is Monine. Oh, yeah. I love Monine. Next At registry and infrastructure to essentially and, IT and then heard of IT, yeah.

Variants of it, and make that essentially seamless for you to install.

all of that stuff along with it. Do you he we asked him this question. I'll ask you as well.

you know, importing

it is it's a nightmare trying to get it all published. And that's part of why

and actually, the dual module hazard that exists, dual package hazard that exists, tools. Like, I I tip my hat So what NX is doing, and I I think that they naughty naughty naughty things that You didn't want in the in the feed.

Common JS, ESM, And and when you look into the node modules folder, you worked, I guess, like, the last one we'll talk about is you worked at n p At NPM, how did you end up there?

Man, and we had Mark Erickson on who maintains

at the same time. And then we started that That project's are also playing in Toronto in the next month or 2. So today, and and make sure that it's it's good.

structure.

and, workspaces, was it, or was that more of Like a learn nothing. I don't know. I think they came out around the same time. I'm not sure if you guys remember before I I decided to build Volt, and and we'll probably be working very closely together in the future.

yeah, I was Yeah. I remember being at your place till, like, 3 in the morning, 4 in the morning. So

Never actually went through with that, but it was a a I wrote like a 100 pages of a book

Jack Cooper, that company that was eventually acquired by by Shopify.

or I started helping you with Deal Page. And, you know, companies that has sprung out of this and and this massive issue though is is socket. And they they are actually doing analysis

Yeah.

And I, unfortunately, didn't get to work with some of the amazing people that helped build the software to where it was at before I got there. Yeah.

enterprise tools from coming along and just building the tool on top of that. And then before you know it, you're you're stuck paying the bandwidth bill For everyone's 10 PM install, which I bet is not cheap,

Like, there there is

Mild. Yeah. Man, that's true. I I didn't even think about that because,

And, he he was telling us some stories

at the meetup, not the last one, but the first one in in Hamilton.

but,

but that doesn't stop other tools,

Oh, really? No joke.

you give you give people a spot to put files, all the different parts of the BlackBerry movie and says, like,

that I'm sure I'm not allowed to repeat on the air, but

I can't even imagine. We had,

Ahmed Nasser

you know, the free bandwidth. So you you have to kinda get into the nuance of what that actually means because there's some package managers,

So if you have a cache Today, that has a whole bunch of packages.

And in the blog post, they actually outlined the very dangerous

So security companies, I think, again, have inaccurate

of packages themselves.

can be essentially,

you know, you can essentially throw whatever you want in it. And it can be completely different one from what's actually inside of the package.

What it you might find is that you get inconsistent buggy behavior when you're, I worked very closely with the Microsoft Office team while I was at at GitHub.

information today. So they might show you licenses that are completely inaccurate, That's one thing I'll I'll be looking at for sure, but I don't think it solves the problem of copying your entire, you know, dependency Yeah. Free into your node modules. What are the problems with SIM linking? There's some benefits to that because you can create policy engines, and and try to Codify heuristics about what is and isn't a good package to be,

package

problem here, which is that the metadata registry.

And I'm not sure if you folks have ever read that magazine, but it's It's great. I do. I love it. I have never read it. I don't think I've read that in a long time, the space is that we'll be providing the registry proxy with the clients.

You you look you look smarter than you are. I can't understand half of what's in it. I don't actually, Like GitHub or piggyback off GitHub, by default wants to index all packages.

brain. I I hope. I really hope. They just hold it up close to you. Yeah.

I think the last magazine I got was called mind bending physics. I don't know what the heck they're talking about in it, but it's it's great. It's If you surround yourself with enough of that stuff, Some of it permeates into the

like, I know they're they're super expensive. They're not they are not cheap.

but that, Like, even just even holding that on a plane makes you look like such a baller.

What what is the thing now? The thing is Nespresso machine? Have you you guys got one of these? Oh, yeah. We got one of those. We don't have one, but I often talk about

Like, I've had it many times, and I'm always just like, this is good. This is really good. You hit a button and you walk away and you come back, and it's It's it's great coffee.

his YouTube channel is called Wood Gears. And he's just like a brilliant engineer, and He applies that engineering to everything.

That's hilarious. That is not

And, he has a video on his YouTube channel where he goes through I've been on my own. So

for us, and, hopefully, we can find it.

pro account that tells you all that stuff. It's on iTunes, Google Play, Amazon, and Vudu.

That was real. Jim was really like that. No. That was a bit exaggerated. I don't know who they're talking about here. So I'm gonna try it. Is it out on, like, streaming yet?

But but, like, he's in a chair, so he he has the he backed it up with stats, and he did all the testing. Anyways, he he used to work at BlackBerry

music. I know. So we're both both a bit of a a punk rocker. I remember you introduced me to Sleeping with Sirens, which is, like, And where does

He even he even did a video after I asked him a question. I said, is it better to have a fan pointing into the house or out of the house on a hot day. You know, everyone argues about that. So, like, he

Awesome.

In, like, very early days, like, back when it was all,

One of my favorite bands ever. So I'm curious what you got for me now. Yeah. So I'm not sure if either of you have been listening to the new Blink album. Oh, yeah. It just dropped.

in Canada yet. It's not streaming anywhere in this room. Yeah. I'll go, fire up some software from my friend, a projector

have a Full Blast shirt that's a medium,

at

Full Blast was also like a band that I just, like, love. I still I And then roughly a year later, we were acquired by GitHub.

history? Scott, you like Monine. Yeah. Some of those You know many shows or or some of those albums oh my gosh. I could listen to those on on repeat. I could,

I've got one. It's called retired scene store metalcore, he would build, like, radios for the BlackBerrys.

That's good. I had, like, a Monique hoodie.

I feel like we're in, like, a really good era of

one of his. And then I also have a retired emo kids

Emo, punk, post hardcore lately. Don't you don't you feel like that? Like, there's, like, it's back? Yeah.

Yeah. And we're really happy with who we are right now. Yeah. And they're playing,

and it's just me Me and another guy, Nathan Noller,

for sure. Them and I think the last show we went together actually to, Wes, was the Full Blast.

I think the first one I got was The Red Tree, and then I went Yeah. The Oh, no. Are We Really Happy with who yeah. All of all of these albums are great. I'm a big fan.

and, like, this guy knows exactly what can go on it. Like, he he knows like, And yeah. I I just went we always went through that with, is probably taking up terabytes

Which is a a small band that came out of the story so far. Mhmm. Neck Deep and the Wonder Years all have, like, new music in the last year that I've just been, like, Crush it, man. I need to you you need to get on to a couple I have a couple playlists.

Do you listen to Bring Me the Horizon? Oh, yeah. Yeah. So they've been also dropping a whole bunch of singles, one of the biggest projects to, like, be mindful of as we were building out the workspace implementation in NPM. And And,

Music last year and,

Yeah. And then, like, a week later or 2 weeks later, protest is coming So the same spot, protest the hero.

and Devil Wears Prada, this agency called Jet Cooper in Toronto, made the golden convict, and it's because they are all so different. Some of them need to be able to be statically analyzed. Some of them need to be able to extend

and there was a a sign for Moneen coming. I had no idea they were They're still up there. I saw them for $5 at at the YMCA. Yeah.

man. In fact, now I gotta put them all on. Yeah.

the History Music Hall.

Sometimes people add stuff, and I I remove it, but never have I removed

bands that I'm like, Comeback Kid also dropped new

You know? And, I think it depends on which how much you get because of the doodle or the poodle aspect. Poodles are very smart.

my sister has a

or most actually a magazine. I just got back from like a registry prebuilt. So if you look at, like, Homebrew, like, there's, like, a a registry there, and it's curated by the contributors, But you're right. The the Yarn actual market share dropped while I was at, Yeah. That was awesome. We when we built that, it was it was it code igniter Air or cake. Code code igniter. I was a big code igniter fan back in the day. Yeah. I was too. Yeah. I I was, like, I really appreciate the NBC

Oh, you 2? The dog has changed my life for sure. He's a goldendoodle,

Spain. And and when I'm traveling,

Don't know about golden retrievers. I think that golden retrievers are smart to an extent, but I think they're kind of, you know, happy dumb. Yeah. I I like it when he licks my face. I don't like it when he is knows exactly where all his toys are hidden.

and, he's not the smartest dog in the world, but great great spirits.

That's great. And then, yeah, I have a book I know Jared from Bun has found a fast copy, It also gives you us the ability to build some really cool, again, hopefully, started to grow, so we built, and goal of this.

Yeah. My my sister has, I'm pretty sure, a gold doodle or yeah. She has a goldendoodle,

Anyways, he's he's amazing. You don't have a dog around places. And so, like, you know, they do deserve a lot of respect.

I was speaking at a conference there in Spain.

And I I know you have a dog, Wes. I'm not sure. Scott, do you have a dog? Oh, I got a couple. Yeah.

say get it? Yeah. Shout out to dogs, For real.

package manager manager. It's it's kinda similar to, like, Core pack. I I don't know. There's your your depths on disk It's a it's a slower strategy by default, They also just dropped new music, which I've been listening to. And it's like, of that. So clients, you you see, Bun has come out, but he's a Flat haired goldendoodle. I don't know if you've ever seen this. No. It's he looks just like a golden retriever.

a long output or So I came on board as sort of, both a product manager and a engineering manager for show Show the ecosystem that I cared about what, what they thought about what we were doing, and,

everything's a package manager to me at this point. Dependabot's a package manager, like, you know, all these tools that try to help you, You know, orchestrate or or maintain dependencies are are basically package managers in my mind.

people like yourself, Wes, you know, be angry at me all the time for for

hit over the head a few times for the the name because it's very similar to Volta,

Is when there's been a lack of innovation config files is because you are utilizing a 100 different projects that probably are providing some level of value.

and having, But, yeah, the the last December, I decided to Quick, after

whatever it was. We got a lot of Hate for peer depths and and things like that. But, to to give you some numbers, I saw like a mill half Half a Live in their parents. So there's actually, like, whole ecosystems. Like, I think the ESLint ecosystem sort of relies on this, Being able to access packages you never or dependencies you never is where you might have misconfigured, This is I'm sorry, Scott. You're outnumbered on on on the Canadianness.

do you need to be able to import things into it? What kind of should it support Wasm? Like, it's

If you are concerned about people touching those things or or For it to be noisy, then that's like another another problem. You can easily hide those things. I think Cyndrasaurus has, like, a great Git Plug in that helps you hide them in in, in some cases, I've even seen,

fold into

why do we have all these different standards? I'm going to create A new standard, that will be the standard for all standards. Yeah. And and then also, now you have 3 competing standards, and Are you able to talk about explain why that is and, like, what your thoughts to that approach would be? Like a string, and it would assume that the file would be there. But because So me and Darcy, like, rebuilt it

And, like, deeply then a lot of this becomes

it's a minefield, and that's why Nobody has really figured that out. There's there's the config folder, which is great. You can you can hide some of a lot of those files, but, Man. So,

yeah, I I I had one of those tweets too. I took a screenshot of all the config, and people are like

Like you know? Yeah. So for anyone,

Git repo. So or GitHub repo. So you don't even have to see

you know, a new place to put all your config.

how we've evolved that ecosystem today. I think you're gonna end up with, like, an x k c d situation if you try to, like, standardize a new,

overwrite everything, or does it fold in? Does it fold in just just that object or all the parents? And,

not familiar with that xkcd article, it's,

And their location to me isn't that that problematic.

does that then

or docs or things like that. All this cruft that you might not need.

Right? And, Miss my family. So Yeah.

if you look at your node modules, like you were saying, Wes, like the But also the ecosystem broadly, Fras and I have spoken a lot.

shared,

And by being able to do that, we'll be faster than any other package manager out there. That feels like it makes too much sense. It feels like like, Why yeah. That so many so many things could be solved with that. Like, 1 guy putting an 8 meg screenshot

packages with sort of some innovative This came up actually. I worked very closely in the the node tooling working group, It's he tried to buy a hockey team and bring it to Hamilton, pressing sort of projects. So, like I did projects for, like, the Grammys. I did projects for

When you apply that to essentially the package management space, you say, how many times a day are we all installing the same even at MPM, we wouldn't have, You know, I think been pushing as hard as we were in in a bunch of different

Yet, It was it was pretty down to the bare bones. It was like a skeleton crew.

it it would actually make the installation process Exponentially faster than on all package managers if they could essentially, packages to to

And resolving the same dependency graphs. And how can we make that more efficient? And then I was writing a book actually for Manning Publications, Yeah. There was always movies, torrents. If that like, our support team was actually really good at that time of knowing if there was a new, like, Marvel movie out to To be, like, watching for these keywords.

of a package which leaves behind its tests

that was, like, a really good learning experience for me because, like, I remember needing to send somebody But it's, yeah, it's it's much much better. And I had never strayed to using yarn or PMPM for any of my courses because I knew

created new lists Spur City. And, like, I didn't even know what transactional email was at the time. Now that I know, I'm like, I coulda just looked over the database and

So it pulled in Groupon and LivingSocial, and there was, like like, probably 50 of them in Toronto that were every day, they had a new deal out. And I would aggregate them all into 1 page, and it got pretty popular.

of the daily deals in their area. And I, like, I use Mailchimp, and I, like, programmatically

That was a lot of fun. We did some we pulled some crazy nights to get that done. I think I was working Yeah. 3 3 different jobs. I was still at,

and,

and Made it for, like, all of North America,

like, I had this thing called Deal Page, which was a daily deal aggregator.

Incentive, but I had I had no idea.

at MPM and GitHub, and and we saw that,

story? 4 just came out. So I I I do wanna give. Wow. Yeah. Yarn 4th. I wanna give a big shout out to to Myel and and the group that does maintain yarn.

Hopefully, direct competitors with with tooling like NX, which, you know, is trying to make,

You know? Yeah. Made that so much easier. That I was that was a world of hurt when you couldn't do that. Now it's it's supported in all the tools, and

with the MPM. And and without PMPM,

Yeah. I think Yaron was the 1st package manager to do it,

which was first.

It's it's great. Yeah. The the workspace support is is interesting. Like, monorepo, development me and my business partner Nick Law, just before me, mirror you know, those variants for for you. So in the case of, let's say, a legacy,

in all the various numbers we were tracking. Yeah. And Yarn was the first to do part of sort of a blessed community. And and, some ecosystems like the React Native ecosystem.

Innovation. And so

Man, I I can't imagine. Like, we we had a story on the the recent horror story podcast of,

quote unquote, that have access to you, like, even more than just the packages on the NPM registry.

I I can't say his name, but he said he had one of the 6 laptops in the world,

that could fix NPM if it went down. And, like, Like, NPM is it's probably by far the biggest package manager

put it into

and and other package managers and other ecosystems also Space. So if you know JFrog or you know Sonotype,

that come with them pre,

put it in a category with package managers. Sometimes there's just clients We wanna show that we're still able to scale and grow, and then there's still, worked really hard to build up a a new team, you know, IBM was really good at, like, sharing their learnings, and and obviously, it was a company that was sort of blazing

like system call for, I think, Linux and and Mac OS. He's found a trick there to to make copying a bit faster.

which makes it slower to interpret your dependency graph. And also how it lays,

feature rich

more so than any of the existing or the other legacy Package managers is what I'll call them,

Well, they're not supported by Windows.

where you essentially only write to disk The one time for the package, and you don't have to copy around.

where it actually copies to disk, ship a package.

and highly configurable,

space. We had a whole discussion about what to do with configs, team, open source team, like I have the right I have the perfect person in mind. I had like a quick

I I'm curious about your personal thoughts on on

Man, that's a that's a wild world when you go from, like, slinging together a little They will abuse it.

all the libraries, and it has to be tested. And, Now they are a good good chunk of Shopify.

Thousands of packages. You know? Photoshopped on the browser. Yeah. Yeah. Yeah.

versus, like,

The concept of, like, people talking about you know, because we're we're getting into config a little bit here. But, like, people saying, like, alright. We have all these massive amount of config files.

config as as just like, will use both of those things, the metadata and the,

The fact that you have a 100,

and that I recently saw that got kind of, Yeah. And then I had a movie, which I think I hope you both have seen by this point. But, the Blackberry very movie? I have not. No. I'm not Canadian, though, so it hits less hard for me. To.

your dot m file by default, and they were sort of pushed into that by button. I think this has become a a problem again.

What are your personal thoughts on the solution for that?

Svelte site

on there. So I have an experience Or a memory of that where I was literally outside of that that that store in the middle of the night with my laptop tethered to my my iPhone 3 g, They're my favorite band So it's it's not like a major problem trying to annualize and you can, I definitely WordPress You know, scaffolded and and and, And dependence confusion

you were able to make the tweets, like, Like, load in. Like, they would animate down as new ones came in. And, like, we didn't have CSS transitions back then, so that was all in probably jQuery. Right? And, And

Like, what do you do? You know? Like, the it's locked, and the it's basically just a web page on a projector,

It was primarily at night, and you it was, like, in, like, a locked glass building. Right? And, like, it was on Queen Street, I think, and everybody would walk by and see. Wow. But if something went Wrong.

I and

some of my 1st open source contributions

were to, like, docs websites, adding, like, better CSS and styling just to, like, even improve the experience that I had. And then you worked at like, I I don't wanna make this, like, a Darcy hour of work work. Like, you you you worked at so many Cool place like what was that? F I, Fantasy Interactive in New York? Yeah. So I moved to after Themify sort of,

of PHP. And the PHP docs were, like, to me, The quintessential like, that's how I learned so much about the language was the PHP docs were actually, like, really good back in the day.

you know, there was a there was a gate before, So it gives you an opportunity to essentially manage, One of the ones in Florida. One of the like, Maybe in the 2000 2010.

or even,

On their platform, it's, you know, it's only a small Yeah. And so that that company was doing really, really amazing, I've already gotten And all that cruft could be, of that package, then you can install it. But that's Yeah. Wes, if you haven't seen it, it's really good. And and Scott, like just even for the tech tech, net new to Distribute that.

Like, in that era of NPM, was it insanity, or what what was it was it exciting? Like,

I I just I talked to myself about it. Yeah. It was a it was pretty, what's on

it's actually a lot bigger than just the numbers that you see on npmjs.com So Holy.

to that corpus of of open source without having To be, you know, file system and similar to PMPM's, And then once MB Star came around, I was like, what is why are people, like, throwing everything together? mostly annoying and and not necessarily helpful for finding malware. Wow. Beautiful.

percentage of actually the packages that exist in the ecosystem. But, yeah, MPM is definitely the most, I would say most notable, and it I think it definitely unlocked one as well. So I'm an emo kid for sure. I was I was like a get up kids fan, dashboard

for the ecosystem that I've been listening to a lot. And then, derive, doing node school for So package management space. So, that's why I think it's it's really important that we both provide a new client as well as the infrastructure to to sort of add some some new value there.

that kinda got unlocked by by making publishing really easy. Totally. What was what was working at npm And, trying to I looked like a crazy person trying to, like, hack the store on the other side of the glass. Just you know? Yeah. Yeah. I have I have such fun memories of that time in web development. Even just I remember

AppGM itself even will download packages from GitHub or remote tar balls. And so when you talk about the scope and the scale of the actual Package ecosystem, And so I came on board, Strangers and Lost, like, 22 ones three and a half years, basically working on the largest package manager in the world, MPM,

private instances of the of this registry yourself.

If you want to, if you're an enterprise, you need to be firewalled.

APIs,

We had the hardest time trying to get it on Vercel's Mhmm. File system. And, like, that's the beauty of all these this modern JavaScript tooling is that, like,

The required tree was that WASM file.

entertain that install strategy. Mhmm. What happened to yarn? Like, it was it was popping for a long time, and then I know yarn 2 came out. And Do you have, like, a a a Kohl's notes of that whole,

It was simply just a string that it hoped would be there.

which did something similar in terms of a virtual, I I saw And beyond just how do we make the resolution more efficient? Do we actually make the bundle size more efficient? if you happen to be an attacker that knows might have known about some, like, leaked package names companies are using.

A lot easier. I was gonna say, yeah, it's a problem for either live and and pulling from the registry or,

So you have to use junctions, and it becomes a big mess,

If it knows that it is there. You know? If you import the thing or if it can statically analyze it somehow, what's on GitHub

the same idea with the Vercel file system is that, like, we had a package that simply just referenced,

They weren't able to adopt Or even, I think, try out Yarn's PMP implementation,

if it

And so we did some interesting hacks to, like, get it to essentially refresh the entire experience.

Back in those days, there was, like, no live reload. There's no, like, a mechanism for, like, on the infrastructure side,

the

It gets pushed to NPM, and now next thing we're going on is airplane trying to download all this stuff. Right. Yeah. Because your client is dumb. Like, your client isn't very smart. It's not asking for the right things. Really, the infrastructure needs to be like, oh, let's create optimized versions of all these packages, Yeah. So if you actually go to the vlt.sh

essentially request for for those. If you need the source, if you need types, if you need, with abundance of of great technology.

Original version or variants we we are burdened has been by, you don't need the read me.

of space sitting on some server somewhere. Bandwidth. Because Yeah. You don't even think, oh, yeah. Like, I'll put a little screenshot of of what this is in the read me, and, yeah, you forget to add it to your npm ignore, and then it it gets

Right? So there's immediately.

Alongside his Readme

We've been having that on every single dinner. We don't we we listen to music during dinner, so we'll just have that album on every night during dinner right now. Did you get tickets to the the Toronto show? Alexa on Fire's opening for them. No. I didn't see that.

But I do have tickets to the next And most clients today, most package managers today, actually

store, I think they're now just called Fantasy.

the packages at rest differently than essentially how they're stored in the registry infrastructure.

So it's essentially you are hydrating an instance of the, installation strategy, I think it Breaks on, as well, didn't you? I no. I I don't think I I Contributed back to PHP 7, but I definitely worked on the docs, these capabilities for for companies. So they will literally,