Why Is node_modules So Big?

times that by a 1000 node modules. Everybody's adding an accidental extra value. Like you look on GitHub, you don't see what's actually published to NPM. And now NPM finally has like a which is included in I think sometimes what was the reason for 3? Yeah. I think maybe there was browser gets every single language that it's ever been translated to. And that's 4 Megs, right? Like, None of these things are very significant.

transpile it to JavaScript. So generally, and that will this type doesn't have significant enough overlap? What is that TypeScript error where you're trying to Change the type of something. You're trying to cast it, and this is, you can't do that. Like, they have that in SDK,

because there's no world where somebody needs to see that what that looks like, then I did a search for all markdown files, and then I tallied up the total the built version, These are the polyfills you need, and it won't add in anymore.

screenshot in the read me in their node modules folder, and it's it's that's half a meg right there. You know? And, like so or whether that's because people don't know that there is a file called NPM ignore Babble plug in,

the bundled version either. Whether that's because they like to be able to reference it, you pop up in your no modules, can you see the actual source Most of that I would say I would say almost all of it is not the actual JavaScript that will make it into but a lot of people don't take the source files out of it. And like, PMPM already saves you a lot of space because anyone brings up working in JavaScript. It's sort of like the punching bag of, pastes in,

they are shipping

writing it ESM, and they want to be able to ship a common JS version or for whatever. Someone wrote it in CoffeeScript and they need to be able to Especially when you get into Check. I did a a search of the NP kill has, like, the best UI. Like, use your arrow keys, hit space to delete it.

in components or,

And we said under the hood in a quarter of them, which is

I personally, But for me, I have an ex EXP, which is my experiments folder. Anytime I'm working on anything that I just want a quick hack on, I throw it in there, and sometimes you forget about that stuff. Yeah. So they just regularly run a Just nuke on all of the node modules within this experiments folder. Nuke it. That's fine. Yeah. Experiments is one that I, again. It scares me. I know that it works fine, but I love the types it says TypeScript has 4 megs of translation strings alone.

and they had 14 on it. It's just a joke Alright. That's it for today. Thank you everybody for tuning in. Catch you on Wednesday.

alone of just types. So I was struggling, but it's also because, like, Scott and I, we record video for a living. Right? Yeah. You wanna see what takes up big files. It's video. It's true. I have I have, Let's get on into it.

bring in a self component or something like that, being able to dive into that. What that component actually looks like under the hood Will allow you to understand its implementation a little bit more and how to use it properly. Under the hood, we did a,

out of hard drive space. Yeah. Nuke what's in there because chances are, especially like the the one that's in your user folder or wherever it may live, Automatically

Significant. Definitely significant. Next up, we have types. This actually is

I was installing the transcription package.

I really do appreciate being able to dive into

megs pretty large in that, AWS has JavaScript to work on older browsers, older versions of Node.

the Remove them. If you find a directory named node modules on your computer, remove them. Easy script. That always that always scares me In that, I I would delete the wrong thing or or, like, also, like, it's a pain because I am working on 6 or 7 projects. I don't wanna delete those node modules just yet.

the source code for what I'm working on to further understand it because not all documentation is great. Not only that, but it it honestly helps me become a better developer to be able to read source code and being able to read the source code of the thing I'm using, especially

for cache modules can end up getting pretty big. You know, I, nuke those files out? It's probably not saving enough to make a difference, but That is

a 1,000,000 services, and I needed to, So, Even if you do care about

4 megs of TypeScript translations being put into the final thing. Then you can just run your code through any bundler Vite or Esbuild like

your entire application into a single executable last 130 InstaDex podcasts.

And out the other end, you're going to get So the license sort of has to come along, change logs for every single thing that has ever changed. Read me all of those good things key.

file and that includes your code, that includes even if you are running on the most modern version, they will often include Core. Js as the dependency. And then we npm install it. That comes along and And, I don't need that. Oh, man. It's the best. I'm gonna say it on me. Come to me when you delete the wrong Gerkley by accident.

are like, okay, well, I need to like, 1 hour of screen recording can be, like, a 30 gig file for me. You know? It's Yeah. Ridiculous. And then here we are whining about 4 megs of translation strings. Right? Well, Les Wes, let me tell you. I've been working in videos since, like, 2004, with a colon.

somebody like PNPM or any of these install scripts

That code over and over again. That's why the PMPM install is so much faster than regular MPM install. Word. So what is the solution to all of this madness? The reality is that almost all of this code isn't need to actually run your application.

the little things that we've been talking about here that you might not be using. I I think that's probably a bad idea. Yeah. Who knows? I feel like that would just you would just run into the code to the compiled code. So that way when you debug, the debugger can know where the issue is in the actual source code, not the compiled code. And the source map, as we've seen occasionally, I I had a tweet about it a little while ago that was like, what's up with all these, semicolons? No. It would you'd run it from the directory that you're currently in.

How to get rid of like, nobody even knows if they're importing common JS or ESM. Like, they wouldn't know which versions to to import. It would just be what if, like, change logs or read me's and licenses, like, automatically

removed CJS if you're using an ESM project. Automatically

like nobody really knows how to

change log or read me or license file automatically.

it just links them to the versions that are already installed on your computer rather than duplicating gigs hits the tab key twice, puts in a postal code, and then hits the tab key and enters it.

Removed, if you're using international things like that. So,

Like, the typescript Where were we? Polyfills. Oh, it's it's Babble or or Babel. Every single 20, 30, again, text takes up a lot of room.

So first of

any of I'm in a directory. I would like to remove it from this directory, and I even use it just Just to nuke out my node module sometime instead of just an RMRF because you can run the same thing for, a tricky situation.

any doing anything in video was I had a LaCie Welcome to syntax on this Monday hasty treat. We're gonna be talking about node modules, And, why is that folder so big? Why are the node modules folder what's going on inside of that giant behemoth of a folder

Yeah. Like, imagine

option to

It's not big. Yeah. But they add up. Speaking of all of this, though, like, the exported It's generally run through a bundler. You can publish straight up JavaScript to NPM.

Could have, like, some sort of configuration it kinda makes you wonder if

you had a package installer that just removed

And I had 30 gigs of node modules that I deleted, which is significant. That's fairly large, right? The spoiler alert is that it's text. Text is surprisingly large, So just colon, Braintree and Stripe. Oh. So Men. Colon cc. Let me tell you what I've I've done once

a quick command line find for node modules

So that's great because you can always get them back. You can always npm install and get all that stuff back, and it's a great way to your bundle or your runtime or something like that. So I don't know, like 400 ks. And it was being added to the bundle. And that's totally unnecessary hokey doodle, 14 megs of type most node modules, includes, And it's actually version, a Node ESM version, and then a Node common JS version.

To just simply find and remove all node modules on your computer. It's like an exceedingly like, find

node modules,

you know, all of your site experiments. Like, I don't know how you structure your site folder.

a 100 times smaller. I don't know how much smaller. It was only like I still have to trim it down, but it was like 300 ks or something like that when I gzipped it because all, chill out. It's not that big of a deal. Yeah, for sure. Second of all, bundle your code.

for every time that it is, even if it's in there 6,000 times. Yeah. And and, you know, one common way that developers can illustrate just how big text is in general is log files. Right? We all have had the experience where a log file gets written and written and written. And all of a sudden, your, I kinda wish that was better. Yeah. Well and also too okay. We're living in this world where we have

of JavaScript and browsers and node runtimes and all that stuff. So

every package.

Often they will ship what's called Core JS,

Encore. Js is polyfill to make sure that newer JavaScript features work on older versions the JSON keys of start and end you want your TypeScript to just work in 14 minutes. That's probably an outlier. But the AWS SDK is a single package,

Not every package, but Core. Js is like one of the most popular like, it's not your end. JavaScript is not actually going to be that big.

SC would autocomplete to my email. Now I can't type any words with s e in it without it auto completing my email and me getting really frustrated. So, I'm going to delete that at this very second. Prefix all my

Oh, man, I have this little thing for Raycast in the web development world that the Node modules folder

per project. So that's easy, And if you are looking to free up some space on your hard drive, like, I remember I did it once and I hadn't done it. It was the 1st time I'd done it, like, ever.

BetterTouchTool.

Oh, yeah. Do you read bundle phobia? Bundle phobia shows you how big a an NPM package is, which I guess is relevant to this episode. Maybe we should link to NPM phobia. It allows you to look up and see. Know, I use NPM phobia a lot myself when I'm trying to debug something that I'm shipping to see how big it actually is or, what I'm shipping to users. And also speaking of Raycast in the side west, I accidentally set up a snippet for Raycast and, like, not thinking that

of types. And I couldn't I couldn't really figure this one out. I think it's because every babel plug in ships what's actually big in here and and, like, where is it coming from? So the first one is And some of these files are,

you know, one of those processes where you just try it over and over again. Alright. Back to new model.

expansions significant. Yeah, I didn't think I realized. No, I didn't either. That's why I love this DaisyDisk.

babel plugins And if you are worried that, okay, well, this thing needs to be as small as possible and I can't be having Like, you have to Publishes something to NPM.

860 k source files. So when somebody anytime it's it's kind of a bit of a a meme

like, 8 k of code and almost

You do it once. It doesn't work. You do it again. It doesn't work. You do it again. You it it ends up becoming, I I like it sometimes when they Keep the source files in node modules like you mentioned. A lot of people do that even if it is taking up my hard drive space when I'm working in a third party package.

So BetterTouchTool

Very rarely at like under a meg, probably just 50, 100, depends on how large the application is. The actual code that runs your application is not very large in comparison to the rest of this stuff. Yeah. I mean, it none of it really matters. It's yet to this day. Right? things that are descriptions and polyfills and even for like I'm working on the syntax transcripts right now.

It pastes the the fake Stripe credit card for 24242, and see what's there. It will tell you where everything is, but it's not very good.

40 AWS SDK. So this is one of my projects had the AWS Node modules out there. Let's take a look at it on NPM.

called Just drop it in and you could take a look at. Oh, wow. I didn't realize So I opened up my react project for the player for my boss monster, and

30, 30 megs right add it to your project every time that you install it.

hits the tab key, with your open source code. That's kind of the rule, right? You can't you can't just get rid of those things. The licenses are you're allowed to redistribute code or you're allowed to link back to code.

So if you're testing Stripe, so that's a pain, which you do. Yeah. It it's just something if you're ever testing your credit card number or anything like that,

is massive. And

inside of every single one of your projects. My name is Scott Talinski. I'm a developer from Denver. With me, as always, is Wes,

like, sometimes it's like, well,

Yeah. So,

The, node modules boss.

yeah, LOL. It's so huge. And then there's that meme with the heaviest things on Earth and the node modules. The biggest one. Down. Yeah. It's like this. How do you center stuff in CSS of JavaScript? Yeah. And it's I I laugh at those, but also, package, and it brings me right to the npm you're on. Oh, bless. Love it. Yeah. 38 Is that an extension, or did you write that? No. It's extension that you can just install. You know what I would like is, like, one that would link to bundle phobia.

Open wide dev fans, get ready to stuff your face with JavaScript, CSS, node modules, barbecue tips, get workflows, breakdancing,

Monday. Monday. Monday.

Boss, and Scott And that Oh, that's Yeah. That way, I never Why are you so smart? I've I've been doing that for probably, like, 10 years, and it's awesome because or any of these tools.

soft skill, web development, the hastiest, the craziest, the tastiest TS web development treats coming in hot. Here is Wes, Barracuda,

anytime we run an NPM install. Anyways, I actually used DaisyDisk to

the bigger it is, the more space it's going to take up. And, eventually, it could take up a lot of space. So, yeah, you know, there doesn't need to be a lot of multimedia things to have, very large files. If you ever used, software like DaisyDisk or any of those things that'll help you I love DaisyDisk. Yeah. DaisyDisk is fantastic. If you're a Mac user, I'm very positive there's similar things for Windows, but what it does is it's a visualizer that shows you how big all of your folders are, and then you can dive in and see where exactly the space on your hard disk is being taken up with. Anybody's ever used any of those will instantly see how big their cache node modules or even their project node modules folders actually are on your hard disk. They are taking up a lot of space, which is one of the reasons why we also Often don't commit them

to GitHub or anything like that. They're they're big. I mean and we can generate them automatically nuke out certain things. Now granted this sounds like a And don't forget to subscribe in your podcast player

anything that gets stored in a file,

hard drive disk is out of space, and and you're wondering where that went. And it turns out it's just from writing a lot of text to log files. So At the end of the day, anytime you're dealing with

Head on over to syntax.fm

Hey, Hey, man. You're talking to the guy who ran get clean on his home directory. So yeah.

Peace.

for a full archive of all of our shows,

or drop a review if you like this show.

I found myself just thinking like, wow. This is all significant. Oh, all of the stuff that isn't the actual code we're using at any given point. Right? Just dump it. Yeah. If you want to know, like, what is

Turkish and Chinese and every other one. Right? And Mhmm.

You know how many languages

I don't like it doesn't make any sense that you would just go like, oh, I'm installing TypeScript. Now let me go install the the Chinese translations for that. It just just has to work, right? So that's why every single person that installs TypeScript the in literally of markdown. They're Jeez. Read me's licenses.

We'll talk about that on a different show. But, anyways, what else do we have here? Use npm ignore. Submit pull requests to people. Say, hey. I don't think you I've had translations can just could be almost be put on a floppy disk.

maybe even longer.

200 gigabyte Wow.

that was, like, the greatest thing in the entire world because now all of a sudden I could do more than 1 video project at a time without deleting. It's the best being able to have it. I even love it on my computer. Like when we went SSD and Apple is trying to sell your computer with 256 exclude it. Like, I I've had published a couple node modules, and I had some screenshots from the GitHub read me. And that screenshot was, like,

external hard drive

And most like, My computer's got 2 terabytes of hard drive space. What's a, you know, couple 100 meg? Oh, man. When I was when I was on 1 terabyte,

And we have, like, annotations for every single word of when they start and finish.

that was there, right? Because you almost never look at But you install literally

a search. You can click on the tab and see the code to npm. So you can add npm, ignore, use n we use PNPM. We talked about that. Clean up after you drop older versions of Node or older versions of browsers. This is something that for that type of thing. But now that the SSDs are somewhat coming down, Apple still charges you through the roof for a 2 terabyte, but it's so nice just having every project I've worked on in the last year, and then I just archive the rest.

happy, but you have doubled up the files. Yeah. Yeah. It's it's sort of that's the ecosystem, right? You got to make it work with both of them. So you ship literally 2 versions of Gzip is the compression algorithm that servers can send and the browser can unpack it.

Or and then sometimes there's even just like a version you can include with a script tag, you know? So that's pretty crazy.

each of them or sometimes there's even 3 versions of it being shipped. Why was what was the reason for that?

ESM, ECMAScript modules, as well as CJS common JS. We you know, they're not super compatible all the time. So would would one of the strategies for dealing with that is just shipping them both, which again, duplicating files, duplicating the same thing. Here's 2 different ways to import the same code That's compiled from some source code. So now we ship ESM and we ship CJS, and everybody's

Next one is markdown. Markdowns, the Node. Js runtime, and that's going to include all of your dependencies.

It's pretty crazy. It is. That is significant.

you have to ship licenses month and I thought, let's do a show and just talk about, like, what is actually in there? Why are these things so big because

people like to complain about it being so big. And certainly, there are issues that could be solved, but There's a lot of stuff in there that makes your development environment very smooth and simple and speed up, clean up your computer if you do need some more space. It's also just really easy to run actually Gzips

make sure that it runs on older versions of node and whatnot. So,

translations.

Yeah.

Also,

The back end node modules, pop that in there. And then I took the front end, like the React application node modules, and I took a couple other projects and I just opened them up and said, all right, will allow you to research the show. Oh, really? I just took I took I took a couple like, I opened up a couple of instances of it, and I took, like, Boss Monster, which is my course platform. I took the whole, like, And it has literally like like the example is we have a show coming up on this is that Node allows you to Package

So those are what take up a good chunk as well. Yep. Yep. And some of it is source maps, which source maps are really just a map something else like a plus or something, but I I'm gonna use Colin. Yeah. Colin's me. I I have another one, which is c c for the, fake credit card number for what's it called? Sim links for everything. Yep. You have a monorepo? It's doing sim links. It's always it's always sim linking everything together so that way you're not loading 1 package a 100 times on your machine. You're just loading it once

they have to be shipped if you want to have a better debugging experience.

There's just, like, a whole document full of semicolons, and that takes up a lot of space. And source maps are are just kinda wild if if you ever look into one exactly what's going on. Somebody actually maybe I'll find a link to that tweet because somebody explained what's up with all of those semicolons in a source map. But, you know, the source map files are text, and from the,

Next up is polyfills. So if you want your very well because are the same thing on every single word, right? And the browser only has to send that do I ever hit a weird thing where it expands unintentionally.

per version, so to say. So PNPM is a great solution for that, and it's It's very fast because of that. It has to download less. It's more efficient because it doesn't have to take up as much hard drive space, And there's a lot of great features on top of it. So if if you're like a Yarn folk or a NPM folk, give give PNPM an actual try because it's it's really easy to switch over to. Mhmm. And I think you might like it. It's it's been a nice little shift for me. I've been on it for definitely over a year and a half now and have had nothing but great things to say. And then finally, there's a tool called n p kill. You just run this you run n p x n p kill, and it will search your hard drive for

I've been using PNPM for a little while. The switch was not that painful. Maybe there's a couple things here and there, but PNPM, What it does is it stores all of them in a root folder under your username and caches them so that you only have to download each of them once, then it does,

Yeah. You could also just use PNPM, which isn't necessarily an option for everybody considering,

you know, moving your package manager is a bit of a buy in, but you know what?

doodle. We should check how many times hokey doodle has been said on this show.

That's a good idea.

So, yeah, you need types. And I I kinda hate that when you have to go install the types yourself. So a lot of packages just ship them by default because a lot of developers are using TypeScript. Or if you're not even using TypeScript, those types are still used by Versus Code to give you really nice auto completion.

the SDK for everything on AWS 10 megs every other babel plug in's types, but Every babel plug in has a meg, almost a meg of types Jeez. In it, which is is is quite a bit because you might have You could bundle your code and ship it like if you S3 and cloud, whatever they have. And I was like, hokey

Mable Preset E and V is very good at However, often people are writing it in TypeScript or people are

in that it will tell it I'm supporting these browsers and it will figure out 200 k. It really adds up. And I have had node modules folders that are gigs and gigs, which is like a good spot to find space if you run and then whatever it is, all of my expansions are are prefixed with that. You could you could use another node modules folders. It will tell you how long ago you last worked on that project, and it will tell you how large those are.

the

But yeah, that's way too big to load on the website. But it

like 7 megs. And I was like, we can't load that on the website.