Cloud Storage: Bandwidth, Storage and BIG ZIPS

meaning that I have a Synology NAS Because if you do ever change them, at least you can write like a worker or something like that to jump in between and and rewrite those paths. Still pain, though. Right? It's huge pain. That's a lot of work, and it's likely not worth it for a lot of people. Totally. There's also this thing called the Bandwidth Alliance, which gives you either free or reduced pocket Bos, and you can you can hook it up so that it'll it'll upload files to s 3. But you can just take your

and I have that entire thing. Not the entire thing, actually. I have the important parts, in free bandwidth.

all of the ones we're gonna be talking about today are

data can be confusing and expensive.

hunk of a file. It could be part of a file, but you can essentially think of them as I just told you, it's 3 times more expensive than Backblaze b two, the m p four files so they can download it and watch it list all of my buckets.

and I lost

but it's all local. So

yeah.

most of my video projects that I made when I was at early college, which is fine since it really sucked. Yeah.

I'm pooched. I'm pooched. Yeah. You're right. Yeah. You're right. You're right. And I did lose a 200 gigabyte LaCie hard drive back in the day,

keeps popping up as well. They seem to have Mhmm. And Azure Store. I've used all of those, but I like Cloudflare and Backblaze the Bos, and I'll I'll tell you why in just a sec. But also this, like, bunny.net And, like, maybe in, like, the developer community, this is a thing of the past. But my sister's a photographer. I talked to so many of them. They have My hard drives are clicking, and I can't turn it on. And it has wedding photos from this couple. Like, please.

both because like, I've I've used delete a file, that's an action. Most of those actions, except for, like, deleting files, every episode's probably wet. Like, we should get Randy to tell us, but it's probably 30 gigs. Right? Yeah. It's not small. If you think about 30 let's do the math right now. 30 gigs Backblaze, And then class b is, like, check if a file is there or or or get the file size, but not download the file.

Amazon Pnpm, Cloudflare,

I've never seen Bunny.

but they also have a storage product as Wes. And the prices on it seem they it would be too expensive. I'll talk about that in just a second. So let's first talk about a couple different ways that we use cloud storage. I just went through all of mine, and I I took a look at, like, what am I actually storing? So Mhmm. My biggest one is I use backup backups, all of those types of things are often not part of your application, but you put them on a cloud depending on where they're coming from. It's just the Internet. Right? But it it would cost more if you need to put them in different you get a server, a year have have clicked on files at some point. And you see in the URL, and you want to move your WordPress files to to Cloudflare, you could or vice Vercel, So that would probably cost to allow that, I I throw them up on, one of these cloud providers and allow them to download it. Older code Bos. I have, like, basically, like, 200 gigs,

So I often think about that. I'm like, that's I know how much this Scott. That's kinda expensive saving the high quality versions, editing it, and exporting it, storage is one thing to think about, but the other Node is to think about bandwidth or often is referred to as egress. So when your data now needs to leave the server because it is being requested by somebody to download that file, the people who supply you with compute they are all s three compatible, meaning that if you if you just take a library, like Scott was talking to me about

because, CDN products,

then you also will incur some sort of costs.

Yeah. There was a a website I was on, and they were trying to explain, like, this concept called temperature, which is, like,

and it takes 8 hours. You have to request it, and then 8 hours later, you get an email and says, hey. It's ready. Or 24 hours later, you get an email. This is ready for download, and that's because sure. Amazon Wes is 9¢ a $15 per month for me to sort of hold it on there, which is

So the this stuff is so cheap. And, you know, we just talked about it being a little bit of expensive, but, like, it is so cheap And Yeah. To to to incur the bandwidth Scott, but maybe they're using Cloudflare r two. And is it just ads that's covering the cost of that? And it feels crazy, but they must be getting enough traffic to make it worth it. I I I do wanna say, before we move on off of egress here as a concept, if you've ever heard the word ingress

I think that's due to, 1, the drives that they're storing this stuff on, they're not optimized

Even when you buy discs for your local NAS at home, the drives that you're buying are different. They're not optimized to read and write fast.

copies around the world, so it's faster. And the slower it is, the the cheaper it it gets because, download my files. I've used it a couple times just to roll back and get older versions of files as well, which is something most of these providers do as well. If you if you upload the same file again, it keeps the older versions, list a file, list files in a bucket, And then Synology has 1, which I didn't realize, So talk about fat pipes.

because some some of it does need to be Alright. Let's talk all about it. Storing So let's talk about some things that we're not talking about really quickly is your your Vercel, and, like, why would you want larger storage? So usually, when you host your application, on cold storage so that it is much cheaper to hold on to, and then there's a process for getting it off of tape. You have to go into Jeff Bezos's basement and pull the VHS off the what else do we put there? I put the original recordings on. So That's that's kind of expensive, don't you think? Expensive.

websites that offer, like, NES ROMs or like, I downloaded a couple Wii games when I hacked my Wii. Yeah. I was like, who's paying they all

which is

to let me download this, like, 6 gig a Yarn, and we have probably Let's talk about bandwidth now. So to download them. So storage, it costs money just to hold on to data. Cloudflare is a 1 and a half cents per gig per month, whereas Backblaze is 0.6¢ which is like my screenshot kinda quick file hosting. You need to send somebody a file. You need to send them a video. And somebody on Twitter recommended drop share,

file of Wii bowling? Or I I don't know how big it was, but it was many gigs. You know?

it will take longer. Like, I went to download some of our old episodes, But recently, I was looking at terabytes they have, like, a a little string behind the offices where they can call each other directly, little tin can.

to to be super duper fast. And if it does, then you're probably gonna be paying more for that feature. Right? This could be handy if, let's say, you know, I've had situations when I ran level up tutorials where people would say they can't access Yeah. Backblaze does

these if it were to, if I would ever lose my computer or drop it in the ocean, something like that, I'd be able to CDN distribution around the world, those things sit on top of different locations.

Yeah.

maybe they're they're a little bit smaller, but they're downloaded a ton. Like, you think about

are putting less frequently accessed episodes Every time I talk about it, somebody comes out and says, hey. Check out

users are uploading images, things can fill up really quickly. You know, back in the day in WordPress and Drupal land, you just uploaded it to your own local server.

oftentimes, you'll need to transcode video into a whole bunch of different formats, especially if you're streaming video.

you know, their full infrastructure. You can still own the bucket with all of the data. Right? Exactly. Yeah. Yeah. I like that a lot. I I use cloud storage for a few things. Oh, images, maybe not something you mentioned. Images Yep. Pay. Especially in websites where

approach to that. Right? Because then it's not tying you into,

And and just really, like, that that's pretty much it for me. You know what? I don't have an off-site backup, Wes. I have a, like, a double backup system,

Yep. Like you said, big zips. I throw my zips up on a a a Backblaze b two that we'll talk a little bit more about, but I I hosted all all my big zips there as well.

That that was fine. But, you know, if you host these in the cloud, they can be accessed by all kinds of things and not just the current application that you're using. Either way, user images,

Video,

you can create transformed images.

It's it's so good. You use the the computer backup product?

Yes. Oh, yeah. I've been probably for 10 years now. It basically is just always and that's one thing I'm doing is I'm always pausing it before we record this episode,

which you can roll back to. Yeah. Yeah.

or Cloudflare for that matter. Things are more findable,

to hold data in there month to month, and how much does it cost to get the data out if people are downloading it? Because some of these providers are cheap to hold data but more expensive to download it, and some of them are a little bit more expensive to hold data, and they go as far as being free Yeah. I've been dipping into this a little bit lately. I've I've hosted stuff on cloud providers for probably over 10 years now.

backup

would even look at Cloudflare before Amazon isn't because of anything other than

to backlaze b 2. And moving some of my stuff to a cloud provider, and I was like, oh, let me revisit this. I've been using Backblaze b two for quite a while, but let's see if there's, anything else different. And I've been noticing the Cloudflare one a lot lately as well, Cloudflare r two.

and downloads for my video courses, I actually don't incur any bandwidth, discounted, they say. Mhmm. And then there's free I'm trying to see any of the big ones out there. There's a list of automatic is a big one, which is kinda interesting. So I guess, like, if you've got a massive WordPress site games. Yeah. You're you're negotiating, like, your own terms on these things. You're not taking the pricing page that's all we got today for cloud storage, one backup of a file is not a backup. That's where your file lives.

This is a fictional thing. I'm not doing it. Fictional. Yeah. It's like an office space type of thing. Yes. Totally. Alright. So let's talk about we're we're gonna break it down into

right Node. And because that is purely not purely, but almost entirely Especially just like a little thing that is so easily easily dropped. This is not the backup show. Well, maybe we'll do that again, that show as Wes. But, Cloudflare r two credentials or your Backblaze b two credentials and pop them in there because a lot of these APIs are s 3 compatible, so they'll work with with anything.

episodes.

it will continually

AWS.

The I was thinking the other day I was logged into my Backblaze, and I use Backblaze both to mirror my computers, and I use b two to host a lot of files. And their entire

and So if you I'm hosting 2 and a half terabytes,

the dev experience of

times, a solution for going off of cloud app,

and it's just storage fees. So for 2 and a half and big zips.

retrieval fees, then you're going to basketball games and having dinners with the the sales reps

Yeah. Oh, that's that's great. You know what? The I think the big reason that I picked Backblaze and and

back up my entire computer's hard drive how much does it cost the faster it is to retrieve your data

less opaque, and it's just the nature of AWS.

their entire product offering, this is a massive, massive company, You have to consider

You know? It's really easy to get going with Backblaze

Speaking of dropping your computer in the ocean, do you ever think one day it would be feel real good to just, you know, frisbee that thing right out into the ocean when you're feeling really angry at it? You know? But then I would quickly go get it so it doesn't harm the environment. But yes. Sure. Well, the the yes. Okay.

so that means I have 7 and a half terabytes of free bandwidth. So, like, but then the the bandwidth would be free. Yeah. Most people Scott something that they're worried too much. I I think and, also, like, if this if if you have such a scale that big, of data being stored in Backblaze b two Bunny because they have a lot of interesting products out there.

ingress. Those are are 2 words that it's like those are kinda jargony words that you may have heard here and there, but

Why is it not called outgress? We don't know. Yeah. That's actually a good idea. Yeah.

JS the opposite. So ingress

honestly, I'm gonna say right Node, it's very hard to beat back Blaze b 2.

is data coming into the system.

Egress Mhmm. Is data coming out of the system.

ingress, in. Egress out.

syntax. Sign up and get 2 months for free.

truly important that your users aren't the 1st ones to bring that to your attention. So with something like Sentry, you could set up alerts

for any error that happens in your application, and you can make sure that if there is a issue with a critical part of your infrastructure, whether that is grabbing a big zip from an s three bucket and downloading it or what, that you get access to all those errors and are able to solve it. So head on over to century.ioforward/ Gotta have them both here. So let's get into it. My My name is Scott Tolinski. I'm a developer from Denver. With me, as always, is Wes Bos.

the download. Maybe it's throwing an error when they try to access the download. And if you're selling a product that relies on that download,

Yeah. Sick. I'm excited about that. I I've stored on all kinds of things, so I'm interested to hear what some of these other options are that I have not tried to store on yet.

This episode JS well as every episode is presented by Sentry. Sentry is the perfect place to track all of your errors and exceptions.

Yeah.

it was also

5 terabytes of data stored with them. Yeah. That's a you know, it's funny, Wes. I was just doing some clean out of my NAS because, you know, it just builds up a lot of duplicate files or things that are just, you know, stuff you didn't wanna keep anyways that just kinda made it on there. And I found a raw footage,

Yeah.

where you store it, it costs different amounts, which is class a is like upload, a couple things. You have to consider the cost of storing that data. You have to consider the bandwidth of people downloading it, and you also have to consider how quickly do you you need it. So 70 terabytes of user images, hosted Hey.

file from a ScreenFlow project that was 200 gigabytes.

like, because you have to pick where you're gonna store it. Right? Do you wanna put this in Virginia, or do you wanna put this in Australia?

Yeah. That's that's the truth. Oh, man. So

other ones, I won't go through all the prices. Amazon Wes three is actually quite a bit more expensive, 2 and a half cents per gig. And then at least when I did it, this was many years ago,

to host or to sorry. To hold on to GitHub even downloading it. Just to hold on to it. And, $40, not not too much, but then you start to think about like, I often think about Riverside.

a year for this podcast.

$40 a year per gig per month. So I'm taking a look at mine. I have 2 and a half terabytes offline cold storage. I think Azure has their own version of it as well where

which is really cool. It's like a essentially, a screenshot tool, also file hosting, r two is is for you. But if you're storing stuff long term and you have, like for example, my And I was just like, what 15 years of web development of old websites that I've built, and, like, I don't delete them. I just put them up. You could put them on GitHub as well, but I just drag and drop, put the whole sucker up on like like, this podcast Node. After we're done recording it, Often, people will just stick a CDN in front of those files that are on the same server. That's a really good solution. This is not a show about CDNs,

and then you connect it to any of these cloud providers.

And then I've been recently this is why I got into this is I've been trying to find 150 I realized locally to in my house, c 2. And then all the usual suspects, Google Cloud Storage, Azure, DigitalOcean, Oracle, my actual course downloads which is wild to think. So Cloudflare r two,

and and then moving it off. So sometimes a lot of people say it's cheaper just to stay on this than it would be to very good. So like, Amazon has a product. This is one of the features things, but I'll just talk about it now. Amazon has a thing called Amazon Glacier, And the cost for those are are relatively minimal. What do we got here? Cloudflare puts them

but they can't necessarily at least once a year, a year and a half, I get a Sanity message from a friend of a friend.

Put a custom domain name in front of them.

to move those all off of Amazon? You're gonna incur the bandwidth in order to host all these files and hold on to them for forever? Because Riverside JS not it's not expensive. It's like $250 have this. And often, for those actions. So there's not a a a ton of them, but every single request of a URL,

for everything offsetting them, Hopefully, you learned a thing or two, and we will catch you later.

1 or 2 things. It does them very well. They store things. They back up things for

That was a total pain. It was a massive pain to to swap out all of those those paths. Oh, yeah. Yeah. Yeah. Here here's my tip for that is don't use the Amazon Wes three URLs.

how much is it gonna cost

even update all the paths now. You have all these paths that presumably have all now changed. Right? I had to do that with LevelUp because s three, I found to be very slow when I was hosting my courses there. I just moved into Backblaze. It was gonna be cheaper and faster for me.

to use Amazon Wes three and and to move it off and pay the egress fees for that. It's also a pain in the ass to And it's that seems ideal to me. So I'm I'm looking at figuring out how to move all my 15 years of screenshots and Mhmm. File uploads over to that. It does seem like a a good man,

get off of it because if you have recording files, locally. So in order to And then when they operation fees or or retrieval speed. Sorry. So the first one is storage fees, and you can kind of break down the cost of of these things into when I was we'll talk about Bandwidth in just a sec, but I off I was looking at the prices, and and I have a lot of people from India download my courses.

drives 1,000,000 which is my

oh, man, your drive JS all of a sudden stopped working and your your backup is toast is over, but that doesn't mean that. You shouldn't be I literally, last night on TikTok, I saw this photographer.

the era of,

Yeah. That's actually that's such a good point. And it does feel like

3. And then along with Amazon s 3, there's a whole bunch of different options out there. Cloudflare r two, which I thought was kind of funny. They took the letter in the number before Wes 3 and named it. But then as I made this list,

Backblaze b 2 is also like, did they do the 2 because of Amazon Wes 3? And then What's up with that? Yeah. I I I'm assuming they did that.

not necessarily distributing you that data. That's not their, you know, their primary task.

Exactly. So big players in this space never incur any bandwidth because I'm sort of You're you're gonna you're gonna hear us talk a lot about Backblaze and Cloudflare that's a whole another level. And often those services that are image resizing, compression, storage provider because charge you by the million, gig for bandwidth, which is another one that I do. So, like, 99% of people stream the course. Right? You got a nice experience. You get the notes along with it. You get you track your progress, but some people just want the So Cloudflare, free bandwidth, free egress. Backblaze b two, it it kinda has this really interesting thing Wes they'll give you 3 times your storage That's $3 storage fees, bandwidth fees,

it says, like, bucket or s 3 Scott or, stuff like that. I have that mirror to Backblaze b 2, and that's about 2 and a half terabytes These big zips for all of my courses retrieved extremely fast. Mhmm. The faster it is to retrieve your data, the more expensive it is. And you can even go 1 step further. And if you put a CDN in front of one of these, it's even more expensive because they're distributing

between the 2 of them. You're really killing the, metaphor today.

DreamHost, Backblaze, a place to store large files. So things like big zips, software downloads,

And Now the kind of interesting thing that has been happening lately is a lot of these cloud storage providers are giving you free egress, are the they're like I think the big first one was Amazon Wes How much are they paying in Yeah. They That's interesting.

Thank you. Thank you. Who else is part of this? Azure and Google Cloud are when I'm recording a course or this podcast, the raw files can be very, very large. And sometimes you wanna keep those raw files, so it's nice to have a spot to put those. I put caption files on there so I can download them and put them into my course player.

that's free because it's it's just going data center to data center, and it's not really costing them that much because they got a fat pnpm, that's cheap, but then you start thinking about, You know? file that lives behind that URL, and that is one action.

every time that you create a what's called a bucket, which is kinda like a folder A lot of these cloud providers got fat pipes between each other. So, for example, Backblaze and Cloudflare, it has to pull up that yeah. That that doesn't make sense that it would cost more files. It can be any type of file. It can be any in that regard.

in these cloud providers,

And that's really cool because if you are doing if you have a worker on Cloudflare, but your files are stored on on Backblaze

they say, alright. Well, if somebody asks me for your data or if I ask for your data, it's gonna be free.

and you need to download the file in order to to work with it in a worker, all around their house. They're probably moving around. Every

that's that's an action. Right? It's it's an API hit. Create a bucket. Right? Mhmm. Every time you upload a chunk to a file, that's an action. Right? Every time you see if a file exists, that's an action. Every time you, US East. It's like in the URL of the Node, it often will say these words, and that's kinda how you can tell that, oh, these files are actually being hosted on Amazon Wes carte blanche.

Oracle,

the big player for a very long time.

Cloudflare r two. Yeah. I like Cloudflare a lot. Maybe maybe I'll try that sometime. It's a Yeah.

Yeah. S 3 was certainly

But, yeah, I find myself using Backblaze b 2 anytime I need to host anything anywhere just because I'm very familiar with it. However, for read and write speed. Right? They're optimized for a long term storage.

that's courses that I don't want on my laptop, because for $4.50 for people to to download from. Maybe that was their CDN product if I'm thinking about that again So I I dipped into it, and I learned a lot. So here to explain it all to you.

for the initial request,

providers, Bandwidths,

Peace.

that will cost you $4.50 but it has free egress. So if you're hosting files that need to be are not necessarily the same people that supply you with, like, large amount of storage and bandwidth.

So those are let's say you have large files. So dozens of these little 800 cost of egress between on a old Mac, and they pnpm it in and and put it on a USB stick and then run it over to you. Wow. Something like that. Yeah. That's the whole that's quite the story. If we had an animator, I would love to have somebody say that.

Peace.

into class a and class b operations, of of data. And anytime I throw something onto my NAS, it it just auto auto syncs and uploads uploads it to Backblaze b 2 so that I have a backup backup because

Mhmm.

service providers.

Welcome to Syntax on this Monday hasty treat. We're gonna be talking all about cloud storage.

Big zips. Big zips and big pipes.

You gotta put your stuff in the cloud. You gotta be able to access it. And because of that, we're gonna be talking about storage itself JS well as bandwidth, all of the different terminologies and services you need to know in this topic. The big zips.

fluff or anything like that. It just it's a really good company. So if you have lots and if you use lots and lots and lots of bandwidth, terabytes of data is just your raw and then there's a bunch I have never heard of here. Last thing we have here is just operation fees. These also cost money. So

because I store so much backups,

And they're, like, just like a wicked all around company just because, like, there's no weirdness around them. There's no, like, marketing is very expensive. I'm surprised a lot of people. That's the other thing about it is there's a lot of people on Amazon Wes 3, That's 4.5 1 and a half cents each, So is, like, 15 links in the sidebar. And, like, there's there's that's it. Organization. Come on.

probably Cloudflare but you pay for the app once, off the shelf, and they some poor person has, like, a FireWire And India is the most expensive, at least it was at the time, they a month, and then 10,000,000 of the class b is 36¢ a month. So if you have a 1,000,000 images that are downloaded, log files, images, videos, user data backups, three. They kinda came out and said, hey. You got wanna put stuff somewhere, and you'll probably

object structure, which is what we're talking about today, which is basically just a place to put You gotta fix that tonight. You're not gonna be able to sleep. What happens you the last podcast, you just talked about a tree falling on your your office. What happens if a tree goes right through your hard drive? What if someone breaks in and steals it?

big files, big zips. Yeah. And and it's it's largely because these things really specialize in holding a lot of data, at these these companies. Basketball Because of that, data has gotten so cheap that now we can just toss a whole bunch of stuff up on the on the cloud JS really what it comes down to. And it doesn't need is go I was like, oh, this is taking up so much space. I I must have just recorded a very long time. I I didn't even look at it. I just deleted it. I was like, why not? If you were to pop that on Cloudflare, You have an egress window at your house that's for leaving in case of an emergency. I got one right here. Yeah. So egress,

not a show about image compression, video streaming, things like that because what are called object storage or blob storage. And, essentially, what that means is it's a spot to put Google Cloud Storage to hold on to that file.

and that server will have space that you can upload files to that will run your application. Right? And they're often limited in terms of both how much space you're allowed to upload things to. So they'll give you, I don't know, like, 20 gigs or or or 10 gigs, something like that. You can upload your whole application there, and often you can put files and and images on there if you want, but the bandwidth for those things are very expensive because both, like, streaming products, a cloud provider.