Spooky Web Dev Horror Stories - PART 1

the entire

Oh. That always scared me. When people, like, move, like, oh, we're gonna launch our new website,

as CIO.

of spooky stories coming for you on Wednesday.

gosh. 3 weeks But sometimes,

Damn those infinite loops.

specific objects back, it catches it and says that's not a POJO,

that I had deleted all of their email records. And it turned out that because I had moved it from their hosting provider to my own, and then their hosting provider deleted all the email addresses in the cPanel. And I said, oh, crap. And I was I was sweating for a couple hours, and then I realized

Spooky.

So this is this is pretty big. I was in charge of back and front end, I was supposed to connect to an API to generate coupons and send them to email to people

Sometimes, Wes, you just think,

which changes all the DNS management. And if you have anything else on your domain name, like MX records for email or, SPF authentication That's a good way to put it. Yeah. Assume

Alright. This next one is, from the founder and ex CEO of GitHub. This one is straight from Twitter. It was a gorgeous Sunday morning. Birds were chirping and squirrels were swirling. Coffee in hand, I began upgrading GitHub's testing infrastructure.

Next one is my squeal horror. During my 1st years as a software developer, I had to rewrite a login register form for one of our biggest customer CMS portals. Part of it was the password forgotten page Wes I implemented a whole submit your email and reset your password flow. I then went on to write a nice little SQL query

Spooky.

were stories where this had happened. Someone forgot the where clause.

yeah, that is a a fear of mine. So I'm always always be backing up. Yeah.

to reset the user's password whenever they requested a reset link.

circular references better. Man Oh, man. Have you ever logged, like, a giant object and even your local host is, like, struggling with it? Yeah. Yeah. It chokes or even, like like, some things if you try to JSON something, like in SvelteKit, 200 times within a few minutes on both channels.

When I was done, I ran a quick test and deleted Everyone who was around helped out, but our Dogek

references like, objects aren't just data. And if you try to serialize JSON stringify

Wes, deleting and restoring data regularly, and 4, it can happen to you. Yeah. If, it it is funny. Whenever I I do have a database spooky stories. We have rounded up some of the best

people an email saying, thank you for your application, but we decided to move forward with someone else. Most of them didn't even apply.

but I have started just littering my code with to dos anytime Bones

Client was mad. We sent the coupons weeks late to thousands of people and got some very angry messages in response. I never sweat so much in my life. Yeah. You didn't take the test that it actually sent a email? And I realized, oh, they actually weren't using the cPanel email. They were using external MX records. So, thankfully, I was spared there.

it's still only local. You know? You you push it somewhere else. Or Vercel Node now has that, like, automatic backups. Even between commits, you can lose stuff. So a scary part of

So the 1st spooky story is Node A Coffee.

Yes. Oh, man. That is actually one of the big reasons why I started.

I got the email, like, 15 minutes later. I'm so sorry, but we accidentally sent that out. Like, that's yeah. That was rough. No. Thank you.

I had forgotten for sending one off emails, transactional emails for spam detection, and it took 3 days until it was resolved. Both times, the client didn't have the credentials to the registrar,

We should also have known our events table of users were affected before Wes rolled back successfully.

who played the game. Supposed to. Interesting.

millions of it sold around the country.

Yeah.

you're writing your own log serialization

Yeah. Scary stuff indeed.

that sucks.

but it was the opposite where they admitted everyone. So I got the email saying I was admitted to Scott up school. This Wes back in the early days of LevelUp Tutorials, and I was so stoked. I told everybody in my my house right away. I Wes, like, really? And then sure enough,

We released the whole thing, and everybody was quite happy, especially me JS it was my biggest project to date. After a few weeks, started getting emails from users telling us they had to reset the password every time they wanted to log in to the CMS.

the spooky stories

3, 4, 5000

in part 2 of the annual spooky stories episode. And, again, if you have any stories of your own, we want to hear them. We will read them on the show next year and share Node delight with all of the horrors

what people would often do is they would just change the name servers of the domain,

Save server upgrades for the weekday when your team is working so they can help you.

I went to the source code to check what was happening, and I looked at my send coupon email function. Oh, no. It was empty.

would be a doozy.

some other piece of information they've from in another field, they've been able to figure it out, but

We launched the campaign, and everything seemed to be going well. Client was happy and so on. After a few weeks, the client called and said a customer JS complaining he didn't receive his coupon.

yeah. No kidding. Right? spooky stories from our community.

Yeah. Careful with that. Yeah. Infinite loop. I once sent the same email and text notification to a single user initiative. Wow.

Yeah.

that the quality of data in their database was very poor.

Man. But the main thing that went wrong, our GitHub application assumed it was running in production mode unless told otherwise.

circular horror.

you got a lot out of that program if you were accepted, and they were accepting, like, 20 people or something.

Wes the server memory would spike almost immediately

The first one is simple. I moved a client's name servers to my hosting provider and forgot to add the MX records to Google Workspace.

so it took a full day to track down the guy who originally set it up.

This is awful because, first of all, that's a security issue, meaning that you could set reset your password, and then your password is then the password for everybody's The the fix was quickly implemented and deployed to reset the passwords once again, and everything was fine. Since then, I triple checked my Wes causes.

And they did that, Yes, folks. It is time for our annual spooky story episode.

scary stuff.

and then make sure, When I ran my test, I forgot to set the test environment.

awful. Awful. For real. Get corrupted. I once lost 3 weeks' worth of work without noticing.

Wes if you forget the where, it will literally update every single thing

Next Node, pnpm company. This could also be pet cemetery. That's a scary Oh, that that would have been a better title. I like that. No. It's alright. Forgot a Wes clause on an update and changed every customer master record to that I'm mid project. Just so Yeah. I do a command f for to do before I push anything

date set in the future on my NAS. I pushed a remote 10 times a day now. This is this is honestly one of my fears is even if you are git committing often locally, logic.

I already did that. I already did that. Is that has that ever happened to you where you're just, like, alright. Now that that's Deno, and then you move on to the next thing, and you're, like, oh, it's half done. You didn't even really get there. That's that's kind of how I work sometimes. Just put a to do in there.

I forgot

ran a second update, and told the CEO

If you're rejecting or accepting people for something important, yeah, be extra cautious. Careful with people's lives.

and we only found out about it days later after his client started messaging him wondering why he is not answering his emails. Oh,

So we're gonna continue That that stinks. I I may have told this before on the show, but I had applied to Y Combinator Startup School, which Wes, like, a really

Those are awful because you can't stop it. Once you send 50,000 emails, you're gonna get

redacted large pet retailer.

Yes. And, like, honestly, it's kind of like a

100 and then thousands and then tens of thousands We're gonna be talking I know people poo poo did to dos. They poo poo to dos,

in the database. And probably we'll we're not gonna read all of them, but we've probably about half of the stories we got submitted

logic with an out of memory error.

if you try to send

Next Node,

He did it again.

So a connection was established to the production database, which was promptly deleted.

It was a holiday.

This might be the spookiest ever. Yeah. And there's been some spooky ones.

the user reference in the where clause.

database.

It was a holiday again, then you can get into some big trouble there, especially when it's

And, And they're awful in the time, but I'm glad that we can laugh about

And I asked how they ever did accurate sales reporting. It's like a master I almost got in my car and drove away. Luckily, there was another field that had roughly the same information, Jeez.

I moved the domains right before a holiday, stories of web development.

I always make sure I click back up to, like, a rollback type of situation. I know that's a smaller scale than GitHub. But, like, for me personally, the server over and over.

The reason I later discovered an automatic r sync backup,

I thought that's weird. Only when I visited GitHub Scott seconds later did I realize how bad things were. Nothing worked. As I explained in the blog post, we always wiped our test DB before running tests. Turns out people don't want their data wiped on a Sunday morning for no reason.

It was my 3rd day on the job

Started a master data management and I need to do something, you know, major on it, whether that's a migration even though it's not that major, but still a migration or something where I'm modifying the database,

there's a couple where we're able to say the company or at least having I made a website for a very big company selling coffee all around the world. It was a website promoting an online game where people could win a trip. In other things, the minimum prize for a coupon was 20¢ off a pack of coffee.

Added a custom ESLint plug in to prevent that from ever happening again ESLint addition to handling

which wasn't excluded due to a system

There was a lot of things that went wrong. Our tests should not have had access to production. Yeah. That's that's 1. Our production Node shouldn't have been wipeable. 2, we should have been able to restore the DB faster.

It caused a reset page. Every time a user resetted their password, Wes were actually resetting the password for all users of the company's account.

It's a whole don't deploy and try anything.

There's something very poetic about GitHub like, you can get into some big trouble. I I once did it where I had moved a domain name, and I thought

to code it.

really saved the day by quickly restoring the DB and punting on the events table. We also immediately locked down production so it wouldn't happen again. My main takeaway JS don't let anything access production except production.

the lady's like, we're still getting emails.

and then restart That's a hell, like, f you in particular, Can you use favicons in URLs, like, as a URL path path name

Man, these are great. Next one is the haunting of the forgotten MX records.

while using version control.

tell you what we think it is.

There was a QR code on every pack of coffee,

type of move right there. That's hilarious. Oh,

account.

Sucks. It those are some of the worst stories we've had over the years where We are keeping everybody anonymous for the sake of nobody getting in trouble.

which is a plain old JavaScript object.

emails back being like, what's this about? What's going on? You know? And even if you send up a follow-up, hey. Sorry. That wasn't meant for you. You're still you just cannot stop it, and the poor support people are just gonna be slammed for days. Yeah.

with my NAS was corrupting my Git folder, an object that refers to another object and it's circular,

Serialization

SQL lost data. You know? Oh my

Yeah. Classic.

there is nothing that needs I know that's a whole controversial thing about that in general. But, hey, I I like to dos for that.

Totally empty.

people had to piece things back together with Oh, yeah. Alright. What do we have? Like, often, we've had people go through logs, and they've been able to piece it back with logs or

don't assume production by default. That's a good tip. That is a good tip. Always assume either that it won't work or that it's connecting to a local or something. Yeah. I remember exactly where I was sitting in my apartment next to Dolores Park. The Wes rang quickly at that point. So when I started my test and it just hung, I immediately knew something was wrong. But I thought it was a connection issue.

That hurts.

I own a domain name that is just a 1 a 1 letter Node, but, yeah, you you can put them. Like, I have a fab.farm,

and serves it up as a fav Scott. So you certainly can.

boss. What's up, my dude?

and you can do forward slash any emoji, and then that turns it into an SVG or PNG

Yeah.

Yeah. So we should do that. We should have a little ghost emote. You could Yeah. That sounds like a annoying that sounds like a fun joke that will be a pain in the ass to have to support for the rest of our lives.

You Sanity a certain ESLint.

or not favicons emoji? We had to deploy for the send money app at PayPal

Emojis.

Be they to get converted into what's called punnycode.

I can imagine it does not look like a phone call.

how it went and also learn a lot about how to, like, not get into the situations that these folks are in. If you have your own spooky story, go to syntax.fmforward/spooky The truest sense of it can happen to you for sure. Next Node we have here is called rejected. I once sent 50,000

This is one of my favorite episodes. We do 2 of them every single year Wes you submit your spooky stories of just the most put your head in the sand 2 years later, I had to close the account again, and I forgot to move the name servers.

and pop it in there. We collect them all year round, And then once a year for Halloween, we read them.

Hey. That's the best kind of fun joke here. Yeah. Let's get into the first question. Some of these stories are longer. Some of them are just 1 or 2 sentences.

I'm gonna have to take this mask off because it's just it's cutting into my eyes.

Spooky.

you best believe you need century at century.io.

Well, we're gonna be talking all about those things in this episode. But if your code is spooky,

Sign up and get 2 months for free. Wes.

people.

Welcome to syntax syntax. On this Monday Sanity treat, we're gonna be Tolinski

Maybe you dropped the database.

web development I yeah. That stuff always freaks me out. I I know so much of it JS not a big deal, but, yeah, I email in general just because I know how to Never wanna goof it up. Yeah. Between spam and Node spam. That that's a that's a hard hard hard task there. Man, these these have been well, this is a good good kickoff, Wes. I'm, like, pretty stoked about Yeah. These spooky stories here. Well and and folks, if if you if this is your 1st spooky stories episode, we have a whole hour Maybe you sent a test email to 1,000,000 In seconds, of the things that we do at work all day. Alright. We'll catch you in that one on Wednesday.

spooky stories.