Spooky Web Dev Stories — Part 2

I had built a samples request wizard for an international flooring company on WordPress using jQuery steps and Sanity forms. At the end of the wizard, you submit the form by clicking the finish button on the wizard.

Sign up for the year and save 25%.

homepage

The marketing site was the main login for the app, so support blew up. The company basically stopped. Senior leadership team was doing rounds trying to get to the bottom of it. Because the break happened via fields inside of the CMS, the CMS was down 60 to 70? Oh, my Node. Yikes. I would delete, like, Node development site and freak out. Sixty or 70 production sites. Yay. Yikes.

I made an Electron game to run on in store screens in 5 major brand stores for the launch of a new flagship device.

And every time I hear it, it brings me right back. Yeah. That's very Myspace era music. That's funny how music can do that. Next one is called bad words again. So this is another one about my words.

which is hilarious to me, but has backlight. The the numbers are huge. Most importantly, it reads very fast. So some of these cheap ones you get at the grocery store, What's not a nightmare, though, Scott? To this day, I use 2 database clients, 1 for local database and 1 for production staging, and I will only use a read only access for production. Thankfully, it didn't get me fired.

Yeah. So, oh, that is so funny. That is very funny. So for those of you who who aren't picking up, basically, they had a hook on DB change that would send an email or something,

they do, like, gift cards and stuff like are not great. Like It's like you if you get invited to, like, a birthday party or,

the charts were missing data from

I've since

to know that you could write some software that that would have that kind of consequence.

create a new empty site.

lost revenue from ads.

I actually have used that JSON file before. There there is a file on the web of every bad word you could possibly ever imagine.

So I sat at my desk drinking a glass of Prosecco and ran r m r f r m Wes you can sign up to become a pro member and gain access to a new tutorial series every single month along with our entire

Have to check that out. The API looks similar to Lambda instead of, Express like, which is kinda cool. It's it's not that much different, but Yeah. I just noted that. Cool. Shameless plugs.

Oh, man. I'm just looking

Next 1, $1,000,000 scramble. Hey. You know what? I have, like, $1,000,000 scrambled eggs right about now. That sounds good. These aren't scrambled eggs. Okay.

by Sysrix

Wes Yarn of why people aren't good cooks is because they don't know when something is cooked all the way through, so they overcook it so they don't die of salmonella poisoning or or something like that. And you can for, like, $15, sites probably would have hurt a lot of these, but then we wouldn't have the show every Halloween. So

Our deployment got deleted about an hour before the demo. Oh. It was the maddest scramble I've ever seen to try to recover. Yeah. Yeah. Yeah.

Oh, it Wes not so what the what's kind of lost here is that,

Wes, 1,024

the the the word is is actually spelled out. They're not shredding d. No. It's the same word as the sporting goods store from earlier, which we were allowed to say.

last week at my last job, and I was due to start my new job. I worked a 12 hour day to keep up with all the handovers, etcetera, so colleagues could have a chance to continue working on the solutions that I had taken care of. Node project was a process tool

Fastify was the one that's kept on popping up over and over again for me. And I started looking into this, and it's a really neat server. So I will see pick this. I've been given it a test out just to see. There's some really neat GraphQL stuff in here too. And they basically did some neat little, a pub at 8 AM to celebrate the job well done. Yeah. If if yeah.

retail chain,

version control and things like that, where if you if you deleted it, it's gone.

Sadly, he pushed it to the wrong branch.

Speaking of deleting production,

Alright.

shot that went 2,000.

I don't know if you've done any Yarn linking or Npm. I have. But in a lot of cases, if I have to edit a like, a module that I'm using, I just go into node modules and start hacking away at it and then start hacking. Yeah, it's usually you're working on the bundled version, not on the main version, but it's it's a pain.

I am going to shamelessly plug all of my courses Wes,

or next story.

our 1st ever demo of the product, and it was I come to find out that I had built a race condition that sometimes made the page change without submitting the form.

She had called him in a panic at 1 AM. Oh, gosh. That's scary. Turns out, because of a time Node error, So don't do those things, and, make sure you take good backups. Man. Good backups.

yarn install, and yarn link again. This, on average, would take at least half an hour out of each dev's day. Oh, and another nightmare that there was was there JS no hot reloading for the sites. Every time you made a change, even if it was just a change of margin and a SaaS issue, it would take 40 to 50 seconds to rebuild.

Last time, I I swore, wrote witty, despising push notification messages. I think that's sort of a trend of these

Wes? No.

Man. Yeah. You never know when people are gonna see it.

I was in my

So glad I wasn't in the office. Tester forms in all browsers at all network speeds. So the form was firing an event early and submitting the form without collecting all of the information.

Turns out that anytime anybody clicked on the chat widget, even if there was a friendly message

Again,

the SEO

which is my new website. Check it out. It's forward slash courses. Has a list of all my courses, most recent one being Master Gatsby. Make sure you use a coupon code syntax for $10 off.

server in here that has, like, data loader essentially built into it. They make it, like, part like a first class citizen, and, it really solves a lot of the the pain points I was having personally with DataLoader. So I'm into this. It's very cool. Sweet.

in a WordPress template

going out from my CMS.

requests a second more. So, obviously, this is their benchmark. So, you know, take that with a grain of Scott. Do your own benchmarks. But Fastify, for me, has been not only very fast but very easy. The logging, all the stuff, set up the plug in, the whole environment has been very cool. There's a lot of neat plug ins involved here. So check this Scott. If you're looking for a Node server,

So for every email sent to us, 2 more notifications were generated and also sent to us by email. This email storm ripped through our office like a category 4 hurricane.

you gotta leave it stuck in for Next one is bad e n v. Not sure if that's the kind of story you're after, but in a company I work for, we had an absolute madness of packages with weird dependencies

combined with consonant pairs one Deno. So nearly double the amount of requests per second that it was expressed was able to deliver with the same Node. And same with, happy. Koa was a little bit more, but Bestify was still, like, 20,000 You think dicks is in there? I they probably had to do a custom. They probably had to to tweak it.

Once in my career, I needed to create an FTP user for a client's website.

and nobody was able to pay.

temporal email or something just because if you accidentally trigger

And so at any given point, it'll tell you, Node, use the current Vercel, whatever this is, the past version. Any given point, if I have a mistake, I just click the old version and click, you know, go back to this Node. And I can go back to any prior Vercel, and it just redeploys it for free and, like, instantly.

Next one is just called yikes.

using SendGrid.

JS well. A senior eventually had to go in and manually change the values from the database to get things up and running again.

If someone bought a product from one of our clients, we would send them an email asking them for feedback and a review. In other words, we spam people with emails.

catalog, which is constantly growing. And let me tell you, I am really excited for the next upcoming year. We have 3 or 4 guest teachers lined up that you are all going to be very excited about. I can't talk too much about it just yet, but we have some guest creators coming on. I have some new courses coming out. I have one that just came out on Svelte animations. That is fantastic.

disaster.

a session replay. Now what is a session replay? Well, it's a video scrubbable video that gives you the network tabs. It shows you the user's mouse Wes they clicked on, how they, were able to do all this stuff. So this way, you would have seen potentially somebody firing off this, Gatling and seeing that it was your mouse. It was the the the killers coming from inside of the house, that whole thing. So you'll wanna check out LogRocket at logrocket.comforward/syntax, So I've been diving into the world of Node servers lately because I wanted to see, like, what's changed since the last time I looked. I know a lot of people talk about Nest Or you just had a funeral for a family member a month ago, and you send out a please attend TED's funeral email to everybody again. Oh, that one is very painful, parent repo is like a mono Deno, and then you're just, like, asking to,

assistant and title.

Express and Apollo

Woah.

Yeah. People people don't like that, especially any sales teams that need to keep track of their their, their people. They don't like that. I don't like it when you you take away their leads. Like, huge loss in sales. Like, sometimes I submit a form on a website that's, like, a local business. I'm like, I'm never getting a reply to this. And sometimes you feel that way. Sometimes you're filling out these, like Yeah. Junky

Nobody tested data integrity.

which company it was, and they said it was fine. This one is called

Like, for instance, suppose you have your out of office reply on and you send an email to someone who

if you're on the barbecue or something because you key you have to keep your hand on it, or you got this plastic thing in the way of the heat. So go get yourself a nice thermometer. I have submit the form was just window Scott location, and they didn't await for the submit to come back properly.

Yeah. Not great odds for the first and only required clicking a button. I made sure to test my code more thoroughly from now on.

Is it Netlify,

Oh, no. I found the I found the exact one that I have. It's not called mister Shepherd, unfortunately.

Next Node, FTP.

3 minutes later, we all start getting alert emails and calls saying someone Wes trying to take our servers down. I also didn't realize it was me until

Yeah. I host on a Meteor Galaxy, right, because it's the Meteor host. And they have, like, just like a history of every version you've ever deployed.

But anyway, we used to do beer Fridays, but I'm a woman who doesn't drink.

a

The site completely

so if you wanna be a good cook, I was writing a Microsoft Word environment. Umpteen million pnpm errors appear appeared, which is strange because there should only be about 20 commands.

So I went to the 1st store with the crew to oversee.

I actually thought this was pretty slick at the time, and, yeah, that's pretty sick. We had just deployed a change to run a bunch of DB migrations Yeah. This is a great use case for delete your old stuff if you don't need it. No kidding. You can get it. You can get it back from requests per second

orders we had without a backup.

there is even an identity platform that allows you to create a login user account on your front end code site, all just through Netlify itself, all this and more. So check out netlify.comforward/syntax Hashtag ghost.

you can go get a really good thermometer that you just poke into your food that you're working, and it tells you what it is. And, like, I feel like it made me a better cook when I got a really not not necessarily, like, a really nice version because there's, like, thermal pens that are, like, $100. But I got this one. It's called mister Sheffer,

for a deal

extra work, or maybe you have to bring in more people and not be able to pay them. And, Every single word backwards. Woah. And so example, dear customer became

Mister Sheffer will be coming to, mister Tolinski house at some point because I I said that at some point and, like, yeah, I I do need to get one. And and just, like, it was, like, 2 days ago, we were cooking some pork and we were using the stick and we got a way for it. And it just is, like, you're looking at it. You're like, I don't even know if this is accurate. Like, I have no idea. Pork is the best one because at least in Canada, a couple years ago, they changed and, it will roll back that change for you, which I thought was was awesome. That's neat. It's sort of like that, you know, when you change the resolution That's it. It's like, do you want to keep this resolution?

That's I wonder what that was. Do you think that was like a They still owe me some money.

pnpm pack a library,

32, 64, a 128,

downplays here.

Fastify dot io, The function had roughly a 1 in 300 chance of including the c word somewhere within the generated string. 1 in 300.

and left the also delete user home directory.

and fixing it. Wow. That's framework.

I was absolutely shocked. It turns out the back end team was using this to filter out search requests that had bad words listed in the file. I couldn't believe what was in this list was loaded into the client side. What if somebody found it? Well, a month later so I didn't have absolutely nothing about it.

Big big big gas. Alright. Last one we have here is Adidas.

Pure, live edit the database style. Again, this is my 1st day moving into engineering. I was mortified, and soon they were going to boot me right in back into Design Forever. Yes.

The site is gone with lists and everything. Lists are like a SharePoint thing, sort of like SQLite.

It was actually funny. A little bit ago, you you started one of these, and you're like, you know what? I thought you're gonna do a Netlify ad transition, and then you, like, started reading the next story.

There are no backups of the acceptance environment, although that is very important. I feel just a little bit panicked. How am I going to solve this? However, I remember testing a tool 6 months ago to copy entire environments

Thanks, Jack, for, recording this for us, by the way. It's, fantastic.

It just cut cuts over the domain. Now, like, Netlfi's, I they'll all do that as well because they just have multiple versions of your app Yeah. A lot of these people never deploy on Fridays. Or have a, like, a good set of tests would have or a pretty basic test would have caught that. I'm sure they have a test for it now.

just because it's scary.

on Fridays. I work for an ecommerce agency at the time. I made some small updates to our site. I forgot exactly what it was, and I thought I'd update the project's dependencies at the same time. So I made an update and deployed on a Friday afternoon. I know. I know. Yeah. So this is what I refer to as a YOLO update, where you're like,

I once left some debug code in when checking an error for a user that logged everybody into his account.

Litigious. I was trying to debug an error on our staging server, and anytime I touch the files on the server, the changes didn't seem to be

And I needed to see what was in the notification, which triggered when something was sent through the API.

Snowpack right Node. And I'm just I'm very looking forward to the future where I save something and it's just updated that next instant.

Those ones where you can screw it up by accidentally typing the wrong thing or hitting the wrong button or running a command in the wrong directory.

has an awesome UI for this, and when you hit save and it makes Bos possibly a breaking change, like, you change the the the URL of or you change the firewall rules or something like that, it will save.

Yeah. Definitely check out Darknet Diaries. I really appreciate it.

And, someone got angry, unsightly women.

We managed to recover 99% Next one we have is never

I think this one's very relatable. I've done this. I don't know if I've done this. I feel like I've done this in WordPress. I've done WordPress. Maybe even Expression ESLint. But, like, there is, like, a lot of specific situations when you can edit something in a WYSIWYG and bring down the whole site.

See, our email servers were hosted on premises in our offices. So not only did I knock out email for everyone in our office, but the exponential growth of inbound emails

where Express with the same code was only able to get If you block Google's crawler, it will not deliver content related

generate myself a coupon.

So setting deny all in the robots dot t x t did not only lead to 50% loss of visibility in Google measured Also, r r m r f is just so dangerous. Didn't you have something where you use some of the the I feel like you had this picture? It's trash or something. Install resolver and then one calls the other, you could end up in a situation where you're getting, like, 500 database queries. Because, let's say, you have 20 tutorials. Each tutorial has 20 videos in it. And, like, the way your system might be set up, it's not gonna do all of those requests in Node fell swoop. It's gonna do the 1 and then the individuals and loop in and whatever.

We were demoing a product Wes were building to a potential client,

reminiscing about how much I love this thing. So this is a, instant read

But then I got 2 more,

Going strong, so I really like it. I'll put a link for it in the show notes.

That's a lot of problems.

This resulted in the 1st mail containing 1 mail's content. The 2nd mail contained 2 mails' content.

I bounced the server thinking cash, still nothing.

The source of all bad things that happened was the idea of a colleague to remove our testing installations from Google.

Netlify. Isn't that crazy? 1000000 developers? That's a lot. A lot of people using this thing. There is just a ton of features that allow you to really, really gain productivity

right now, check out Fastify Scott I o. It's one that I've been having my eyeballs on quite a bit lately. Cool. I'll have to check that. It looks like they have a lot of middleware as well, which is something you wanna you wanna think about if you're picking a new server. Like, you probably are gonna need a bunch of plug ins or middleware. It looks like they've got hundreds of community ones, so sick. And there's some that, like, Yarn, like, make not only GraphQL servers easy, but also make I don't know if you've ever gotten into DataLoader.

would create a string, which wasn't a real word, but could be pronounced and therefore would be easier to remember.

it was hundreds of words that I simply cannot repeat on this platform.

yikes as that you've gained from us? I just keep thinking about the one with the 500,000 and bypasses

Company was freaking out. I won't say for certain that this was the final nail in the coffin for the company, but it certainly didn't help.

Hey.

where I am very happy that my job is as low stakes as it is. I can spend a month working on a course and I get that course out and I get I get a chance to review and watch it 800 times before it goes out.

I don't know. That's that's a good question.

Europe's largest cooking community by accident. Oh, yeah.

and their crawling script just fired off against every record.

man, People from branches just called. We see people from different cities in our system

you should probably check this. I'm like, We did it. Not long after, I'm informed the QR code was a placeholder,

You can get it back. Node you need to. Cool. Next Node, always be closing. When I was in my twenties, I forgot a closing table tag in the mail,

and the potential big ones we had on our list. Big day finally comes, and I finally get the wallet address just before launch.

One time, I was working on a client site and running tests to try to debug some email template issues

fire on create save events.

like crazy. I host my site in Netlify. I know Wes host his site in Netlify, and it is fantastic.

to for us to anonymize it because they're worried about the government getting litigious.

and see what everyone's talking about because trust me, this is the place to host your front end code. It is so dang easy and simple. You know what? Now if I have it, I just checked. If you go to one of your commits

to recovering was from 4 PM to 2 AM with 4 engineers.

and costly bug. Yeah. That is frightening. That's very frightening,

and I would be like, what? Like, what's bad about fruit cocktail? Launch day is coming, and we're preparing for months. The smart contract is already out there on the e Ethereum network. No going back. Our website had a countdown and everything for all the investors we spent months acquiring

So that's, like, under the same I'm in a sick pick, something I've sick picked in the past, but I just got a a second one of them. And I was just

Interesting.

stop Stop putting bad words

I'm very into the name of mister Jaffer.

500,000

large queries like that from being a problem. It's basically a caching solution that will make, your n plus 1 queries way more performant, like, exceedingly more performant. And so there's a really neat, Be very careful. Don't push on Friday. Any other parting pieces of advice Sanity. Spicy.

soon as you send them an email, their auto reply sends you an email right back saying they're out of office. But wait. You also have the same auto reply on too. So shouldn't your inbox respond saying, hey. I'm out of office too. And then as soon as that email gets sent to them, their responders comes back saying, I'm still out of office. Why are you trying to email me? And so on and so on.

to ease things a bit.

Like, that's very scary. You want me to tell you a good UI for this JS I've done it on WordPress before. You change the site URL on WordPress, and then you save it, and then it's just white screen. And, like, you can't you have to go into the database to fix it. Yeah. My router

emails.

that was wiped after a database cleanse. That was a bad day. Not sure how I get my job.

See, every time me or that other developer would get an email, SendGrid would trigger that notification email and tell me that emails were sent successfully.

JS it just fruitcon? No. It just it just, like it was, like, matching part of the word. Oh, it did. Wasn't it wasn't the software probably wasn't smart enough to I thought I Or maybe it's because I was typing it. As I was typing it, it realized the word, and I hadn't had time to finish it.

What is Evite? Do they oh, they do, like,

Sometimes I just buy things based on the name, and that is a great name.

and the countdown and excitement.

but not yet being trained to pay close attention if I was looking at the site .dev rather than site .com. I swapped them out on prod.

I find the clone environment. It can be used in the same tool to clone it back. It only took 8 to 12 hours of work to create all the new things done in the environment in the last 6 months instead of the x number of hours to rebuild

Alright. Next 1. 1 in the 300 chance of the c word. I once wrote a pronounceable password generator. The theory being that it generated passwords made up of random vowels

So I wrote a webhook for SendGrid to send a post request, and that would forward the contents of the post request to myself and another developer

Because I do, and it's it, like, saves my butt all the time.

yeah.

I used to work for a Scott up company that had an ICO back when it was a thing, TM. So ICO JS initial coin offering. So when, like, a new type of crypto comes out, they have an ICO where people can buy in.

Let's get going with the different stories.

worth 1,000,000 of dollars. So we got a $1,000,000 deal here. A a DevOps engineer just so happened to be going through what they thought were our old clusters and deleting them.

and some engineer found the deleted page and readded it in crisis averted. My boss did not fire me.

In almost all of these, we didn't actually mention the name of the company or the Vercel. It's just that we get no one in trouble. But this person, it's kind of important for eating pork I set it for, like, a dollar, and then I buy the course myself just to make sure it actually works with a real.

I had at the cottage. I left it outside. It poured rain on it for days. I dropped it off the deck.

Next 1, push notification hell. A new to me codebase was working on some push notification issues,

We have new course every single month. It's sort of like a magazine subscription. Check out what the latest course is. So level up tutorials.comforward/pro.

on Fastify,

Evite.

using active model. We had a bunch of callback methods into models themselves that would

they'd left to this bit that deletes the user's home directory, but they It's when you have a app. App? You deleted the FTP user. So what did it do? It, like, went to the app then or something? When you add a FTP user, usually what it does is it gives that new user its own, like, subdirectory. But he created an FTP for the root. And then when you delete it, it says, hey. You're getting rid of this user. You also wanna delete their files, but it was the root. I got rid of the the root. Gotcha. Seems like there should be some production for this. Yeah. Rough.

out a few days later

It was all out at the office.

This script failed to skip the app permission, the callbacks, and basically

forms. That was rough. Like, this is definitely a waste of my time. A perfect example. People always ask, like, what's a race condition? Like, that is a perfect example where they hit submit, and then the the the code after updates to my package, Jason? Sure. It's fine.

right after that. So who cares about how much, quote, unquote, money we lost in those few hours? Mhmm. The company went under the next year before I left.

Yikes.

So that account has been compromised, I would say. I think that's safe to say that account's been compromised.

So advertisement for DICK'S. Shout out to DICK'S. Alright. Next question. CMS with over a 100 people, and you're waiting to announce until 2 weeks before. And all of a sudden, a bunch of emails start going out to everyone attending.

Back in 2005, I worked on a web app for a DVD So that that is much funnier to me that it's,

store names in one of my courses, and it just takes, like, an adjective and a couple vowels.

saw that something was wrong on Monday, and we could fix it very fast. But it burned lots of money, and I was in fear for a full week until we figured out that it was a good accident.

in words and alerted banking HR managers that they were saying ass and t I t when they were writing or you put a you don't put the window location in a callback. You're pooched.

on the back end and using React with either JavaScript or TypeScript and hosting everything in Azure 99% the website was there.

all lowercase letters, v 61. I don't know if it still exists. It looks like it's closed. But this was like a music streaming app where people would

They had to really fork it and release their own bad words, that JSON.

Yeah. I know not to deploy large changes on a Friday.

and, like, it comes in over the email. It was pretty popular, Oh,

Yeah. Let me describe to you a nightmare. Here's a nightmare.

if my business partner

Or you are a bride planning a wedding and you send out a bunch of emails to your attendees ESLint prematurely.

which had contained the c word. And then he wrote, yes, that c word.

meant to log in to a server by SSH, run 30 commands, and check to see if everything is fine. I remember those days, Wes. I remember those days.

12 AM to 1 AM.

Alright.

and that will put it in your trash instead of r m. I never use r m r f, things like that. And he just, like, tells it. My favorite ones are the pen testers warp people, like The pen tester interview. Into, like, real world situations, and it's just an awesome podcast. And he agreed to take one of these stories and read it, and, like, he stayed true to the story, but he just the way he embellished upon it and, learned not to run a process like this on a production machine, and now we all use GitLab runners with a scheduled task to anonymize.

for a full archive of all of our shows, and don't forget to subscribe in your podcast player or drop a review if you like this show.

Yeah. I don't think CSS right to left ESLint? Does it reverse the No. I don't think it does. Well, I guess it does. You know, I don't know.

recursively

We set up all the plug ins and then imported the content years' worth of pull posts. Unfortunately, we didn't notice there was an email subscribers plugin

your generator. That's pretty good. Scott some I have random, There was no visible problem on the website, so we all left it for the weekend.

for each go live because, you know, it always works. Yeah. This this kind of happens to me too. Like, you get complacent with things working all the time and that you lack your guard a little bit and something breaks.

I was working on a few subtle UX changes to the widget that would give users a much more intuitive interactions in the behavior of the chat.

Sure enough, a month later, I pop open Reddit, and I see trending on our web dev, Dick's Sporting Goods, bad words Scott JSON. It was an interesting day

About 15 minutes later, it dawned on me what I did. Thankfully, the server host had a very decent backup. So so what they hyphen r f to, for those of you don't know, that command removes

spend for ESPN and YouTube

and I needed to test SendGrid's email API.

So thank you so much for now at Netlify.com GraphQL

Dirty Dick's JSON, which is from Dick's Sporting Goods. I worked for Dick's Sporting Goods for several years. While I was there, the structure of the engineering tier was a bit odd to say the least. The back end engineering team and the front end engineering team were completely separate teams. They didn't have any direct interaction and didn't even touch some of the code Bos. Due to the separation, my team JS front end engineers often had no clue how about the back end was actually set up. It sounds ridiculous, but it's true. Well, one day, I was monitoring XHR requests on the website and saw an odd JSON called banned words Scott JSON

when it comes when and where back end Node sends emails. I hope Luckily,

All day, I delete a site.

for this GraphQL thing to find that it's, like, So, if there's any major appointments, don't start deleting stuff. Just don't. Just wait. Just wait. You never know what you're deleting. Know. We literally probably had, like, 3 or 4 stories like that so far.

As you should after any deployment, I went ahead and checked the site for any possible issues. Searched for some products, added it to the basket, went through the checkout, all seemed well. I go through the test a second time just to be sure. I deployed on a Friday after all. Better make sure it works. Right? No issues spotted. I closed up my lap laptop and headed home for the weekend. Monday, I had a Mailtrap installed locally, so no real emails got sent from PHP's mail function. Okay. Okay. So good. Okay. Okay. Unfortunately oh, Props to whoever wrote this. This is very well done. Happy chappy. Can I see you in the meeting room, please? Oh, no.

76835 It was used during the password reset flow of a web app whose demographic skewed towards very non tech savvy, many of which would have problems even copy and pasting.

the fucking this is what you're looking for. This person may have single handedly killed Evite.

shopping list for my mom, That's awesome. I I just associate that music with learning to code Myspaces.

and this this sounds like this was done

Oh, gosh. To this day, I'm hypervigilant

Finally, my office make mate goes, hey. Did you know the whole website is down? Which website?

It worked by motion sensors placed on the wall of the house. I pushed a change to the charts that I thought fixed a bug. 2 days later, we've received a call that a woman was convinced her elderly father had fallen in the bathroom.

support, The first thing the script does is force delete the site and then try And that's too hot electric toothbrush.

r e m o s. It's it's like Scott just, like, customer dear. It's the actual word was backwards. Remote suck read.

Luckily, that doesn't happen because someone must have made a mistake at some point in the past.

I have no idea.

500,000 concurrent problems.

Installation happened the night before,

Next one is the Oh Node Hotel. Oh, Node hotel.

This stops certain cards from being allowed to pay. No one noticed for a year.

all that after a brutal 2 day launch. All of those 3 days, I listened to David Bowie's album, Let's Dance, 70 times. Let's dance.

My worst production related and you'll get 14 days for free. Again, you get a scrubbable video replay that includes not only the video of what happened, but also The mail servers of the multinational Write tests. Do not run our MRF unless you are very, very sure of it. Although, I did that once where I ran git clean in a non git directory and deleted half the computer. I talked about that last year.

What about

I arrive at my desk. The mood in the office is more

Hi. It's it's days like this

meat thermometer. Really, not necessarily just meat. I gotta get Node of these. Like you said this last time I sickened it. And

So DataLoader is like a caching mechanism that prevents

Deployments had been done in pairs from there on out. Never deploy on Fridays. Oh.

On the launch day, I discovered that the central d b sync from the client d Bos was

I'll make sure that I'm using, like, a mail catcher or

pandemonium.

by email

The crypto craze died And then if it doesn't get a ping back from the browser within a minute, it will revert the change back because, obviously, you went missing to the browser,

firing off emails left and right for every single event 10 d in this system. Oh. Shooting off emails.

to 145.

I accidentally mounted an endpoint in production that pointed to a staging route for hotel reservations.

working on Magento.

I notified everyone on the team that I was just getting Wes messages out. Shoot. Not a big deal. It's a tech company, and

That's the whole nightmare. It is very hard, and you never know if what you're doing is working or not working. It's it's very frustrating.

make yourself very frustrated.

15 seconds before it actually reads accurately.

and there was a difference between the local setup and the production setup while doing that. I sent a dozen increasingly more frustrated test push notifications.

and one of the things on the list was fruit cocktail.

There are definitely Wes things to read from others, but the worst thing in my career was to block all search engines from

JS, not Next, but Nest.

That's is don't ever write a swear in any of your console logs or images or You just never know.

I of these using email logs, spreadsheets, and bulk inserting. The process of deleting Oh, that's that's even why, like, when I use, what do you think? Like a CSS right to left?

Do you have any rollback set up in your in your deployment process? That's exactly what I do. Yeah. I just had to Node because yeah. Especially

I deleted the production database thinking it was my local because my database client and prod local looked the same. The most recent backup was at 10 AM. I deleted it at 4 PM, and there was around 600 Yeah. Like, what yeah. Yeah. How do companies like Heroku or you know, how do they deal with that stuff? I don't know. That's that's a good question. Like, Netlify does it as Wes, or they, like, generate random

I was on Winter Park Mary Jane, and I had to ski all the way down the hill to my car to go get my computer to fix a bug was in production at that very moment that I had gotten an email about while I was on the chairlift. So, yeah, shout out to you. I had a similar situation. Man, I this one, I thought about all weekend. It's just I just kept thinking back to the poor guy sitting in the back of a cab that's probably just sitting there running Doing good. Tired. Tired style. I had meetings because I have meetings now. I don't know if you know this. I have meetings because there's people on my dev team that I we we meet. And so I've been just tucking into a microphone all day. Just meetings for the rest of your life. Yeah. I know. Right?

The production website. It's 404ing everything.

very, very fast. Okay. So here here's how fast this thing is. So on their benchmarks page and this is their own benchmark. So take take that for what it is. But according to this, they have some code here, and they got

that emails a subscriber every single time that a post is published. So this is pretty common. People say, yeah. I wanna get an email when there's a new blog post. There's a plug in. You throw it in, and it works like that. So when we imported 100 plus posts, it ended up sending an email for every single post to every single subscriber.

I had used was n t.

instead of one where they You can take different update it. Yeah.

My first job out of college was at a prominent review and feedback management Scott.

hosted in SharePoint online. The guy who would oversee it had negative 1% experience with SharePoint, which I pointed out to my bosses. But to make things easier, I made it a JavaScript I log in to the environment and double check if I accidentally entered the wrong values in the script, which looks okay according to me, but I get a four zero four error when trying to reach the environment.

And every time I wrote it, this huge thing would pop up on our my computer, emails, that's kind of a nightmare to come back from. Kind of a nightmare. Yeah.

I wrote a bad regex to check for bad words in a Node, and it alerted the user humorously So you just ran the, like, probably the largest SEO test Oh, no.

Certainly, my most shameful

dash g trash dash CLI

LogRocket for sponsoring this spooky episode. Oh,

Accidentally deleted all the accounts and then deleted the entire page.

Next Node, production target. When I joined as a new grad, I was asked to learn about load testing and load testing our staging servers. I learned about Gatling,

moving on from the server plus client apps required by dial up to a single full time online app.

in those environments are just so high stakes. So that's what I So high stakes. Obviously, like, backups and automated testing lots of money lost or potential for money to lost. I still do. Like, when I launch my courses, I still even though I I test on Stripe all day long, I still

I fell in, and I can't get up. Alright. This Node this one is, yeah, pretty rough. At my 1st job, I worked for a company that built a system for monitoring the vulnerable folks in air homes.

to because and you can view that commit, it's already built. You can just publish it. You're gonna roll it back if you accidentally

let's deploy those changes Good tests.

all directories of which you specify.

That's very funny. I have a a a close relationship with Dick's. I think I've been there a bunch of times because I bought all my workout equipment there on Black Friday. So I've been I pop into DICK'S all the time and and, get plates or that's a cheap workout equipment there. So shout out to DICK'S Sporting Goods if you are looking to buy a squat rack on the cheap. Ours has held up really nicely.

certainly not a nightmare. Wes, I don't even know why you put them in the same sentence.

Alright. Next story is log in as One expensive race condition. This just happened today, so it's fresh in my mind. Oh, fresh one.

So you've got me. So ESLint,

you, like, do an image tag incorrectly, and all of a sudden, your support team blows up. And, like, you're you're causing

emails just showed up in my inbox and so on and so on. It only took me a minute or so to realize my mistake. But by that time, my mistake had triggered more than 75,000

Yeah. I got a final written warning and confidence destroyed.

Yikes.

to each other.

like, a wedding, you can RSVP, screw it up. Hey. Cool.

notification inviting them to join a conversation, it would just disappear completely.

Buy the course Vercel. I refund Vercel. Whatever. Yeah. I do that. R m r f. My Halloween horror story. Friday afternoon in the office working on a Magento site, frightening enough in itself. I love that note that, this author added because it is frightening, and had generated thousands of passwords before another dev received a new password, That's great. And let me tell you, you also will not get fired if you use one of our sponsors. I'm talking about LogRocket because LogRocket allows you to find bugs in your application very easily and very quickly Pretty yeah. That happens. Next TypeScript.

Spicy. That's a spicy one.

another bad word. It seems hilarious that so many people have made this very same mistake.

get loaded in. So I opened up that bad boy to see what was inside. And to my surprise,

So please continue to make major mistakes. Of course, none that will result in the permanent harm of anybody. But if 800 people get the whole sent to them in their text messages, then I think that's probably okay as a as a bug. That sounds that sounds pretty good. Keep sending us those. Oh, that's great. Alright. Let's move into some sick picks.

Yes. So this Node, horror story. I wrote masquerade, a com command line tool to anonymize databases with. In one of the earliest versions, I did an array merge, but got the order of the arrays wrong. The content of these arrays was database connection config,

our sponsor for this episode? Netlify is

If I do, by chance, happen to release that course with a a tiny little typo in it, that is not a problem of the magnitude of any of Node that we have read on this show. I it just One of the consonant pairs Basically, if you were having a party, wedding, whatever, you wanted to have a digital experience, you would use us or a Paperless Post.

It was in production for several months

like, 5, 10 years ago. Did you get my Evite to to the Halloween party? Did you get my Yeah.

Turns out, it wasn't just me that was getting the messages, and the entire team That's why we need

I'm gonna shamelessly plug level up tutorial.com You can do all sorts of things like serverless functions

So you could forget 1108

Luckily for me, rolling backwards is instantaneous

Wes my first intent was made here in the acceptance environment.

c word into your tanked.

Google started to crawl the new content that was better than the old one, and rankings had been higher recursive information, I suppose. Possibly leaking The last step is to replace some global Yarn, the local points you did the dev URL instead of prod URL, and millions of pages removed from the index, but also ing 2 local PCs, dumping the databases through MySQL admins, and compiling the central DB manually, and I'm talking about Fastify. Have you heard of Fastify?

Express. There's Koa.

Now imagine the horror when you're in the middle of America planning a surprise birthday party for your father's 50th Next story, you've ruined the surprise.

Node little hashtag there. I spent the day VNC

dust the office's Internet connection for the rest of the day.

and the day that we started to implement automated deployments with security checks that prevented some of the possible mistakes.

I probably shouldn't have that possibility, but it's also random. So

Yeah, I know. I'm spending some time implementing AI. Well, we need it.

And after I figured out how it works, I ran my Gatling load test script. I think I set it about a 100,000 concurrent users.

websites.

catches the form finish event, submits the form, then location Scott h ref equals thank you very much, page.

that's real, man. I can't even laugh at that one. That's real. I'm glad that Right. I know. I'm like, that not only was it the internal team, but the entire user base.

the network tab and the console and all sorts of fantastic things to help you find and solve those bugs.

online, and it's not called mister Sheffer in the States. Oh, why? Why did they do that to us? Why would they possibly do that to us? Can I get the Canadian version? The American version comes with a bottle opener. I would rather get the mister Shepherd than the one with the bottle opener.

So what am I gonna pick today? At this point in the company, we had around

Those are probably the biggest pains to me is, like, when development of the time, 1% SharePoint.

In the good old days,

I log in to the admin interface, and I discovered the site is gone. Also checking the trash can, there are no things there. Very strange. I find that I'm in a different folder than the one where I saved my script. In that folder, there is an old deploy script that was used when the project was started a 1000 years ago, which was not used after this project was finished.

that's such a good word.

takeover. This was back when Facebook apps were big and there was no tiered permissions.

of the time Yes. Which is hilarious. I'm glad it worked out. Next one, I call it just oof. I couldn't even think of a name for this one. Horror story. I'm surprised you named all these. I I named a whole bunch from last time. I did not. I didn't get a chance to name any of these, although I think it's probably for the better because your names are awful.

Eventually, they had to get in the car and come do an on-site visit and fix the problem. And for a few days after that, our offices still didn't have a functional email server.

it's just the mistakes

Command to delete a folder so After a grueling few hours of setting up the AMP stack natively on my machine,

I'm Jack Rhysider, and I hope you enjoyed the story.

attempt to control the web. Not yes. Yeah. Yeah. AMP pages or whatever that is. Shout out to expression engines too. I haven't I haven't used that in a little bit.

They left it in for a year. And how did Node knows for a year? That seems like operational issues. Yeah. Yeah. That's crazy.

500,000 concurrent users, and around 5% of them would engage with the widget. Upon my discovery of this bug, my heart sank into my stomach and a cold sweat started. I had deployed this and then gone into an hour long meeting.

A couple Yarn ago, I was working for a well known company that offers a very widely used website chat widget.

I don't know, man. I don't know. I have no idea on this one. Alright. Next Node, taxi coding.

Starts with the terminal and runs the script warp is slow. You're like, I can code and think much faster, but this thing is getting in my way, and it's I'm so slow and frustrated with it.

My work handled it very professionally or understanding.

the one from a config file and the other from CLI params passed to the binary. I meant for it to have the CLI params take precedence so a cron would run to anonymize a replica It turns out that the update updates I so bravely deployed caused a basket issue that prevented customers from adding more than 1 item to the basket at any time. Of course, I never spotted it as I only added 1 item during my test. The issue was not reported until Sunday.

I think about 400 people made a reservation And if you if you pull up piece nice piece of pork off the barbecue at 1:45, it'll go up to 1 540 or sorry. You pull it off at 1 40, goes up to 1 45 after ESLint, You cut it open, you think that that is still raw. And it's like there's red in it, and it it doesn't look like it's cooked all the way through, but just, like, knowing, like, yes. It's it's cooked. I I'm did the temperature on it. It is safe for my family to eat. And it's funny because some like, I have my, like, parents over there. Like, are you sure that's cooked all the way? Because they grew up in the the age of the higher pork temperature, and they probably went over that because they didn't have

JavaScript library, Yarn link. That's it. That's the nightmare.

385 And we were in the same communities of making songs, and he sent me a couple of messages because he liked some of my music. And then he got really popular and I never heard from him again.

into the web root. Once I was done, I removed the FTP user

Very, very scary. Yeah. Like, that that's Yikes.

Yeah. Or Yarn am I upside down right now? Yes.

or or Deno based analytics, which are actually very, very nice compared to normal script based analytics. Node

I deleted the Adidas Facebook page at 8 PM the night before a $3,000,000 concurrent users Welcome to Syntax. Today, we've got part 2 of a spooky web development stories. These are stories and they use, like, a little mini sandblaster

But this is a story about an email storm that didn't end so well.

D hole? Dancing with the Devil with 5% battery and 3 docker images to build and publish, managed to find And there's, like, Nest. There's Happy. There's It was called the if this is not what it's called, let me, let me Node. The 60 you might not have ever heard of this. It's called the 61, be safe and be careful, y'all.

Oh my god.

and all, I got XPression ESLint up and running. AMP is Apache, MySQL, and PHP, not the Google And it also works a 100% of the time in development because it's fast as out. Right?

I still work for the company 5 years later.

After emerging and deploying the changes, we started to see a couple small errors.

and ours are missing.

got 2 of these now. I've the mister cheffer

offense happened while I was at a Scott up in LA. They were trying to become a more modern

Mister d hole.

So I sent the post request and got the email notification.

That's a huge disappointment, Wes. Very big disappointment. Okay. I am going to, thermometers at that time. I gotta I gotta get me a good meat thermometer, especially a mister Jaffer.

probably about 60

We were having a formatting issue in some of our templates. A senior dev pushed up a change, and we marked the bug as fixed. The next day, we received several frustrated calls when they found out all their emails are being sent backwards.

The poor DevOps guy spent the entire weekend trying to recover it by some miracle.

Like, I could just roll back a commit and and redeploy, and it would fix it. You would have to redeploy it. Yeah.

JS the fastest way to build the fastest sites JS in you can deploy your front end code on this thing and just with a git push. Look at TypeScript and Node git push. Your entire site builds and is constantly deploying anytime you push a commit to a specific branch, all for free, easy to use. There JS awesome, awesome, awesome features that are added to Netlify constantly over a 1000000 developers currently using This next story is Mo Money. As a junior dev in my 1st web dev job, I left a variable as 0 instead of 1 in the payment gateway.

that's rough. Glad that you can find a pub at 8 AM. That would not happen in Canada.

and the team lead pulls me aside. He does not look like a happy chap.

Yeah. Yeah. Yeah. Oopsies.

Pooched.

Oh, we found

broken.

Everybody gets logged in JS Oh. That sounds fun. Sucks. Email subscribers plug in. So my task was transferring a WordPress site from 1 server to another. We warp pretty rookie, so we're setting it all up manually.

Beautiful. Alright. Thanks so much for tuning in, and we will catch you on Monday. Have a spooky Halloween.

This was a nightmare scenario which has haunted me for years after.

possible pairs. And, like, how do you go through all the permutations? Because there's millions of possible combos.

I noticed that I set the production script to be target the production servers. I did not get fired, thankfully.

Yeah. Yeah. That's how you learn.

which I just looked that up, and that is an open source load testing

after 2 weeks than it had been before. Impressive. One line of code, a lot of money burned, a horrible week to wait for new search result measurements, by giving you My code

everything from scratch.

I got the names. You got the ad transitions.

if he thought his order for mister d hole was real or not.

somber than usual. I'm grabbing whatever preferred choice of caffeine was for the day,

upvote songs. And so you could enter them under hashtags and people upvote them or whatever. And the Owl City guy, he, like, kind of blew up on there first and foremost. Wow.

Spooky. Spooky.

Head on over to syntax.fm So Oh, yeah. Dude, you gotta get an electric toothbrush. Sonicare all the way day, all day. I may getting ready for some sick pics in the future.

So every time we had a switch between projects, which is regular because we worked on 6 different sites, we had to yarn unlink r m r f, the whole Yarn link directory, 400 deploying that last one where you have the gambling and you forgot the Boolean of test of true. There's there's so many, like, money ones like that. Like, of bad words, and they said not to say it. Unfortunately, I was a tech director in clearing out all the designer project manager accounts so no one would delete anything.

CSS I could recompile

and then 4 more, and then 8 more, then 16, Have you ever thought about email storms? Immediately, we called the IT support company who managed this email Vercel. But every time they would try to remote into it and clear the queue, the remote session would just freeze up. Over and over, they desperately tried to weather the storm, but they couldn't.

very painful.

httpd.conf and it went off without a hitch. We then had a migration script that would walk the DB and upgrade each record with default fields.

I miss the Ruby SaaS times where the refresh took less than 10 seconds.

I never got told how much money my this cost my client, only that I really, really did not want to know.

Never heard of it. Paperless Post is another good one. So, anyways, I had this pretty amateur rails web app API that we had built, typical MVC architecture I moved it from design to engineering. So I had some code experience, but never really dealt with environments.

Also, these kind of things are are a little more harmless than, like, you know, know, sending out a notification about some of these funerals. Not not exactly harmless.

like, a local development, the PHP mail function. So it was funny when the client called our office and asked

I've had actually my share of pullover. I had to once one time I was on the ski hill,

256,

requests per second, nearly 77,000

Face palm.

to 70

that's like, that hurts. Like, that person was probably so super rattled. So good story. Yeah. For real.

And, oh, jeez. Like, Oh, I remember when I was a kid, But no. I I think I I would like to have something like that.

Yeah. So con dangers of, No. There's, like, an issue in GraphQL where, like, let's use Vercel up tutorials as an example. I say I want all of the playlists on the site, and then I want all of the tutorials on the site. And if you set up your GraphQL server in a normal way where you have your resolver and then another

way back before that we had to parse order confirmation emails from said grid in order to retrieve the data.

Yeah. He probably whenever you hear that music, you just it's true. I remember I have that with, Owl City. Remember Owl City? You know what? Actually, back when I was trying to become a musician, I had a lot of contacts with the Owl City guy because Really? We were doing similar types of music. And there was a website.

Next Node, Bos Ackwards.

Due to some wonky permissions, I had set up their home directory to the root of the site so I could FTP

That's kinda cool, actually. Maybe we should do a show about that. I think that'd be kind of a cool tool. I'm interested. Yeah.

s h I t happens until the tweets start coming in and emails from investors

Oh, you think it would detect that? There's a lot of these. It's like, oh, man. There should be Check some some system if the

in testing.

That's good to know. Next Node, deleting a government website. Many years ago, I worked for a company that ran the state of, and I blanked this out already, Wes website. So, this person asked

and accidentally left out the forward slash. Somehow, I deleted that the company's entire development server, which contained Resolution. Early on, the company decided that deploys and rollback would be easy and instant.

ads on websites, and this burned 1,000 of euros per day.

Right. That that is the actually, that is the big problem there is that it does. It works always warp like, race conditions are so often not apparent in development just because it's super fast.

I found a breaking bug and pnpm the next 12 hours coding in a taxi next to the client on three g. So much pressure, I couldn't remember if a JS filter removed or kept values in a true condition.

Guys,

I was working on an app which included email integration,

company crashed as a result.

Remote suck read. The commit was reverted, but thousands of emails had already been sent. It was an interesting day, and I always remember watching our senior dev furiously revert code for the 1st time. I love the show in y'all's course.

Very funny.

was notified.

this next one is a happy SEO ending.

Don't on my DigitalOcean.

As I was already thinking about the weekend, I skipped the check every file of all commits if they're reasonable on the checklist benchmarking against

Scott was chit chatting with his bud, Jack Resider, over on Twitter. And as you know, Jack Resider runs the Darknet Diaries podcast, which is probably one of my favorite podcasts. Fantastic. So he he just talks about, like like, what? Like, security issues, dark web, it matched substrings

breaches,

is amazing. So we are going to play that for you right now, and then we'll come back with just regular Scott and Wes, which is not as good.

Listen to this. We've

on my teeth. Have you ever had that happen? Yeah. Cool. My dentist is is, like, hypermodern.

and I use an app to do it. And they so now they tell all of their other clients. They're like, he used an app. I gotta use an app. They're getting you flossing.

They do X rays with this thing that looks like a literal 19 fifties depiction of a ray gun. Like, it looks just like that, and you just, like, click it on the side of your head, and you're done. It takes, like, 2 seconds. But the the cleaning, they do almost entirely via a, like, a water, like, a water sprayer. Is that weird? Like, a water blaster? I I was much more of that over the scratching with the little pick.

I think I'm gonna pull the trigger. We got we got the the latest Sonicare.

They're they're trying to get me to buy a, a sensitive information from 1 email to another. Right? that's all I could say to some of these Wes these stories that came in. I am a developer in a consulting firm in Sweden rating c sharp Don't assume it works. Prove it does, which is a great point. Yeah. Check that. So this is the first time I heard of someone actually getting canned for a mistake like that. A year.

Got the the wireless charging. You just pop it in. It buzzes on your teeth. It tells you when you're done. It couldn't be Sanity easier, and my teeth have been way better for it. I'm doing it. I'm doing it. Yeah. I'm I'm an adult now. Alright. Well, so we've got the stories for you today. Banter. Lots of banter. You want banter? We got it. And none of it is about the weather, so you're welcome.

How are you doing today, Scott? the safe temperature

Oh my gosh. It's so efficient and so good. I have, like, a a receding gum on the inside because I used to have a tongue twister. So it, like, receded my gum, and I hate going to the dentist. Berate me all day long at the dentist. Oh. I can't take it. They don't berate me. They, use me as a shining example because I started flossing in the past, like, 5 or 6 years more more routinely, Client misses 60% of their leads. Boss is furious and chews me out over Slack.

I just went to the dentist,

and second one is Netlify. We'll talk about them partway through the episode. Oh, I forgot to do spooky voices.

cringe worthy stories of web development where it all went wrong. Today, we are sponsored by 2 awesome companies. First1 is LogRocket,

and Wes Bosch.

You're listening to Syntax, the podcast with the tastiest web development treats out there. Strap yourself in and get ready. Here is Scott Tolinski

Oh.

that people have sent in, and we're gonna read them off to you. Just horror stories of deleting the database or accidentally pushing something to production or, It's a QR code I uploaded to the website in time just, the module I was using uses its own SMTP implementation Our sister media agency called Facebook in the UK,