Skip to main content

almost 5 years ago Syntax Podcast

Horror Web Dev Stories - 2021

Wes Bos

Wes Bos Host

Scott Tolinski

Scott Tolinski Host

Topic 0 38:46

Updating all records

Scott Tolinski

Oh, tale of terror.

Wes Bos

have to pass there because it's gotta be the to stop this type of thing from happening. Otherwise, Maybe it shouldn't because then we would not have this hilarious stories.

Wes Bos

There we go, Wes. When I first started out working for a consultancy, they trained us in sales meetings to help managers get promoted because we were coming in to make them look good. This was okay because, obviously, we're coming in as contractor. However, after being rid Laid off due to 911. Yes. That was about 20 years ago. There was a couple stories here that were I think we've this is our 4th story that's, Like, 20, 30 years old. I was looking for a new job, and during the interview, I was asked where I'd like to be in x years. I mentioned the hiring manager that I wanted to eventually do what he was doing.

Wes Bos

but if you wanna literally update many records in MongoDB, you have to Oh my gosh. You didn't even tell me he was doing this. This is amazing.

Wes Bos

We should say, though, like, is this a bit of a issue with the SQL Yes.

Wes Bos

I want your job.

Scott Tolinski

One problem, Alright. Next question here or next question. Next Horror story. Let's see. We gotta get a come up with a good title for this one. Extreme and then it filled it up. And it stopped automatically when it was supposed to stop, and it it I didn't have to even, like, check to see, like, what the pressure level was. It just filled it up and then stopped. I was like, well, this is, like, super handy for bike tires or anything you got around the house. It says in here that you can use it to Fill up all sorts of things, including a car tire. And I'm just thinking, like Oh, yeah. That'd be wild. Nice to have in your car. The auto cutoff is re He because, like, if you can tell it to go to, 33TSI Right. That's gonna be awesome. Then it it just cuts off. Like, that's You're saving your tires because sometimes people are like, ah, a little over, a little under, whatever, and then you wear your tires unevenly. Hey. Can we talk about those? The air tire pressure gauges. You push it, and it says one thing. You take it out. You push it, and it says something else. You take it out. You put it, and you're like, I don't even know. I'm just guessing at this point. I'm better off just using my eyes and pretending that I know what it is. Rid Yeah. Those things are the worst. So Man, I need this because we I have at the cottage, we've got a tractor and a ATV and kids' bikes, and All of them are far away from where you actually need to pump it up. Yeah. And I always have to drag an extension cord out or bring the thing close enough to the cigarette adapter, But just being able to, like, walk over, this is awesome. It's it's pretty cheap. I was shocked that this thing existed and for as cheap as it did. That's rid Yeah. And it worked very well. 1st shot, I was not expecting. I was, like, $34. It's a good bet. I'll I'll give it a try. Whatever. It's gotta at least work to fill up a soccer ball, And it definitely did. It was definitely succeeded there. Alright.

Scott Tolinski

At the time, I was working on an online travel booking site. It was after 11, and the last thing I had to do for the night was to rename one of the hotels in our production database.

Scott Tolinski

That's when I learned that working late hours is not worth it because at some point, You get too tired, and you no longer make good decisions.

Scott Tolinski

1 night, I was burning the midnight oil trying to get caught up on the never ending workload.

Scott Tolinski

Fortunately, I had a dump of the production database that I created only a few hours earlier sitting on my local hard rid So, thankfully, I was able to restore almost all of the hotel names, save for a couple that signed up for the data dump, rid. And my boss was none the wiser.

Scott Tolinski

Yeah. Let's keep it for the stories.

Scott Tolinski

hotels in our database all had the same name. This is around 2003, so well before the point in time restores, and they're only backing up the database rid Every week at that point,

Scott Tolinski

I was panicking.

Scott Tolinski

That's actually a great lesson because I personally I learn that all the time. You know, you're you had a couple beers or something. You're working on some almost never productive, HD.

Scott Tolinski

Name some hotel chain. Oh, boy. I bet you can all see where this is going.

Scott Tolinski

or you, like, end up hitting a a bug that you, like, solve, and then you spend hours and you stay up all night trying to solve it. And then you solve it first thing in the morning because that's just how your brain works. Very good. Yeah. Burning the midnight oil is not always a great idea.

Wes Bos

and take his spot. So safe to say I didn't get hired. Oh, that's so funny.

Wes Bos

job.

Scott Tolinski

I forgot the were clause. Yo. We should name this title of this episode werewolf because it seems like so many people are forgetting the were

Scott Tolinski

tale of terror.

Wes Bos

Well, I guess he didn't take of that as I wanted to make him get promoted

Scott Tolinski

over 5,000

Wes Bos

Next one, we are going to call Ivanti's

Topic 1 35:27

Bug from old code

Scott Tolinski

re Basically, it's a CDN for your videos that gives you on demand

Scott Tolinski

To suit the needs of any of your audience depending on where they're listening to it, if they are if they're watching your video in some place, their Internet's not good, it's gonna serve them up at a lower quality version automatically, and it's gonna do this all via streaming. Now, take it from me, I've done a lot of video streaming, and it is not an easy problem to solve. You end up having to write all of these re Transcoders, and you have to write your whole ingest flow and move things around and have a database for all your files. No. Thank you. Re and Mux gets to do all of that for you and so much more. Thumbnails, animated GIF previews, the little thumbnail scrubber thingy, all of that is solved with a simple get request From Mux. So check it out. Mux Video is the API first video platform that makes it insanely easy for developers to build beautiful video. So head on over to mux.com

Scott Tolinski

Thank you so much for Mux for sponsoring.

Scott Tolinski

up.

Wes Bos

I log in to the app, Search a few case numbers, and sure enough, every case's data is exactly the same. I began freaking out as I had no clue how this could have happened. I mean, it could never have happened. All the dev work testing and months of lies production use. After I investigated Even worse for me, I'd only been with the company a month or 2.

Scott Tolinski

You live, laugh, laugh. You know what? It is it's so funny. You get, like, a couple sentences into each of these, and it's, like, almost like there's, like, one little bit of foreshadowing that, like, Yeah. If you've been around the block a little bit, you can be like, oh, I know it's coming.

Scott Tolinski

transformed transcoded

Wes Bos

I know it's gonna happen. It's true. There's probably only, like, rid I don't know. 5 or 10 things that go wrong in all of these stories, Sick pic. I have a sick pic for you today, and I recently upgraded ready. The new iPhone 13 Pro Max

Scott Tolinski

re Sleepy

Scott Tolinski

and it's helpful just to know these things so that you when you are in a similar situation, hopefully, these Spooky stories will come eking back at you. Yeah. Seriously, like, what have we learned here? Have a a staging environment that mimics your real production environment for testing. Don't have something that's only only only production. Yeah. What else are we learning? Don't swear. Don't write swears. Don't write silly stuff. Test your code. Yeah. Rid. Read things twice. Yeah. Do a dry run of database You'll have to pay it for the 10%. Even if you gain whatever you gain, you're gonna have to pay taxes on it. So, like, nah.

Scott Tolinski

video Here's my development, Hey. A safe run.

Scott Tolinski

This episode is also sponsored by Mux. Now Mux is the perfect place for you to host your video $90,000,000 to its users. Founder begs them to return it.

Wes Bos

thankfully, we kept the regular backups and were able to restore the data to a recent time frame without losing a ton of work. Rid. We commented out that database update and called redeploy of the code ASAP. Also, senior dev was cool, and he's like, hey. It happens to us, all of us at some point. Rid Let's just say I've learned then that's good. It's also it's not MagSafe.

Scott Tolinski

set The consoles were huge, and the size, You made that oopsie. You gave the money away. I don't think threatening them is going to give the money back, so

Wes Bos

We also realized that it left of the where clause in the related SQL query to specify

Wes Bos

with a senior dev, we realized the rid. Try block had failed, and the update query in the catch block ran for that record.

Wes Bos

a ton since then and definitely steer clear of writing code like that. You live, you learn, I suppose.

Wes Bos

which records needed updating.

Scott Tolinski

Give it a try, especially if you are the type Who doesn't want to deal with all of that annoying stuff around video is one of those things that we use personally over at level up tutorials. And let me tell you, I absolutely adore this service, so check it out, mux.com.

Topic 2 13:48

Accidentally emailing customers

Scott Tolinski

Woah. What? Woah. Is saying you got hacked better than saying we accidentally sent these mailings out that were a test? At least blame it on the intern. That's what HBO did. No kidding. HBO is like, I mean, like I said, there's we got this person just Push the button. I'm so sorry. We can't do anything about it. Oh, man. Never put anything,

Wes Bos

I thought my goose was cooked, and the big boss was Plenty mad, but I owned up right away and apologize.

Wes Bos

Love your show.

Wes Bos

We put out the cover story that we've been hacked and always forgiven.

Wes Bos

Rid Phones were ringing. Voices were raised. Turns out, I had moronically wired myself to the production database.

Wes Bos

swear words, or silly things into tests. Always just write test, test, test. Yeah. Even loremipsum.

Topic 3 11:25

Accidentally deleting server

Guest 5

Hey, loser.

Scott Tolinski

Rid I was very young, so I had to blame it on the server. I'm not proud of it. What'd the server do? Good. The the server just got corrupted.

Scott Tolinski

One time, I needed to format a server.

Scott Tolinski

So yeah. That one's pretty bad. You know, I'm so very paranoid about that. Whenever I'm I'm formatting a hard drive Yeah. You know, when they say, like, you know, the measure twice, cut rid One thing. I'm measuring, like, 18 times. If I have moved over precious files or backed up something, it's like, I need to confirm that those things are where they are. Read. I'm from the the days of a LaCie 200 gigabit drive

Scott Tolinski

My boss was pissed.

Scott Tolinski

But why the heck Would they ask a developer to format a server in the 1st place? By the way, my birthday is on Halloween.

Wes Bos

mass in it, it's, Yeah. Whenever it includes the word mass, it's like, oh, man. Who knows what demon drove me? But I wrote the test mailings like ransom notes. Dear loser, fork over your dollars or else. Well, all was looking great, And I was feeling pretty pleased with myself. Progress bars were sliding and counters were spinning, but I could hear the rising commotion from the marketing guys behind me.

Scott Tolinski

It was an outdated Windows Server. Read. I selected and copied and pasted to an external hard drive. My drive was pretty fast, and it looked like a minute. And I was like, wow. This is a great external HD.

Wes Bos

Yeah. Always have 2 backups of everything and confirm that rid They are working periodically. That's just a awful thing to happen. Yeah. Next one we have here is called Hey, Loser. You wanna say it in your spooky voice?

Scott Tolinski

storing your entire life on it, and that drive just waking up one day and saying, not today. Rid No thanks today. I'm done. So, yeah, I gotta have those backups.

Scott Tolinski

External HD. Okay.

Scott Tolinski

John finishes Love it. Spooky.

Topic 4 26:20

Breaking site with dependencies

Scott Tolinski

That's not on Friday. Don't blame Friday for you.

Wes Bos

Yeah. Risky. Risky. Risky. I definitely would not rid Do that myself on a Friday. No matter how much I tested, it's just a recipe for disaster. Hopefully, they had, like, some, Like, lock files of of the previous dependencies, they were able to roll it back or fix the thing pretty quickly. But Yeah. I love the modern host now just have, like, the rollback.

Scott Tolinski

The back end guy did his integration,

Wes Bos

Next one we have here is stupid selfie horror story for you, Wes. I work one of the biggest retailers in the UK, and we were working on an app That would go on a media wall in their flagship store in London. Cool. Basically, a giant

Wes Bos

when I uploaded a couple of my stupid selfies of my big head in the office. Rid. I get a call the next day to ask why my face is all over the media wall.

Scott Tolinski

rid. That's on you.

Scott Tolinski

and it launched on prop. As it turned out, There were some breaking changes in one of the libraries that decided to update and crashed the entire site on Friday at 4:30,

Wes Bos

200 inch screen In the middle of a store that would show social content can go on to, I actually built a couple of those for a forget what it was, but it was on, like, Part of, like, a Christmas thing, and they would show tweets where you could hashtag it. And I remember having to, like, filter them for swear words and everything. Right on. Turns out, rid. I left my local dev version connected to the production API And then you're running that through a Converter. Converter or inverter that changes it to AC,

Topic 5 42:11

Bad interview answer

Wes Bos

it has the

Wes Bos

check it out. I'll shamelessly plug my advanced react course at advanced react.com,

Scott Tolinski

for a full archive of all of our shows. And don't forget to subscribe in your podcast player or rid

Wes Bos

and I have just updated it ready. For I added a whole bunch of the whole testing module, and the deployment module is coming probably by the time you listen to it. That will be there as well. So check it out if you wanna learn. Rid React, Stripe, Checkout, Ecommerce, GraphQL, That's insane.

Wes Bos

Halo Lock Magnetic Wireless Car Charger.

Scott Tolinski

.com.

Wes Bos

rid Cigarette adapter thing. Probably charges as fast as anything else in my car right now. Yeah. We've got these, like, gas station rid. 5 volt ones and, like, they've been fine for us all this time.

Wes Bos

Don't do it.

Scott Tolinski

wonderful com forward slash pro. Sign up for the year and save 25%. Also, the course that's coming out for this month, October, just about any day now, is going to be rid On Astro. So if you've never heard of Astro, Astro's a neat new tool that allows you to write in any front end framework and have ship way less JavaScript. Ready. It's very, very cool. So if you're interested in learning Astro or Prisma or Svelte or anything that we teach, level up tutorials

Scott Tolinski

And I I I thought there's no way this thing could be any good because this is a little $34

Wes Bos

Rid MedSafe

Wes Bos

That's, like, the most inefficient way to charge your phone ever. Yep.

Wes Bos

and it drives me crazy because the power coming from your your battery and you're alternating your car's DC,

Scott Tolinski

and scary and hilarious all at the same time. So thank you if you've submitted your your horror story, and rid I look forward to seeing the next ones for next year, and happy Halloween for those of you who are are celebrating the spooky Halloween.

Guest 1

this

Wes Bos

Apollo, all of that good stuff. Advancedreact.com.

Scott Tolinski

mini air inflator with a tire indicator, and it charges via USB.

Guest 1

a

Scott Tolinski

air compressor, a $34 little air compressor that doesn't even plug into the wall. Read. And what was so neat is that you charged it up, you turn it on, I plugged it into the soccer ball, I selected ball, migrations.

Wes Bos

all wheel drive edition, whatever it's called.

Scott Tolinski

Modern GraphQL database with the Prisma, and he works with even,

Guest 1

like

Wes Bos

Alright. Let's move into some sick pics. I got Sick pics.

Scott Tolinski

It's very funny. These are both simultaneously has made a spooky mistake in your code. Everything's just gonna be spooky from now on, unless it's just spooky. Spooky. If you've made a spooky mistake in your code and you don't know why, rid. But you get reports of users seeing some unusual behavior in your application. You might be concerned.

Wes Bos

fingers crossed, has those as well. And I just Yeah. Do think sometimes I see people have I see pictures of people who have those, and they stick the, like, rid Regular iPhone wall adapter into that,

Scott Tolinski

dot rid

Wes Bos

which is awesome,

Scott Tolinski

and Prisma, which is a really neat tech, head on over to level up tutorials

Wes Bos

and then you run it through your wall wart Outlet. It turns it back into DC.

Wes Bos

faster or higher voltage,

Scott Tolinski

We love ourselves some good dev horror stories, And we've all done it. Everybody's been there. Everybody's had that. Them person says, you know the face. We all know that face, that, oh, boy. What did I just do face? I'm in double face, that big double.

Wes Bos

Like cigarette adapter, so I'll probably sick pick that in the coming episode once I give it a shot, but I was very impressed with with this. So it's called the ESR Halo Lock. Rid. You have to buy the it's a vent mount one, which is we're getting a new car tomorrow, so I was, like, fine. I the vent mount didn't work well in our rid. Our BMW that we have now, but I hope it does in the in the new van that we get. And they also have one that, like, sticks to the windshield as well, but I was pretty happy with it. Rid I need to get one of these. Yeah. It's a good one. I don't have the MagSafe on my phone, but I have a wireless charging. And I don't have any mount in my car right now. Not that I drive that much, but still. Yeah. Like, honestly, like, it's it's also still a good like, if you don't have it, I probably it's $35, and you could get, like, a decent just magnet mount rid. For probably less than that. But if you just wanna think about, like, future forward,

Wes Bos

charges at, like, 5th 18 watts or something like that, but it should charge fast enough once I have the proper

Guest 1

show.

Guest 1

you

Wes Bos

email, [email protected], And

Guest 1

if

Scott Tolinski

rid com, which is actually, I'll I'll plug 2 of them. We have 2 courses now. 1 that came out last month from Ryan Olinska and is on Prisma,

Wes Bos

You can use coupon code syntax for an extra $10 off. Rid Alright. That's it. Thank you for tuning in to the spooky episode of Syntax. We will catch you later.

Wes Bos

and we'll put it on the list for next year because these are hilarious.

Wes Bos

rid MagSafe charging, and my wife has it as well. So we just said, okay. Let's update the car mount from we previously I probably 3 or 4 years ago, I picked rid. The magnetic car mount that goes in your CD player,

Scott Tolinski

Shameless plugs. I'm going to shamelessly plug the latest course on level uptutorialsdot

Scott Tolinski

Drop

Wes Bos

So getting a proper one, we do have, like, 1 little car adapter that is it detects rid How fast it can charge? Like, it's one of those, like, Smart IQ ones. Yeah. But I think it's from Anker or something like that, and it it charges the kids' iPads super fast. Wow. That was in the back for the kids, not in the front for us. Of my new car, whenever it exists, whenever that may be Yep. It has plugs in it, Like, AC plugs in it. Yeah. Yeah. Like, several of them. I'm like, oh, yeah. That's gonna be the best. That's awesome. We our van, We're getting tomorrow,

Wes Bos

long form horror stories that we have for you today, those are some pretty good. And thank you everybody for submitting them. If you have a horror story of your own, feel free to

Scott Tolinski

all sorts of really neat APIs. It's a really substantial series, and Ryan is an excellent, excellent, excellent teacher. Rid. If you're interested in learning a little bit more about GraphQL And seeing a bunch of warnings I think that's typical, by the way. Seeing a bunch of warnings about a couple packages being outdated, I decided to update some of them. It rang great locally,

Scott Tolinski

Rid I have a a sick pick today, and it's a portable air compressor, which I didn't think this thing would be any good. I needed a bike pump rid For when he got a soccer ball, he's got some bike tires that need pumping up. We didn't have a bike pump. And, you know, the bike pumps are are you know, they're, like, 10, $12 anyways. So I found this one that was a electric one, a portable sign mountains of documents about unreleased vehicles or all sorts of stuff. Oh, yeah. Or, like, when we were doing demos that people were going to be seeing, We had to make sure that we were using, like, specific model. I told you about that one time we were doing a q and a, and that guy was, like, super upset that the Ford Mustang, like, wasn't in the thing. Re Ruck, listen, man. We've been given very specific assets.

Guest 1

review

Wes Bos

And the way that it works is if you have a case rid That's not MagSafe or whatever. Like, if you just clip your phone to it, it works great. But if you have a case like most people do What if you have a phone that doesn't have rid. MagSafe. Could I put this magnet on the back of my phone? Yeah. Well, it's so it's a little stainless steel halo So I can that goes on the back rid your phone. I can do that. Yeah. And it makes the magnet that would later be implemented into a new bigger app that consolidated

Topic 6 08:18

$90 million crypto bug

Wes Bos

I discover

Scott Tolinski

Oh. Couple of doozies. Confirm. Confirm. Ridged. Confirm. Yeah. Yikes. Yep. Good good story, though. Couple of doozies here. Next one we have here is called External

Wes Bos

which meant that accessories were not saved at all, which was discovered after a week. I fixed the air in 5 minutes, and the sellers Had to contact x number of customers to double check what kind of accessories they would want on all their orders.

Wes Bos

I find that I'm in a different folder than was the one where I saved my script. In that folder, there was an old deploy script that was used when the project rid. Was started 1000 of years ago. The first thing the script does is force delete the site and then to create a new empty site. The site is gone with lists In everything. Listen to SharePoint are sort of, like, SQL lites, so the database is gone, essentially, here. Oh. There are no back ends of the acceptance environment, article where all of the names of the things had leaked, and, like,

Wes Bos

rid. Although it is very important, I feel just a little bit panic of how I'm going to solve this. However, I remember a testing tool 6 months ago to copy the entire environments, which re A temp was made to the acceptance environment.

Scott Tolinski

So lesson here the lesson here, backups, rid Frequent backups, if you have things like this, dang. Yeah. Frequent backups for stuff like this. I mean, it seems like they got lucky with a backup, not even, like, one that was prepared to be there. Oh, a little bit of bonus here. Once I updated the a feature that saves accessories on order, same solution. However, I rid. To add all the new fields to the production environment, Thank you so much, and,

Scott Tolinski

Oh.

Wes Bos

Rid. It was my last week of my job before I was due to start a new job. I worked 12 hours a day to keep up with all the handovers to colleagues so rid. They would have a chance to continue working on all the solutions I had taken care of. The project was a process tool hosted on SharePoint online. Rid. The guy who would oversee it had negative 1% experience with SharePoint to which I pointed out to my bosses.

Wes Bos

the site is gone. In checking the trash can, there are no things there.

Wes Bos

Rid. So the story goes, hi. This is my contribution to these stories. I was a developer at a consulting firm in Sweden writing c sharp rid. On the back end and using React with either JavaScript or TypeScript and hosting everything in Azure on the 99% of the time and 1% in SharePoint.

Wes Bos

But To make things a bit easier, I wrote a deploy script to ease things out a bit. Starts in the terminal and runs a script towards the acceptance environment. Rid. Umpteen million errors appear, which is strange because there should be only about 20 commands which can cause errors like these. Rid. I log in to the environment to double check now. I accidentally entered the wrong values in the script, which looked okay according to me, but I got a four zero four error when I was trying to reach the environment. I log in to the admin interface.

Topic 7 23:28

Deleting site with script

Guest 2

Why? He stood there

Wes Bos

95 would have that black

Scott Tolinski

it was more popular in, I think, the the Ruby on Rails world. Okay. HTML and Nuntrux Templates that were later integrated with the back end using some Python witchcraft. Oh, we had snakes and witchcraft.

Scott Tolinski

But I didn't build the prod version

Scott Tolinski

was also a metric, boop, Ton of JS libraries added, like Babra or page transitions, 3 JS for cool interactive animations on the landing page, rid Didn't really get much of all this package dot JSON stuff at that seniority level. So after running Yarn or NPM or whatever In desperation, I called the Nobel in Utah

Wes Bos

and

Wes Bos

Oh, that sucks. I remember doing this exact same thing, not Taking entire business down, but when I was in grade 3, remember Windows

Scott Tolinski

Re Okay. I was a junior dev working on our company's website. They were HTML

Wes Bos

So I push the button in, And then I start, like, looking around and being like, oh, man. Like, who who here is smart enough to know how to shut down a computer? Right? Because, like, I was the tech kid. Because I I couldn't do it with my left hand, so I called over a buddy of mine. He shut down the thing, and, like, Windows 95 took, like, 3 minutes to shut down. So I was sitting there holding the button in until we finally got to that black and orange screen that said it's now safe to shut down your computer, and I I let of go, the button. That's so funny. I don't honestly even know what it's doing in that shutdown process. I've never, like, taken the time to learn rid that actually was about. What's the deal with that shutdown screen? That's a a really good question. Probably super, super strong. I went over some huge speed bumps pretty fast and all kinds of stuff and no problem. Rid. And I was shaking the thing. It did eventually fall off, but much more than it would. I was so skeptical that it would be good, and it is rid. Awesome. Very, very strong, and it charges the phone. I am waiting on a Language that this happens. Like, I know people don't like MongoDB,

Scott Tolinski

as I didn't know there could be any differences. I was working on some minor feature, And the PM decided that there was no time for a code review, so I pushed the repo.

Scott Tolinski

plus Nunchucks templates. I'd never used Nunchucks West. Did you? Rid No. No. I was, Booth.

Wes Bos

he stood there for 4 hours holding the button in until we could let it crash after business hours. We gave him a chair rid After 2 hours.

Scott Tolinski

something with the hard drive. Yeah. It's the same thing with, like, safely ejecting your rid SD card or whatever. I don't do that. No one's got time for that. No one's got time for that. Yank it. Okay. So I gotta do the spooky voice. Don't deploys on Fridays.

Topic 8 15:11

Mass website outage on Friday

Wes Bos

confidential. And sometimes, probably even, like, I wonder Way back when mainframes were king, A guy I work with pushed a button in that, if released, would immediately take down the entire company. He stood there for 4 hours.

Scott Tolinski

Horror story. Rid I was part of the developer team that accidentally leaked the 8 cities in the XFL, We were told never to bring as long as the test passed, I've manually tested immediately afterwards, and Everything is, like, this is a small, small update, but to push some code to production that you did not test after updating dependencies, I figured I'd celebrate my 1st day and push some code to prod.

Scott Tolinski

that's when all 8 cities were leaked. We were using Contentful and Gatsby.

Wes Bos

A lot of developers have to go through, like, trading stocks and things like that. You have information to it's kinda crazy. When I worked at Ford, I had rid The guy who tweeted it out says, if you received a large incorrect amount

Scott Tolinski

So if you are the type of person who So it's Such a common thing. It seems like a common thing, and it seems like something that I would do if I wrote a sequel.

Scott Tolinski

A ready. Junior dev entered the information into the prod space

Scott Tolinski

Rid OP is about to get his head in a a choke slam here. The league went bust because of COVID. Yes. Re If this is used, please don't use my Twitter name. You can be safe. You hate me. We did not expose you on here,

Scott Tolinski

an alternate football league. And for those of you who are not Americans, They're talking about American football here. The XFL was the wrestling XFL one, where they hit people hard and they had some fancy new rules. Either way, A week before their press conference, users or something or leaked something to the re WWE's news press kit. So I don't think this one this one's not too bad. Next one we have here is

Scott Tolinski

I found out after seeing the stories pop up in Google News when I was about to go to sleep, was content down and started getting calls from the CEO or CIO

Scott Tolinski

And I am so sorry that you you accidentally were part of the team that leaked the aid cities. That's actually a pretty major leak if you think about it. I'm sure you could find the news stories about that when that was leaked. Chances are, if you did something where there, like, becomes a news story about whatever you did, rid. That's a pretty big oopsie. He'd linked us to a ewrestling.com

Wes Bos

names are leaked out. That's kinda interesting that, like, a lot of us rid. Developers have access to information that is rid them to eager so they would create a single SQL statement instead of a bunch or I was testing some new code to automate mass mailing to our customers. Oh, these ones always start so good. Whenever it includes the word Around 6 years ago, I was running a large into a large rid. Issue with some of our queries running slowly against our date DB.

Scott Tolinski

update. So as you can imagine, there's new data available, and so the Gatsby site was rebuilt With that new data, that new data then went live.

Scott Tolinski

Gotta have the Mustang.

Scott Tolinski

I'm sorry that the site does not have a Ford Mustang for you on it. For those of you who have never heard that story, We were doing, like, a q and a with, like, real people, and this guy was, like, so hung up. He's like, I hate this design. This design is terrible, and we're just like, why? There's no Mustang.

Scott Tolinski

They're just psycho.

Scott Tolinski

of the WWE.

Topic 9 02:21

Collecting horror stories

Wes Bos

read out the titles in that.

Scott Tolinski

This is fake. Obviously, this is fake. Sorry. I Started off with the fake one. The rest are real. Yeah. The rest are real, I promise. For those of you don't know, they actually needed to use an angle grinder to get into the to access the The physical servers for the Facebook outage? Yeah. Yeah. So that's what this joke is about. Basically, the Facebook, they had, like, goofed up something in their DNS settings. Not just like, oopsies, we added a period here in the DNS. No. They were, like, doing some crazy self managed DNS stuff, and it, like, pulled all their records from the web. And then Yeah. They, like, locked themselves See, that wasn't even the spooky voice. That was just me trying to do it. Suddenly, How's that? Rid Yeah. Sure. Yeah. There's no there's no bats in their bell bell tower. That's for sure, gang. Yeah. I don't really know. So so, Wes, I think we are ready to kick it off with some

Scott Tolinski

out. Yeah. So they brought an angle grinder and tore down the door. Oh, wow.

Guest 2

Woah.

Scott Tolinski

sound decent. It's been a voyage, Wes.

Wes Bos

What's going on here?

Scott Tolinski

Don't ask me how it's complicated.

Scott Tolinski

Yes. As long as it doesn't crash my computer, I'm very good. So I will get started with this first one that I found on, I found on Twitter. I thought this was very funny. Says, hey, guys. Love the show. I wanted to share with you something that happened just the other day. It was October 4th. I was starting my new job at a large tech company.

Guest 2

How did you do that?

Wes Bos

That was well worth it. You'll have to

Scott Tolinski

You would rid not believe how much time I had to put in just for that one little Yeah. You can also use it to buy Coke with some that money you saved. Okay. Oof. Rid A big horror story.

Scott Tolinski

even DNS.

Scott Tolinski

rid I end up bringing down the entire company, all of their servers, and they ended up bringing in a guy with an angle grinder to access the server cage. Rid Wait a second, Wes. Wait a second.

Guest 5

How did I do what? developer fumble.

Guest 5

spooky story.

Scott Tolinski

They use React for everything,

Topic 10 18:09

Performance issues in production

Wes Bos

Next one we have here is called, I don't have memory of this.

Wes Bos

1,000. The change worked fine in my dev environment, QA, and staging. Staging was supposed to be representative of production, This one is called dead button.

Scott Tolinski

Yeah. Like, rid I take backups. I do everything you could possibly do to be secure about it, do dry runs and all that stuff. And we we did some database migrations recently, and I was just like, Dear gosh, please work no matter what. Even though I knew it was gonna work, it's still like,

Wes Bos

so we went live with and within a few minutes, Very strange. Very curious.

Scott Tolinski

and you could go in there and and you fire up your video camera and you say, oh, that's not a ghost. I thought that was a ghost rid stealing my cookies. It's not a ghost. It's actually my my 4 year old child opening the thing. And you could see your 4 year old child sneaking in those cookies with LogRocket.

Wes Bos

and specs of how many pieces of databases you have how many items in your database you have, things can go south. Do you know what? There's just about nothing in the entire world that scares me more than modifying a production database.

Wes Bos

We quickly switched back to lazy joins. We found out that staging had more memory and fewer DB records

Wes Bos

We're using jpa/ Next 1 we have here is called a young dev. I was a young dev right out of cottage. My first job Was that a child support company where we had desktop apps that would handle all the information in a lot of programming languages. Just update it. If it's not there, then insert it. Re Before the 1st production release, I updated the flow to handle create updates in separate functions, but I never removed the update in the catch block of the original function now used for re creates rid. Smookiest cloud computing and Linux servers Finds the clone environment, and we can use the same tool to clone it back. It only took 8 to 12 hours of work to recreate the new things in the environment in the last 6 rid months instead of x number of hours to build everything from scratch. Oh my.

Wes Bos

I had 2 rid. Pretty bad code changes that only showed their problems when they went live into production. The worst kind of bugs are bugs that show up When they only hit production, especially ones that are, like, take the entire site down.

Wes Bos

and had a bunch of joins that were done lazily. I switched

Scott Tolinski

just hold on to your butts. Next story here is called, So I wrote my query, update. Hotels, set.

Wes Bos

both in terms of specs of the server Orange screen, that was that is now safe to turn off your computer. Yeah. I remember the teacher's like, alright. Just shut down the computers because that's what you did when you were done with the computers. You turned rid So I just push the button in, and I immediately recognize, oh, shoot. It's not safe to turn off your computer that way. Right?

Wes Bos

hibernate

Guest 5

Your performance is slowing us down.

Wes Bos

Then production, and they were supposed to be exactly the same. Oh, ain't that funny? Like, you run all these tests. You think about everything, but rid. If the environment is not to a t exactly,

Wes Bos

our entire system went down And was out of memory.

Topic 11 04:12

Wes deletes Denver emails

Wes Bos

I have a bit of a ready. Story myself that I figured I'd start the thing off with is this is about maybe about a year ago that this happened, and re Listener of the show, Tim, a Denver coder on Twitter, he

Scott Tolinski

Yeah. Where are my people at? My people gotta come to, Level up tutorials here.

Wes Bos

messaged me one day, and he was like, hey. I'm trying to buy your course, and it's just not working. Like, It's, like, not giving me an error message or anything. And I was like, that's super weird. Like, what what could possibly be going on? And luckily, he was nice enough to, like, give me all this information.

Wes Bos

Probably be about 20% of them are from a university in Denver. Oh, okay. Okay. Okay. But, rid. Surprisingly, a lot of people have Denver in their mail address, so that was definitely a issue.

Wes Bos

What? more efficiently than using Excel? My first project was to write a proof of concept

Scott Tolinski

could buy any of my courses. And it had, like, never popped up. Like, who's got Denver in their email address? That can't be, like, that common. I I'm you know, somebody lives in Denver. It's not like We all we all got that email address or something. That is very funny. I I actually wanna see if I can see how many people in level up re Orioles database have the email Denver in their email address. None. There's not a single one. There's not a single one. I'm curious about that too. Well, let me look at mine. 63.

Wes Bos

Rid. And I was, like, sitting there being like, what is going on? Like, I'm selling all kinds of other courses. Everything is working fine. Like, 0 things are showing up in rid Yeah. But and it's like, redeploy this to play, please. Yeah. Thank you. Yeah. Honestly, like, that's sort of the way to do it as you deploy the thing on another server And make sure it's all running fine. And then when it is, you cut over to the new one, and instead of having to, like, break the entire thing.

Scott Tolinski

That's super funny. Here's another real world one, not one that was sent to us, I figured this one would be good to share because it turns out, like, the week we decided we were gonna start pulling stories for this episode, There's a ton of, like, high profile dev oopsies. Here's another dev oopsie horror story here that I thought was even just reading that the headline might might get you shaking in your boots. DeFi bug accidentally gives

Wes Bos

and my error tracking. And I was like, what is going on here? And finally, I was just, like, going as I was reading, like, the code base line by line just rid Just trying to figure out what could be possibly happening, and then I realized they came to me. So probably about a year and a half or, like, 6 months before this, rid. I was getting slammed by some spammer. I was trying to just slam my thing with a bunch of fake credit cards, and they were Stripe was handling it and whatnot. But I just wanted to, rid Stop this from happening. So I fairly quickly just jumped in, and I wrote, like, a block. I said any email address that has Denver in it, just reject it. Rid Right? And because the the spammer was specifically using, like, an email address with Denver in it, and I it was just enough to just stop this guy from from hammering the system. Right? I just threw a little if statement in there and returned if it had the email Denver in there, and it went along with my life. Oops. It did. Yep. So the bug was rid. Literally nobody with the Denver in their email address, according to a RegEx,

Topic 12 31:46

Accidentally deleting code

Scott Tolinski

files

Scott Tolinski

to break everything up, and then I thought I would delete all of the print files. I mistakenly

Wes Bos

wrote a try catch Statement that it would attempt to create a record, and if it failed, it would update the record. That's called an upsert Oh, that's a good one. I like that. Werewolf. Werewolf.

Scott Tolinski

from the weekend.

Wes Bos

only. Somehow, I or any other PM or QA never failed to create or to hit this catch block while testing.

Scott Tolinski

Re whatever the delete command was, I don't remember. Now programming files have a dot p r g Extension. I deleted all of my updated and that, kids, is why you don't deploy on Friday. Yeah. You think? You know what? I actually don't mind deploying on Friday. Personally, and sign up today.

Wes Bos

all of the proof of concept for various parts of the child support process. For some odd reason, I still don't know why to this day. My boss wanted to write me through me to write this new app on top of an older app With a bunch of legacy code, I never understood why. But as a young dev fresh out of school, you tend to just do what you're told.

Wes Bos

In school, I mainly use PHP, HTML, and CSS for learning how to work with a database, but this job was c sharp in .net for their desktop apps.

Wes Bos

My young dumb self that is out there, they if you've got

Wes Bos

12 months later, I got a ticket to investigate why every case's data looked the same in production.

Scott Tolinski

Remember my horror story, which is running git clean on the root directory rid of a machine. Oh, yeah. Yeah. These all remind me of that. No. Thank you. I'm stressed out just reading these stories. I know. I think I I have a certain degree of tension in my body after this episode.

Wes Bos

So this Fast forward to probably

Wes Bos

one little hitting a g instead of a t,

Scott Tolinski

Hoping that they could recover the files, but it was no go, and the demo Monday morning was not fun. Oh. Re Failed demo's never any good. I've been in a room with a failed demo where the dev was just sitting there sweating the whole time, And it was not a lot of fun to be in, and I wasn't even the person who worked on it. So yeah. No. Thanks for that one. Oh, man. Like, rid. This is why you should use that trash command. Yes. Yeah. Trash. No matter what,

Wes Bos

So I was doing a lot of learning as well. I remember finally learning how to connect to the database and run some SQL after fighting this old pile of legacy code. In the early versions, rid I chose to handle create updates for those records in the same function.

Scott Tolinski

and I had a ton of them in the main directory from all the testing I had done. I was getting to cleaning out old files, getting ready

Wes Bos

We talk about test coverage a lot. That that's a a missing case in your your test, where if you write some code and that line of code is never run-in all of your tests, rid Then you don't have very good test coverage. Fast forward to probably 9 to 12 months. Sorry. I'm just trying to add some value to the listeners.

Scott Tolinski

instead of dotprt.

Topic 13 00:00

Transcript

Scott Tolinski

stories.

Guest 2

With me, as always, is Spooky Dalinsky.

Scott Tolinski

Ready. It's Spooky Talinsky here with West Get in here and actually

Wes Bos

Rid. So these are always a blast. I'm I'm super psyched. We are sponsored by 3 spooky companies today. First 1 is LogRocket

Announcer

You're listening to Syntax, the podcast with the tastiest web development treats out there. Strap yourself in and get ready. Here is Scott Talinski and Wes Bosch.

Wes Bos

And the next one is Mux. Re They do but now we're like, we want the one that's magnet mount, rid. And it charges the phone because then you just have to click it into it, and you're you're up and running. So I asked on Twitter, like, is it any good? Like, is it gonna fall off? Like, rid. You see the the MagSafe, and it comes off, like, super easy. I was like, there's no way that's gonna go over our neglected streets here in Hamilton. Rid So I found this one called the ESR

Wes Bos

in your server.

Wes Bos

Linode. They are the JavaScript

Wes Bos

So today, we've got rid. Our horror web dev stories for you, which is pretty exciting. We do this every Halloween. It's it's become a yearly thing where we ask the audience

Guest 1

Welcome

Guest 2

rid The spookiest

Wes Bos

video hosting and transcoding for your application.

Scott Tolinski

So if you have a spooky video, they could transcode it. Okay. What we gotta do is set something different than spooky for the third one. What do you got? 3rd one is

Wes Bos

are awful, but, usually, they are in the past, so we can laugh about them now.

Wes Bos

rid To submit their most stick your head in the sand, rid Imagine, like, a little Gatsby error and, like, the entire league

Wes Bos

So if you have ghosts in your application, Re You know what? Preferably, what I would do is I would be kicking you out of your chair, sir. That's what I would do. The guy probably didn't want a new job. That's so funny. That is good. Alright. That's rid All of the

Wes Bos

oof horror stories of things that went wrong as a developer, whether it's rid. Deleting an entire database or shipping

Wes Bos

Some gremlins

Wes Bos

they'll be able to track them down.

Guest 2

web development show in the world.

Guest 2

to

Wes Bos

spookiest

Guest 2

syntax,

Wes Bos

a swear word or all there's all kinds of hilarious stories. So I think this is the 3rd year that we've done it now, and rid The stories rid Supposedly type that in one of a few different ways. You have to pass it as a Boolean or use one of the methods that update all.

Wes Bos

Mhmm. And it's going down. You know who you should be trusting is Linode.

Topic 14 21:10

Company outage from button

Scott Tolinski

Multi headed sounds spooky. Oh, yeah. You know that multithreaded

Scott Tolinski

was running too slow and generate a warning. Rid. Multithreaded.

Scott Tolinski

I was pretty embarrassed by these 2 issues. It went to show that That production is its own special thing and that you really don't know, and your server side code is going to work until it starts running there. One, I am so glad I never have to worry about multi threaded anything in my data g job. If I had to worry about that, that would probably be Not a good situation for the company I was working on. This seems like there's, like, a lot of attention to detail when dealing with when dealing with performance of that kind of nature, but this sounds intense. I don't follow anybody for goofing up something like this and just seeing that performance degradation. At least you caught it before you sent out an email to 20,000 There spooky

Guest 5

No. No. No. No. No. Okay. Back when VMware

Scott Tolinski

Again, production and staging were set up differently. Staging was a bare metal server. Production was running on an ESXI

Scott Tolinski

and the rest of the site's DB work. The code had no impact on my dev, QA, And staging environments,

Scott Tolinski

we were hitting well over our minimum number of concurrent users. We deployed it to production, and then the system got slower and slower, but never crashed.

Scott Tolinski

XI server host.

Scott Tolinski

rid was slowing the whole system down when it tried to synchronize the data across threads.

Wes Bos

From my dad, actually. My dad's worked in tech for many, many years. Bert? No. Not Bert. Not the creator of CSS.

Scott Tolinski

was becoming a thing, like 2010 or so, I was working at an ecom site, and we were seeing slow performance between the app server and some data services. Rid I decided to build a little multi headed logger that could track when a query to Oracle Financials rid Souda.

Scott Tolinski

threaded logger that could track when a query to Oracle Financials was running too slow and generate a warning. Oracle Financials was doing the credit card transaction orders

Topic 15 06:46

Scott leaks XFL cities

Wes Bos

so rid Just erase 40 hours and a bad demo. Yikes. Yikes. Yikes.

Wes Bos

Oh, man. Alright. This one and by the way, we keeping everybody anonymous in this one Just to protect the innocent. Most people ask to be named anonymous, but, literally, everybody is anonymous, so we don't get any people asking us to claw back their story.

Scott Tolinski

So this is this crypto protocol, or I don't know. Honestly, I don't know the the language enough to speak on this, but, basically, rid They gave $90,000,000 to their users because they goofed up, and then the guy went on Twitter and, like, really? I think begging is He threatened them. He was saying he was gonna send the IRS after them and all sorts of stuff. It's like, dude, I know it's $90,000,000.

Scott Tolinski

That's a big time oopsie. I I think, $90,000,000, How hard could a use effect be? Right? Next thing you know, is a tough word? There are 2 t's in there, and they're very quite close succession. Re Multi Head on over to syntax.fm key erased asterisk.prg You might be hiring a a ghost hunter, but we We will hire LogRocket because it allows you to see what's going on in your site, and that's like an error and exception handling service that gives you an actual visual replay. Imagine if someone was stealing your cookies,

Scott Tolinski

oopsie in their career. Oh, Alright. Next one here is

Wes Bos

from a protocol error, please return it. Rid. Keep 10% as white hat. Otherwise, you're being report as income to the IRS, and most of you doxed. Well, If you report it as income to the IRS, you get to keep 50%.

Wes Bos

Yikes. Rid That's a bad day. That's a bad day.

Scott Tolinski

And, also, you have to pay taxes on the IRS for that regardless.

Scott Tolinski

there's there's very few developers who are gonna hit a $90,000,000 Server on a host that was split 4 ways. The multi threaded code was meant to detect performance degradations,

Topic 16 28:20

Developer photos leaked

Scott Tolinski

machine,

Wes Bos

processors

Scott Tolinski

which has been a cloud hosting service that's been around for a long time and does the job way good. Wes, do you wanna talk about Linode? Ready. I do wanna talk about Linode. Linode is a cloud hosting company for your applications, They're tremendous. And I I almost feel like you could have a whole podcast that's just web dev oopsies. It's like, here's a story about some web dev oopsies, and we'll talk about it.

Scott Tolinski

I finished up about 4 AM on Monday morning. I was cleaning out my files. All of this work was being done on a Novell server. Print Files had an extension of PRT,

Wes Bos

But, like, your laptop, rid Too scary. And, like, imagine a mainframe running an entire business, I would just go stick my head in the sand. I would stick my head in quicksand or yeah. Yes. Absolutely something.

Wes Bos

just so that it yeah. Your keyboard, if it gets a coffee on it, you might lose it. That's okay.

Scott Tolinski

Yeah. That is very funny. Alright. This next one is called Formatted the server, and as soon as I realized it didn't copy 10% of the files, rid I had that face. We all know the face. Yes. We absolutely do. Anyways, I tried to restore the files using some HD recovery tools, But they were all corrupted not by the formatting itself, but by the installation of the new OS.

Wes Bos

in free credit, which is pretty sweet. You could use that for some dedicated CPU intensive workloads Or just hosting your your own website. Thank you, Linode, for sponsoring.

Scott Tolinski

You can see where this is going, especially on the 3rd shift. Long story short, someone called my name. I turned around and knocked the glass of soda onto the console. It had to be completely replaced. The machine was down for 2 days, and my boss was not happy. Well, do you think? You're

Wes Bos

To maybe do a whole bunch of CPU intensive workloads, whether that's, like, resizing data or Better than that. Yeah. I would probably just put test, test, test. This is a test. Yeah. Because, Like, many times, I get little push notifications on my phone that say this is a test. I'm like, oh, some Okay. Some developer somewhere just goofed up, but at At least I don't have to look into it. You know, like, loremipsum, especially if you have just, like, regular users, they're gonna be like, what is this? Oh, yeah. They're gonna think it was hacked. Rid Then maybe if the hack is gonna be your excuse, then maybe that's the, that's the answer.

Wes Bos

rid They do today is they have dedicated compute for CPU intensive workloads. So if you need a mainframe Yeah. I wonder there must be, like, a rid flag where you don't you have to explicitly

Wes Bos

rid. On there so you're not sharing it with anybody. You can use the whole go yourself. Check it out at linode.comforward/syntax, rid. And that's gonna get you a $100

Scott Tolinski

speaking of mainframes,

Scott Tolinski

About the size of a dishwashing

Scott Tolinski

customizing

Wes Bos

Yeah. That's a rough one. Spilling anything, not so good. Yep. No. Thank you. Can you imagine, like, every time I knock something over, I specifically have my laptop rid. Up on a little stand

Wes Bos

going over a 1000000000 records and and Updating those records or parsing up music or anything that is CPU intensive, and you need to do a whole bunch of that work at once, they have dedicated

Scott Tolinski

my software package rid For Johnson and Higgins insurance brokers, I had their Asian contract for my insurance broker accounting package. I spent a good 40 hours on Saturday and Sunday making all the changes that were asked for, getting ready for a demo on Monday morning.

Scott Tolinski

I was a computer operator back in the late 19 sixties operating a Honeywell mainframe.

Scott Tolinski

with the console typewriter and printer inset in the middle on top, rid I had a soft drink on the console next to the typewriter mechanism.

Scott Tolinski

I had my own software companies in the nineties, and it was in Singapore,

Wes Bos

Oh. Oh, that sucks. That is funny. Rid That's very nice. Like walking into this, like, high end store in London, and there's just pictures of some guy Goofy developer. Rid With a Coke can beside them? Session replay.

Scott Tolinski

You wanna host your website in a place where you don't have to worry about accidentally spilling a Coke on your mainframe to bring it down? You might wanna check out one of our sponsors, which is Linode, Rid LogRocket, again, is a service to monitor errors and exceptions on your site. So if a user is causing some issues, they're stealing your cookies. You're gonna get a video of that. You're gonna be able to see how they did it, When they did it, where they did it, all that error log, network requests, all that and more. Check it out. Log rocket.comforward/ rid Syntax, sign up, and you'll get 14 days for free. And thank you so much for LogRocket for sponsoring this very spooky episode. This is This is actually honestly, as for a sponsorship read, this is like a dream episode for LogRocket, because you you could've gone anywhere with it. You could've said, is what's the wolf man doing? Is the wolf man is he rid. Destroying your header is Dracula causing errors in your console. Who knows? I could just keep going. Yeah. It's a good episode for LogRocket to sponsor, so sure ready. Sure they'll be happy with that.

Scott Tolinski

Speaking of mainframes, of the UAT space. And when we released some bug fixes, re It picked up the content

Scott Tolinski

also known as a dishwasher,

Wes Bos

and they have all kinds of different services. They're rid. Most popular for their Linux boxes that you can host the node app or Ruby on Rails app or whatever on. But I'm gonna talk about something else

Scott Tolinski

drink into the room, but we all did it.

Share

Empowering developers for over 286196620027 milliseconds!