Logging

I

a storage space of things with just text, just by writing a lot of text. Yeah, you get

maybe query parameters, maybe body parameters. If it's a form, submit you can see. And then I like, when I was having an issue with a specific thing The other day I went in and added more logging,

100,000 then the email being the the link in the email being clicked, an account being created, like, log Files. We'll talk about what to do with these log files, but I've had log files get into the gigs and gigs. I once very early on, when I was just logging to log files on my server, Work in a large context. Right? No. Because you would be like I certainly have had,

because you're running out of space and to only be like, wait. How is that possible? Oh, wait. Let me go check the law. Oh, boy. Yeah. Big old logs. And if you're it. Flocking every request that would, you know, we often forget as developers that, like,

But I think I don't know. Do people do that? Do they log the entire request? I can't imagine. That seems like way too much text for everything that seems like that would never

and I measure that time. And if that time is really fast, you get a rocket ship. If it's

when the request comes in, and then I stop counting when the response is going out Just via middleware,

And you're saving a lot of text or a lot of information. Yeah. I mean, that's it's pretty easy to rack up actual

text is bytes,

And then when I cleared out the logs, they would go. And that's why you you have to have sort of a better approach for this type of stuff. Yeah. I've certainly gotten the email that says your Server space is limited, And then I put some information along in there. So things like URL, in your application to to trace through what's going on. So it's It's actually kind of funny that we talk about logging because a lot of I've been sort of taking my, like, logging a little bit more, Let's let's talk about our sponsor today, which is Sentry. Sentry is the They're an exception.

Why am I seeing so many turtles in this specific route, but not other routes.

Yeah. And it was just getting slower and slower because there's, like, not enough space to run things.

stuff that you specifically need. Unfortunately, at least I don't think you can't log the entire request a quest object because that would be

absolutely massive and your your logs will get pretty large quickly.

succeeds. Alright. 400 is my number. It's

So you could, like, dial in exactly what the rate limiting is. Let's talk about where to put logs. So at the very basic,

so they they can't. You can. And like

and I had a little set time out in a loop Because I knew that if I didn't Yeah. Then it would get hit by the rate limiting. So I was just like, alright. If I set this set time out loop to be 200 milliseconds, So, having that time code that says this happened at this time, Really important to be able to nail down exactly when when that happened.

log files in there that you can, And, Oh, like, of course, you have their user and purchases and things like that, but they probably also have information about and forever will put your logs in a log file. And then when you restart it, I think it deletes it or something like that. And I was running out of space on my server.

6 days to get all of the possible combos through, they can do that. Yeah. I actually

you could go in and download and whatnot. Generally, these log files are massive, massive, Text editors are very good at opening up files that are 2 gigs

For this website, and even if it takes the request along. Like at a certain point, that's also what, requests a day and And then after that, they're deleted,

like a like a straight up v p or VPS server, Why might you wanna log something?

Linode DigitalOcean, something like that will generally have

You can log that to show, but the bots are smart enough to figure out what is the limit

it's funny. I was just writing something the other day to do, like, automating browser tasks, like, a specific instance that happened, I found it to be way easier to find a specific instances or specific issues or really dive into the logs a little bit more with these tools. But at the end of the day, like you said, they're they're some something happened. Right? And you want to investigate, so you're pawing into the logs, Secure needs to be secure in your log

And searching through them now. So it's not so much of an issue.

Alright. It fails after the first one. I set it to 300, this just how much logs can both

Back in the day, it used to be hard to just open them up in your text editor.

pollute and save your bacon sometimes. You know, sometimes if there's too much logging, too much noise, you can, you know, overdo much do certain things. But, you know, part of what Is good about having logging is that you can often go ahead and paw through the logs in a more structured and organized way. So, You know, too much information isn't necessarily a problem.

And then progress on that video being and you can see all of that in your console It's a bot trying. I don't know. I think it was something like via logs They can parse out their logs and and recreate that data in the database.

I'm gonna tell you what we log,

is I'm talking, like, server side logging here, specifically.

personally, what I like to see in a log is I like to see, to Syntax.

You know what it was? until I until I jumped in and go, oh, like this form should probably have a have a CAPTCHA on it so that the bot can't do that.

to see like, okay, like, is somebody being malicious here? Is this coming from the same user? and now they are getting into session replay, It went on for about 6 or 7 hours some of the stuff in this episode where I talked about I'm not using it. I'm mostly still just a console logger. But, the request come in every 2 seconds with the the the new piece of information that they were sent in. And They don't do it. They do it just enough that it doesn't hit any, You would know if the middleware was run before it. And it's kind of hard because you don't know the functions where they are, you know? Yeah, totally. But but we'll we'll throw that on the books. Middleware ones coming up. What else do we have here about logs? I think that might be it. Yeah, I think that's it. It's helpful to log Log stuff that you think you need. Put them in because sometimes you're going to need that information Sometimes I get emails from people and they'll they'll show me their console. And I was like, well, kind of glad I stuck that in there. But on the same breath,

to and the source of the image was undefined.

people were going to I have magic links in my email, AWS like, could not figure out why every couple of weeks the server would slow down Additional logging Wherever they are, they you always have to do like an if statement to check if it's specifically there, which is probably better type practice. But it would be nice if somehow you could Worst case, I've certainly not had to do this before, thankfully, but I've heard Is it like a browser extension? Something like that. I had to add a whole bunch more log in. Then I was like, okay, I see what's happening here. And then I went to my template.

because I had,

magic links.

it was trying to just, I guess when you you have an image tag, it tries to request until I rebooted it. And finally I realized this because it. I was using forever, logging

Yeah. One one, like, really important thing about a log is the a time code in which this thing happened.

There was an undefined request in my log, and I could not figure it out for the life of me. I finally finally trace it trace it down.

It's it's funny, that is happening The total response time was. So I start counting

adding a course, buying the course, checking out, adding a coupon, getting the email being sent,

what was triggering this undefined request. You know, like, it's not an error. It's it actually it was causing no issues to anybody, except somebody starting to view a specific video, as it goes through. And I specifically had an issue where will allow you to do what's called log drains,

visiting the website,

and then we can maybe talk a little about what you should log. But, you know, I I think, Shorter, you get a a bunny rabbit. And if it's really long, then you get a turtle. I've been I've been thinking about expanding this a little bit more, but It's really simple just to see a turtle, a rabbit, or rocket ship in your logs. It makes it, like, for me,

the current URL as the source, and that was triggering

I like those types of things in in my logs. I also like to know what type of log this is. Is this an information log? Is it an error?

accordingly, so that an error log could have a red error or something. So that way, when you're looking at your logs, the errors stand out And the information ones kind of hide back or any of those types of things. Oh, that that's a good idea. I I so I'm half Excited to do this episode just so I can do a little bit of improving on mine because maybe we should talk about that specifically. Like, how do we log right now? So right now what I'm doing is I just have, like, different areas of the, You can log out that information if you just wanna see what endpoints are are being hit with what specific data. You can log that, or something just as simple as, like, an email was Specifically sent. We'll go a little bit more into the different types of Like a like a catch all path set up for the ID in the URL. It was it was catching that one as well and literally the the image pass. So I went into my template and I go, Oh, I had like a typo in my template and I fixed the image tag and whatnot, but that was driving me crazy. And it was only until I added and people were going to

I I like what type of request it was, a get post put, whatever. I like the path that was requested.

A magic URL, whether it's a log in, whether it's a progress update, whether it's somebody has invited somebody to their team account so they can Go in on it, whether somebody has changed their email, all that type of stuff. And I just basically have like an ID or a name for each one. Like I and I put it in square brackets, just magic.

I like the total time of the response.

information.

Login attempted, but login attempted with So not just Either your service will handle the logs for you. They'll stream them in, and you can just go to logs in the dashboard. Or if you're just on, like, a, and they are really helpful because you can do things like

normal, like, informational logging in production.

as part of when somebody does request to be deleted from you, you have to think about,

that is

Get to step into areas of You certainly start services ESLint has like a scrubbing the logs what about like development, staging, production? you can set different levels of logging. It goes From, like, error warning

A lot of people I don't Specifically do this. I really only just log stuff that I I think I might need, but, seriously

user email address. Obviously, don't put passwords in you literally log absolutely everything.

doing, and that could be helpful to follow

maybe you have that in a log somewhere that also should be scrubbed. Yeah. I mean, it's important to know that it could just become another attack vector.

And that could be actually really helpful, specifically, like I know with ESLint.

to get it to get it, some visibility into what's going on, especially in development.

all the way up to verbose and silly where you

Yeah. Yeah. That information is sitting there. Or even like other employees that shouldn't necessarily have access to that information. You put all this work into permissions. You have to be careful that you're not putting anything in the logs that other employees could have access to. Yeah. Especially if it's insulting to those other employees. Yeah. So we'll talk about like

So if you do have something, Head on over to syntax.fm And these can be used primarily you know, we often just throw it in for a quick debug or something like that. But

personally identifiable And it's hilarious. I I don't use Postgres literally anywhere, so that wouldn't have worked. But it's not somebody sitting there trying to do it.

But in production,

and someone get axe gets access to your logs, then that's as bad as, You know, them getting access to the information itself.

Yeah. I I pretty much keep most of my,

you know, Yeah. And and also, just to be clear, logging is not just for

that shouldn't have happened or I need to figure out why somebody specifically

figuring out

Logs Are often helpful as sort of a treasure trail to That's awesome. What else should you log? Descriptive messages about what happened.

what happened, in the last couple months just because I I had some, like, gremlins because at some point you should you shouldn't keep logs around for for that much longer.

hit this URL with what data and it caused them something.

If you need to, everywhere from just you, the developer, needing to figure out, okay, well,

logging. So it helps piecing together things, helps us find ability.

it. On your server or your client,

in At the end of the day, logging is basically giving you visibility

But

to some action

explaining the requests that are coming in. Maybe it's just saying, oh, you know, a message came in at this time or whatever. It depends on what system and what type of logging is enabled in these types of things. And And many even, like, CMSs have, like, baked in logging as well outside of the JavaScript context.

depending on how you're doing this. But at its very most basic, it's a console log. It's a, I got here, but Yeah. It's more complex. It's a a git request was hit to this specific URL at this time, And it took this amount of seconds to resolve or did not resolve or whatever.

You also probably have seen logging Welcome

to output something either on the client side log or your server side log, So what should you log here?

Logtail. I've used

looks nice. Yeah. Log tails great. You know, I've used I've used

so I probably shouldn't be paying a service just to be,

making it a little bit easier. But if I was on a big project with a big team with more requests than what I'm dealing with right now. I think something like this would be essential.

you know, we're talking Like, it's really helpful to be able to see. Alright. A get request came in. You know, this this is just at a glance, like, is what's going on in the site, but also At a glance, even though I do use tools to

LogDNA,

The the big boys for just straight up logging, Log Tail and Paper Trailer are probably the 2 biggest ones. I've used both of them. Both of them, you know, they were great to have, especially as diagnostic tools. When you were looking for, yeah, you say how is this different than error handling?

has logging. Dynatrace has like, there's a lot. Right? Yeah. This is like a big a big thing where some of these do more than just logging. Right? But, unless you're,

Just kind of unneeded expenses for our level of what we what we're working with. You know? I I can dive into the log files no Problem and still get access to the same information, Yeah. So at, you know, at its very most basic, everybody has probably We tossed a console log into their code at some point You're logging a couple lines for every single person. Those add up pretty quickly. Yeah. Oh, okay. So one thing I also wanted to mention that I do, just for fun for our Information logging, especially in our request. I I do a little emoji based on how, like, long or short

I just I yeah. I I don't need it for our size. So when you actually go ahead and log, I said earlier I'm simply just using straight up console log. But there are lots of other packages that will do these for you.

Paper trail, those are 2 of the big ones. Some of the other ones

if you do have an error in that goes into your century or whatever, What? It was an image tag

evaluate my speed of the site Mhmm. It's really nice to see at a glance in the locks, The, emoji that I put in there just for fun. And these are only server side, so it's not like these are, you know, polluting the client side logging experience or anything like that, which honestly, you You know, your site probably shouldn't be outputting too many things to the the client in production

and then I could go to the breadcrumbs. Okay. Okay. Now I have a little bit more insight into To what is happening. So that was super handy too. Yeah. We often talk about the name of the game being visibility

Be included in the breadcrumbs, and that I found is very helpful because

on mine because GDPR

and all. Yeah. Yeah, I don't do any of this because sometimes I look at the

meaning that, okay, we'll keep logs for we'll keep the last 10,000 logs or we'll keep the last week worth of logs.

or there's lots of services out there that are specific log drains, So bots will try to hit

that do hosting application. So whether it is But if you do need to keep them for a little bit longer, then you have to port them into something else. So whether that is as simple as a text file on a silly mode or a verbose mode where it will log every single thing that it's

Search for specific words. You get syntax highlighting. You can filter sorting filters or all that type of stuff graphs It's kind of interesting, you are seeing people go through the process of a month for something like this, you know, but I I look at it, like, specifically this one better stack.

Log tail Hey. How's it going? Not too much. I, And the idea of the magic link was always undefined. It it was the correct magic link, and immediately after, it was undefined. And I could not For the life of me, figure out

But most All right, breakpoints are 4 in your code base, but it can be nice to nice to see what it is doing. Yeah. I mean, there are times when you just just dump them in to try,

and then just use straight up console log for everything else, I guess. Oh, yeah. Yeah,

requests and do this in more of, like, a middleware type of way Is it, and these can things these things can be color coded

It's really robust. It does a lot for you.

Low overhead. It's, overall, a super nice logging package. But,

A package in every file, which it's I think maybe less of an issue now with the auto import in Versus Code. So maybe I should take another look. Do you use a specific package? I always used Pinot before p I n o from, you know, the awesome Matteo Colina.

intercept errors or intercept Debugging, but also just for general information and understanding

And this was the one I used because it came with Fastify Sematext

and Mercurius, and I really like it. There's it's a really just really visual in terms of alright. It. I'm working on this one component, and I've been seeing a ton of turtles in development.

in regards to that same bit, how is this different than just errors and what's going on in your application? Sometimes it's it's it's a little bit also,

of what's going on. Right? are Sumo Logic, being logged,

letting you know exactly what's it happening around what our potential errors, but just what's happening in general.

basically,

Yeah, yeah, like I just I just opened up my logs right now and I'm seeing people.

they don't have the data of like maybe 10,000 sales went through. They can sometimes piece that information back together or at least the important bits,

At the same time, more targeted and less targeted You know what type what, Yeah. You know, also,

because you can have less targeted info

of things crashing databases going down, whatever. And often developers have to, and all these rules around private information and tracking or whatnot. So you certainly make sure you are up to date With all of that, as well as

Because,

2 situations where you think that there is going to be an error or could be an error to give you more context into what could be happening in that given instance.

let's get into it. What You wanna give us a start of, like, why and what is logging? Century has this thing called breadcrumbs, which will it will show you, like, which functions were called up to the error, but anything that you console log will also

the stuff that Sentry does makes it so that you don't necessarily have to piece together logs. And we'll talk about what the difference is

But check it out at century. Io. Use coupon code TASTE That is going to get you 2 months for free. Thank you, Century for sponsoring.

which will allow you to This is not part of the sponsor, but one thing that I really like is

between those types of things.

literally They do so much more of that. They do performance, they do profilings, Other information about

inside of both production and development environment. So my name is Scott Talinski. I'm a developer from Denver. And with me, as always, is Wes you know, Trying to have fun with it, but you gotta kinda remove some of that stuff. I leave probably more than I should in Logs dot I o, fails. 400, instead of trying to use this in a sense where you're dropping it in like a console log, It's something like this, like a service I would probably try to, like, intercept from other frameworks or other things that you've used where you're it. Automatically just getting some things coming in on your your console explaining what's happening maybe at some given point. Maybe it's,

In my app that I couldn't figure out, like, where it was coming from and and what would even cause this request and and whatnot. And, like, I just didn't have enough insight into like, it Wasn't errors or crashing or anything. It was just, like, couldn't figure it out. You know? So we'll talk about, like, what that is. So I I sort of went through and updated my logging. I'm still not like

Really good logging within your application

On this Monday, hasty treat, we're gonna be Talking about logging.

Well, yeah. Let's let's get on into it. Yeah.

Boss.

Logging is something that, well, we sometimes do just to debug. And many times, people overlook the usefulness of having And don't forget to subscribe in your podcast player or drop a review if you like this show.

a bunch of, like, PostgreSQL

Yeah. That's the other thing is, like, bot logs as well.

injections into a form.

and debugging, right, knowing what you're trying to fix. Yeah. And that's just really where logging comes into play here. I think one thing I really love seeing come into my logs is that somebody tried to access WP admin and got a failure.

Common endpoints, bots will try to submit forms with with random data. I had a bot the other day try to submit, Winston is the big one that's usually included in Express apps and Node. Js space.

specifically, maybe an error on your site or,

going back and referencing it is kind of pointless. Right. You'd say at some point in some time, this this event happened. But if you knew when an event happened, logs. Datadog has logging. LogicMonitor

Because without having the time in which something happened,

the craziest,

the tastiest web development treats coming again hot. Here is Wes, Barracuda,

Monday. Monday. Monday.

Boss, and Scott

Open wide dev fans, get ready to stuff your face with JavaScript, CSS, node modules, barbecue tips, get workflows, breakdancing, soft skill, web development, the hastiest,

CSD.

after In your logging and maybe we should talk about that really quickly is

to automatically parse Form data from our forms and make them just available in in SvelteKit form action. I saw that. You you published it. Yeah. I published it. It's like I've had this thing in my code base for a little while. I actually have a few of those. Like, I have a SvelteKit logging one, and I have her off. It's like, hey. It should take me cup couple minutes to put this up live somewhere. So Yeah. Yeah. I Specifically, the the one thing that has got me with the middleware I I use tons of middleware, but I never like, I wish that the TypeScript types would update You know, I would probably But also you can have more targeted

for a full archive of all of our shows.

Peace.

And you realize, oh, shoot. I probably should have had better logging in place. So take a take a couple of minutes and maybe add some better logging to your app. Yeah. Absolutely.

Alright. Thanks for tuning in. Catch you on Wednesday.

in SvelteKit

that makes sense. We should do a show on middleware. I I don't think we've ever done that. Yeah. I just wrote a little middleware that I I think I had been using, see what the user had done For requests that are after the middleware, you know, like before the middleware, you know that it's not going to be there. And after the middleware, They allow you Do different types of logs and pass context to it and all kinds of good stuff like that. I've never liked that because I like the idea of that, but I hate having to import

But it was kinda funny watching

like throttling. Yeah. Rate limiting. That's so that's another thing you should be logging is any throttling or rate limiting that happens in the app level. You Modeling a rate limiting that happens in the app level.