almost 5 years ago Syntax Podcast
Hasty Treat - Webhooks
- 02:33 Linode provides hosting for Node.js apps
- 03:36 Webhooks allow servers to communicate with each other
- 04:53 Webhooks used for payment processing workflows
- 06:21 Webhooks are like automated HTTP requests
- 07:21 Webhooks sent on events, not immediately
- 08:14 Services provide interface to configure webhooks
- 09:33 Web developers implement webhook endpoints
- 11:02 Webhooks have JSON or form encoded payloads
- 11:48 Secure webhooks with authentication tokens
- 13:12 Locally test webhooks with ngrok
- 15:35 Consistent webhook URLs help debugging
- 16:58 Tools like Insomnia help test webhooks
- 18:25 Stripe's VS Code extension simplifies testing
- 19:40 Webhook.site verifies webhook payloads
Webhooks allow servers to communicate with each other
Wes Bos
It would send a web hook off to my server Then you you should get, like, an interface that you log in to. You can say when this thing happens, get, post, put, patch, delete, any of those HTTP methods, and then you can also
Scott Tolinski
and I'm not talking about Spider Man hooks. Did I already do that joke about Spider Man? I think so. So what are webhooks, Wes? Yeah. So webhooks are pretty common in applications
Scott Tolinski
Okay. Webhooks,
Web developers implement webhook endpoints
Scott Tolinski
It would tell you what the request was, what the body of the request was so you could see what data was being sent, f. What the response from the server was, which is really handy if you're trying to figure out what happened, as well as the ability to try the the webhook again, which is super, super handy. Yeah. The best webhook interfaces for the ones that I've, like, sent them will have, like, a full history. It's like sent this message at this time. You click Gone in, you can see the full object of all the data that was like, that to me is the gold standard for dealing with, like, knowing what's sent. But, yeah, most of the times, they do We expect you to send some sort of a specific response, like a 200 response or something and say, like, this worked. We're all good. In that way, they can know that the webhook is at least being received correctly. But, you know, on the receiving end, you have to think about these things as they're normal routes. These are just normal routes. The data's gonna come in as normal request would, and so much of what I I I think I would wanna get across in this episode As these webhooks are are nothing more than just requests. So if you have, like, a server side route that's accepting this request, the normal request coming in, it's just not being initiated Sick. This episode is also sponsored by LogRocket, Welcome to Syntax. In this Monday hasty treat, we're gonna be talking all about webhooks. My name is Scott Tillinski. I'm a developer from Denver, Colorado, and with me as always is my man, the man with the bus. Bus.
Scott Tolinski
From a user, so to say, like I mentioned before. So on the receiving end, we accept that request. We send back whatever we wanna do. And in the middle Between there, we do whatever we need to do with that data, whether it is updating user records or firing off other things or other events, that type of deal. Talk really quickly about payloads.
Transcript
Wes Bos
storage, which means that if you're using AWS packages in any of your stuff, you can just hook up Linode with it, and it would just work Just pretty sweet. So if you wanna host your website or your application or your images anywhere.
Wes Bos
Compatible you sometimes you're not just running a server yourself,
Announcer
Tolinski.
Wes Bos
Yeah. Linode
Scott Tolinski
CSF.
Scott Tolinski
Wes, you wanna take Linode, and I'll take LogRocket?
Wes Bos
block storage. They have AWS
Announcer
Monday. Monday. Monday.
Announcer
Open wide dev fans. Get ready to stuff your face with JavaScript, CSS, node modules, barbecue tips, get workflows, breakdancing,
Wes Bos
365 So check it out. Linode.comforward/syntax.
Scott Tolinski
Like many other things in development or one of those concepts that's way scarier than it actually is. You hear the word webhooks, and you think like, Oh, man. Is this, like, Spider Man or something? No. It's not Spider Man. It well, it's like a really friendly neighborhood Spider Man, but it's it's pretty cool. So Before we get into that, let's talk about some of our sponsors today, which are Linode and LogRocket.
Wes Bos
Kubernetes, For events that happen, when something happens, n g r o k. Yeah. There's another one like Local Tunnel. That's That's one of them as well. Local tunnel. Yeah. Yeah. I used local tunnel for a while, and it was great. But there was a problem with local tunnel Webhooks is that you have a service, whether it's a CMS And if one of the webhooks failed,
Announcer
soft skill, web development, the hastiest, the craziest, the tastiest TS, web development treats coming in hot. Here is Wes, Barracuda,
Wes Bos
Create a free account. It's gonna give you a $100 in free credit, and they also have 247, Generally, the body of the request will either be form encoded data or JSON encoded data, and you have to either parse that out. Or if you're using any sort of Server side framework, they'll often detect that and and parse it out to an object for you. Mhmm. It'll also tell you what is Sent. And then more importantly, it will also come f r. Was that you? They'll be like, yeah. That was me. Don't worry. That's legit. Or no. No. No. No. That's not a legit token. Definitely make email address and lock it because more often than not, someone stolen a credit card and used it to buy a course. So you can automatically just lock those and show them, hey. Sorry. Something happened. Please get in touch with us, and we'll we'll figure it out. Do you use any webhooks in your course platform, Scott? Yeah. And we mostly use them again, like, in our payment flow, especially for subscriptions.
Announcer
Boss, and Scott, El Toro Loco,
Secure webhooks with authentication tokens
Wes Bos
So the way that you sort of Authorize these things is that you can make sure that the like, often, like, Stripe will say, these are our IP addresses. And any request send it over. I I actually had a call with one of our other sponsors the other day, Sanity.
Wes Bos
because f. You will have a secret in your own service environmental variables, and they will send you a token. And using that token and your own You can see if there it is legit or not as well. Yeah. To at least verify.
Scott Tolinski
You know, the information they're sending you is correct because
Wes Bos
tokens, for debugging then let me go ahead and you can select The method
Wes Bos
they want me to mail them a T shirt, and they said 49873 which is awesome because, like, it Probably one of the most pain in the butt things That isn't coming from one of these IP addresses you can consider to be invalid. So you can just reject it or put that in a firewall rule if you're using a a service that has a Firewall. Mhmm. But then, also, you can take those authorization the
Wes Bos
I think you can also
Tools like Insomnia help test webhooks
Wes Bos
that I can rerun forward slash API forward slash webhook. And then I have the JSON body, and that's just a test with a unique URL And you can get the data out of there. Or email addresses as well and see what data comes in Oh, cool. Yeah. And ping it. A lot of people says make sure you use ngrok, approach, What I'll usually do is go into my do it without the request to the servers
Wes Bos
to work on is debugging webhooks. Yes. Yeah. And Stripe, just put that all in your editor. So So there's a a Versus code extension that will allow you to trigger and generate common scenarios of things like credit card rejected or I find this to be really, really handy And when the event comes in, over and over and over again. And then I can work on my local development server. And that's really nice because human Head
Wes Bos
777 and you can go back to the service Like a Snip card or or Stripe and say, hey. Somebody just came over and said will email you if a webhook sends, like, a 404 or 500 error back.
Locally test webhooks with ngrok
Scott Tolinski
is testing them because working locally, you might be thinking, okay. You typically when you you work with a webhook somewhere like a web service like Stripe, they give you a place To have maybe, like, a a URL you can paste in there, and then you can send test events. Like, usually, there's a button to send a test event. In Stripe, I remember that you can send, like, Any of the events, which is awesome. But in Braintree, it's just like send a check event just to send a check, and that's all they do. But either way, they usually give you an option to test these things, and You as a developer might be seeing like, okay. Let me just paste in the local host for 3,000
Scott Tolinski
And let me tell you that doesn't work because that that's a local URL. Right? That's not gonna send anything to your computer, yeah. Yeah. That's no good. Like, I think the next hardest part about webhooks besides the fact of getting over the fact that, you know, it's a jargoning kind of thing, right,
Wes Bos
page and not know what to do, so I had to move over to ngrok. And ngrok doesn't let you have a consistent domain name For free. Yeah. For free. You gotta pay. Yeah. So I had to pay for engrakh, which is not at the end of the world. It was worth my money in that case, and And ngrok was rock solid, and you want a consistent function that will accept the incoming request, And that is a great way to hook different services together. So so when somebody saves a bunch of data, I need to save them to the database. I need to Create a shipping label. I need to change their status from pending to save. A couple of things that needed to happen there. Another example is You wanna trigger a build of a website. So one of our other sponsors, Prismic, Then they could potentially be sending like, oh, well, send me T shirts for free or something like that.
Scott Tolinski
to essentially your IP and your port to send that message.
Scott Tolinski
You might be more worthwhile to give a little bit deeper of an explanation there if one needs be, but The main one of these tools I think that people use the most is ngrok,
Scott Tolinski
to your local host port 3,000. It just doesn't work. So there's a number of Tools that people use for this type of deal, and what they basically do is they set up a public URL that redirects
Wes Bos
And that's great if you want to be able to show somebody a website that's hosted on your computer,
Scott Tolinski
here.
Services provide interface to configure webhooks
Wes Bos
and then it's up to you to go ahead and send a successful
Wes Bos
a product rebuild itself based on a URL that can get pinged, webhook sim card or or whatever and just copy out the body because it will show you exactly and say this person bought a shirt and it would give us a huge payload of data. And I would say, okay, well, I now need to go ahead and do a few things. And one of them is Another one I use is Stripe or PayPal. If somebody uses a stolen credit card to buy a course and that is chargeback happens, and working on both webhooks as well as just like regular REST APIs. It's not specifically just webhooks. Yeah. They got a cute little website here too. Stripe has a, a Versus Code extension, to oh, no. This is sort of the opposite end is that You can ping webhook.site Intermediary similar to ngrok, which is PHP. So if you're using Using PHP, that's good. I bet you could use this with other stuff other than it's got a beautiful website.
Wes Bos
writing some code. This is the sort of the receiving end. You are writing an endpoint or a GraphQL
Wes Bos
send back. A lot of the good webhook services
Webhook.site verifies webhook payloads
Scott Tolinski
Holiday Inn.
Announcer
CSM.
Wes Bos
We can call them. Yeah. It's a request. It happens from one place to another place. You either accept it. You're sending it. Whatever. Easy peasy. Alright. I think that's it. Thanks very much for tuning in, and we'll catch you on Wednesday. Peace.
Wes Bos
I think that's that's all we have Say about webhooks, they're pretty simple things. They sound much more ping this URL with the following data and let them know that this happened. So The best way to sort of understand what that is is just looking at some examples. So when
Wes Bos
interesting than they are. They're not anything special. They're just HTTP callbacks They have, like, this thing called Grok, which is the ability to query anything from the data. And you can use The Grok queries in your webhooks. So you could say whenever where you have a server or a service, and it needs To ping or send some data to another service when something happens. So I like to think of them as callbacks In something like Insomnia or Postman or something like that. I specifically use Insomnia. And what Insomnia is is it's a little I've never heard of that. Yeah. It's a little desktop app that you can download and you can save common requests.
Wes Bos
workflow automation tool. What's the one everyone uses? If This Then That. Right? Yeah. A lot of these low code, no code solutions is or Zapier. That's another really good one So where you take an incoming request, you do a little bit of something to it, and then you send it off to another service. So if you're just stringing services together, is is actually webhook to webhook. This is not even code that I wrote is that A Vercel has the ability to to the wider Internet.
Wes Bos
and you need someone in between.
Wes Bos
So that's another good example. Someone said expose dot dev is a good solution will have a URL that you hit that says webhook or chargeback, and it will say the person with the following email address It did a chargeback, and then I can then look up the account that has that event handler or my endpoint handler.
Scott Tolinski
is something I would use just to verify the data Correctly. Sometimes you don't even need to test that webhook. You just wanna see exactly what that data is coming in for the the services and sending doesn't have, you know, great documentation or anything like that. So that's great. Or if you're just going service to service. Right? Like, if you're using Mhmm. Someone said n 8 n dot I o. It's like a extendable
Wes Bos
Peace.
Scott Tolinski
Low code. No code.
Webhooks are like automated HTTP requests
Scott Tolinski
Computer user.
Scott Tolinski
a server, now by a user. Right? But there's not a whole ton of difference between a request being sent from a webhook and a request being sent from a user. It's just a robotic
Scott Tolinski
so that you have access to what you're paying or not paying for. So we get that information, and then we either downgrade you or upgrade you or whatever based on that webhook. I also actually use them for our video platform too whenever we upload something because we upload a file, and then it needs to process for a little bit on their end, and then you'll get, like, a message saying it's ready. So we use webhooks like that. You know, I like to think about them as being just like, you know, when you have a user interacts with your site, right, or anything, they're sending a post request to your server. But this is sort of like A request that's going from server to server that's being initialized
Scott Tolinski
by
Consistent webhook URLs help debugging
Wes Bos
but then you're thinking at least $5 a month for a a VPN or VPS,
Wes Bos
requests
Wes Bos
n. Which is almost the cost of ngrok per month, so you might as well just let them do it and figure out all the NAT traversal and firewalls and all that pain in the butt stuff. No. Thank you. Yeah. I'd rather just have it have it work, please. Other stuff I've done in the past is I have replicated What else do we got here? I think that's just about it. I asked on Twitter for any tips that people have. Someone said webhook dot Site is really good for testing. It looks very similar But they throw a little, like, hey. This is, not a real website often in front of it Mhmm. Where you just have to press okay because I think local tunnel is used a lot for People getting quick and easy domain names for scamming people. Mhmm. So I was running into a problem with that because the webhook would hit this How's it going, my guy? It is a lovely Monday morning. We are just hanging out here in the dev shack, whatever you got, the full stack shack, and we're just cruising, man. How you doing? Doing pretty good. Pretty good. Excited to talk about webhooks. I was so surprised that we are almost at 400 episodes, and we've never done a a webhooks explain here, we've talked about them, but we've never done a whole show on what webhooks are. So that's what we're here for today. We've said the words webhooks, and webhooks,
Linode provides hosting for Node.js apps
Scott Tolinski
And you'll get 14 days for free. This is one of those tools that makes solving bugs so much faster. Strap yourself onto a rocket and blast off, and You will be solving bugs in no time with LogRocket.
Scott Tolinski
which is the perfect place To see all of your errors and exceptions happen as they happened to your user so that you can see them as if you were your user. I'm talking about they give you a Scrubbable video replay with the network logs and console logs and error handling stuff. Anything you could possibly need to debug this thing, it's all here and more In addition to having that scrubbable video replay, I was holding that 1 in. I wanted to do that earlier, but I didn't wanna interrupt you.
Wes Bos
Support. No tiers or hand off regardless of your plan size, which is pretty cool. Thank you, Linode, for sponsoring.
Scott Tolinski
it's like imagine trying to solve a crime while having a videotape of the crime being committed Along with all of the information
Scott Tolinski
about everything that was involved in the crime. Describes my code. Yes. The crimes were committed. Yes. And LogRocket will help you solve those crimes. They are McGrath the crime dog here. So logrocket.comforward/syntax,
Webhooks sent on events, not immediately
Wes Bos
generally, the way that as web developers think in terms of that anyone can hit. And if somebody figures out what data to appropriately are Of webhooks, if both of the services are using AWS, you can use the Amazon specific version, which is pretty nifty. But let's Talk about the sending and then the receiving end of webhooks. So, and say, this and it's pinging localhost
Wes Bos
or a something like Stripe or PayPal or it's an online store, Users, you can change if you're in the middle of a like, you save, retry it, save, retry it, save, retry it. It's Kind of annoying to go back to the service and and retrigger 1. So if you just replicate what that webhook looks like In Insomnia or Postman is another popular one. There's all kinds of them out there. It's neat. Yeah. It's great because you can just save pretty common ones. You can share them with then ping the following URL. And in that, you can say what data it will get, Any headers that need to go along with it, we'll talk about auth in just a second. And then on the other end, it's generally you, the web developer, So I've got little like a snip card order doc completed webhook. I've got an example right here, And, like, I know specifically, I use Snipkart for selling my T shirts.
Stripe's VS Code extension simplifies testing
Wes Bos
I will just either console log or write the body
Scott Tolinski
test over and over again. You're typically going to, like, a web URL, clicking a button, checking to see if it came in correctly, doing the right thing with that data. Maybe a it's not one of those things that's easy to rapid test. I should also say with the Insomnia
Scott Tolinski
things like that. I like these tools. Webhook.site
Scott Tolinski
Something like that. It's especially a pain when you need to
Wes Bos
All the data that it sent over.
Webhooks used for payment processing workflows
Scott Tolinski
You know, a user's subscription can be updated for several reasons, whether or not Past due. It's been charged again for the the next month. It's been renewed. It's been, credit card's been updated or something like that, and, the payment processor will send The webhook with all of that invalid information or relevant information to our server, and now we don't store any of your information in that regard to our page, but let's say You were past due. You didn't pay. We wanna be able to make sure that we adjust our roles for your account accordingly
Wes Bos
And Prismic has the ability to send a webhook Yeah. It's just a robotic computer user, and they will often not be sent immediately. They'll be sent when something happens, when Credit card is approved. That's another popular thing is sometimes you can take credit card info, but the charge doesn't go through. Like, Amazon does this. The charge doesn't actually go through for 5 minutes. So I'm assuming on their end and somebody actually even sent me a tweet that says Amazon has their own version off headers. You can change the post. You can set cookies that get sent along with the request.
Wes Bos
Stripe will send me a webhook is Updated,
Wes Bos
when some data changes. So you can take the webhook from Vercel, Put it into Prismic, and whenever any of the data in Prismic is updated, it pings Vercel. Vercel rebuilds the website and deploys it too.
Webhooks have JSON or form encoded payloads
Wes Bos
usually, they will come with a bunch of headers that have tokens in them. And those tokens can be used to check if the request coming in was legitimate or not because you have these publicly available URLs on your server is a URL on my website. So somewhere in the URLs of my website that sells courses
Wes Bos
hit that webhook with, Where basically, the way that is you assign, like, west boss dot local tunnel dot me, and then that's your URL for your machine, and it exposes whatever's running on 40,000 CPU. They have high memory. They've got GPU, that is more than $50 of the request to a JSON file on my computer, and then I have an example of what An entire request looks like. You can also usually go into sure that the requests coming in are validated against the service that is supposed to be sending them. It is another back and forth, Which can sometimes if a webhook is waiting on your data to be sent back, that can add a little bit of overhead in terms of time for the data to come back, but you You kinda have to do that. Otherwise,