almost 3 years ago Syntax Podcast
8 Tricks When Using the Fetch() API
Streaming fetch results
Wes Bos
we are like, I've been diving a little bit deeper This is something you used to not be able to do. It's been around for a couple years, so it's in all the browsers, is if the server And then there's also a dot JSON on the end, Is you can use instead of dot JSON, you can use dot text, and that will give you or work with a web stream It consumes it. Right? It consumes it. You can't use that again. Right? And there's there's a property on there called body used or done Mhmm. That you can access to see if it is done. But I we talked about it on the stream show with the dot t. There's also a dot clone method, which you can basically clone a stream. So what I did here is
Wes Bos
You don't have the entire MP 3 just yet, and that's because it's still downloading. So that first response
Wes Bos
the data yet. So if you fire off a fetch request for, when you call JSON dot parse on really popular library out there called Await2JS, So the next one is is something that we talked about on the Rust episode, which I think is coming out on the Wednesday after this, and that is request and response Firefox dev tools will Automatically copy all working with Hano JS. We had Ryan on to talk about remix.
Wes Bos
is sort of it has a bunch of things. It has some headers in it, await fetch to return the data, but you can tag a dot then log progress onto the end of any fetch, and you immediately have then and return dot JSON.
Wes Bos
and in that response is not necessarily Wouldn't it be nice if you could stream If you fire off 2 requests, there's a possibility that 1 would come back after the other and not in the order that you're expecting.
Wes Bos
And it turns out that that first response has a lot of handy dandy stuff in it. Wait. What? And we're not we're just supposed Supposed to ignore it? We're supposed to Yeah. Actually learn about what that is? Okay. Yeah. There's there's some kind of cool stuff in there. And as we get into streaming and I have a button on the News Syntax website All those good things that you're used to, and then you just pass view the progress, and I used the other one just to return in
Wes Bos
start to stream it stream in the data.
Wes Bos
we did the episode Into that 1st part I cloned it.
Wes Bos
returned to you would keep things clean. Yeah. It's it's not all that All that useful every now and then, but, you can do it. You sometimes need to programmatically This many bytes. So you know I have this many bytes so far, and it's this big in total. So, therefore, If you wanna use it twice or whatever. It's it's kinda weird how you can't use it twice. It makes sense. And we talked about that on the streams as why you can't, but the the error message is hilarious Explicitly in a function to handle errors or explicitly in something like I was talking about earlier to try to parse it. I don't like putting try catch in my regular The entire time. Wow. So that's kinda the benefits of obviously, you wouldn't do that with a fetch request. But, and start to view it. So if it is text you can use new request, new response, and new headers, Request headers wait for the next chunk.
Wes Bos
That's being streamed in. You can and then for some reason you But Now I'm back from the transcription. Now I've sent it off to Chat GPT. Now I'm back from chat. You know, like, you can you can give it updates as to how it's going Streams.
Copy as fetch in devtools
Wes Bos
on with you, and then I believe the Chrome Dev Tools has one that's copy as Node. Js fetch, And it will do that. You just basically wrap it in. And, honestly, I think this is how it should have worked what is You immediately have the zip file ready to download. You don't have to sit there and wait for the thing to zip itself up because it's been streaming into a zip file So if you have a fetch or or even you can you can mix and match await and .then.
Wes Bos
create request And response and header objects, this one's not all that useful. But if you're trying to build a fetch request The the thing with web streams is work in a silo that is going to give you all of those pieces, and you don't have to, like, guess. Oh, does it need this cookie? It doesn't need that. It will give you all the pieces and then you can kind of go from there.
Wes Bos
where you are setting being streamed in, and you could you could put that into like a React store, Svelte store or something like that, which would be reactive. You could display it to the user. In In my case, I was just console logging the actual data. Nice. I I did, this clone business. I had a middleware hook where I didn't wanna consume An error.
Scott Tolinski
I I like that. I've never done that myself, but I I like it,
Wes Bos
Last tip I have here is you can programmatically Copy as fetch is a or or base or even just try to debug the parts of some data, promise functions, you realize, oh, I no longer need this thing. Or maybe you're trying to download from 2 sources try is What they download works? Zip works, right? But you could that's how a lot of things work is as it is being streamed in, from the server saying, I'm okay. Now I've started encoding the file. Now I'm done Encoding the file. Now I've sent it off to the transcription.
Scott Tolinski
especially the stream stuff. That that stuff is the stuff that I I know the least about. But, You know, I I do love these flows in in working with the responses and errors. And, yeah, we we take Fetch for granted because we do the same thing over and over again with it. And,
Wes Bos
craft a fetch request and .catch.
Scott Tolinski
for a full archive of all of our shows,
Scott Tolinski
Well, sick. I I thought this was really informative. I thought this was really cool, Use the coupon code tasty treat, all all lowercase, all one word, and get 2 months for free of it. The best error tracking, performance tracking, and everything
Scott Tolinski
Or drop a review if you like this show.
Wes Bos
which is really nice. So if you are ever trying to do any sort of scraping or reverse engineering, And when it when you're done downloading the next chunk, You can pipe it into a zip file. So if you are maybe recording yourself in streams. So In turn. And what you can do is you can say, alright. Well, the first time I got a chunk, GitHub, meaning that, like, you know, like, fetch has the double. If you're, like, fetching some JSON,
Scott Tolinski
Head on over to syntax.fm
Scott Tolinski
and don't forget to subscribe in your podcast player
Wes Bos
your request to a fetch function. Yeah. That's sick.
Error handling with fetch
Wes Bos
a request that has happened in dev tools. You can right click copy as fetch.
Wes Bos
Another function that will but the for await loop will basically just say, like, if you were of fetch, Next one we have here is you can cancel I In case there, we'll try parse it. And if it parses, great. Then we have we have an object.
Wes Bos
from the get go. Yeah. Because, how does something work. What are the pieces that are needed to make this entire I got It's, nice to see some some different stuff. Alright. That's it for today. Thanks for tuning in. We will catch you later. Peace. Peace.
Wes Bos
On the same level, Fetching the transcript, and that will take You can you can regex it. You could run it through like a DOM a Parser library, or you could display it to the user. You could just tell the user, something went wrong. Yeah. Ideally, of how it's going fire it off. And that's a great way to do it. Yeah. Totally.
Wes Bos
at a single high level. And you can return this in an array tuple or you can return it in an object it. With an error and weird because you're expecting JSON. Like, you might you might be getting, like,
Wes Bos
A video on YouTube that shows how to write one of these yourself. I have a video in my upcoming TypeScript that shows how to type this because
Wes Bos
basically, this is a function that takes in a promise like, you're responsible for all of it. I'd rather the browser just do it all for me. So what you can do is you can just clone it. You use 1 to, like, watch for the progress. And then once all the progress is done, you just return the original untouched
Wes Bos
It's very generic. Right? You could you could pass literally any promise function into it, and and the types have to be inferred all the way through.
Cancelling streams
Wes Bos
Common example that I can think of is the autocomplete text. So you type into a search box and then you type a little bit more.
Scott Tolinski
and a promise on it as well. What's what's the use case? Or, like, you would typically be doing this for your Streaming a lot of data, and then you needed to abort and do something else. You wanted to have, like, a user initiated cancel or something. I I think the the most
Wes Bos
throws an error,
Wes Bos
What you would do is you cancel any in flight fetches and It'll return a tuple. The first thing is going to be an error, and the second thing is going to be the data. And it's either gonna return to you An error and no data or no error and the data. Right? It's gonna return 1 or the other. And this is what Rust does with their response. Right? Or what's it called? Reply? like I said, when you're downloading an m p three or a large File.
Wes Bos
or at the very least, just a nice little wrapper function And that will stop the fetch request from running, and that that can be really handy. It's a little bit weird that you now have to have 2 different things, and And that's why a lot of people will reach for a library
Wes Bos
The something with a cancel request You could be taking the output of that
Wes Bos
You might get HTML back, the same as is. I didn't wanna, like, You can collect the chunks and put them in an array and then put the array together and put them into a blob and then return the the value. But that's annoying because, like, Once you start fussing with stream APIs. Go back to the episode we had there to be able to like, MP 3 of syntax, and you wanna download the entire thing. This is something I was doing with the transcript results because which is controller dot signal. And then if you want to ever cancel To what's called a response, Which I think it will also take cookies along for for the ride, and and encode them that way. But that brings me to the next one, which is before we do that. So you're saying that you could have a request somewhere.
Wes Bos
and that will stop the or sorry controller. Abort straight away.
Wes Bos
It would also work on server if you're downloading a large MP 3 is a promise, but that promise resolves and then send the next one over.
Fetch API tricks
Wes Bos
what you what you got? Ever since I did I did a talk on, like, standard And if you fire off a fetch request for an MP 3, you almost immediately get the result back.
Scott Tolinski
in the entire world. So let's get going here on 8 tricks For using the fetch API,
Scott Tolinski
application
Download progress
Wes Bos
You can use the 4 await loop, which is, Boss, and Scott specifically in fetch in this case, This is something I was Just sitting in bed last night thinking about it, and I I woke up this morning and coded it.
Scott Tolinski
The,
Wes Bos
that a success property, whatever is, different parameters on the fetch before you actually
Scott Tolinski
You can't do that
Scott Tolinski
response, but I wanted to access and and throw the data into, like, locals,
Wes Bos
each what's called a chunk, a piece of data that has come in.
Wes Bos
So basically, you can wrap your but it also has the stream, which is handy if you the next tip is going to be how to actually handle progress. So If you are streaming in some data, you can Version of that. I have a tweet I'll link up in the show notes here of the actual code behind it. And it was really nice because you still can just use your regular checking, adding, removing, setting, Run the next iteration of this loop.
Wes Bos
The first response will give you response dot body, and the body may not be done downloading yet because it's a larger file. So what you can do and that's that's really handy to be able to do. And so I realized Error collection and whatnot. So these these are 8 different tips that I've I've come up with. So the first one is, you would get a JSON response that says something went wrong. But I was gonna say yeah. When you're working with third party APIs, you don't necessarily always have the option to to make it do what you want it to do. Right? Yeah. Next 1. Async, await, catch. I've said this a couple times, is you can mix and match Doing that with a fetch request, you could pipe that into any number of different types of streams, be it decode the text so you can view it Or you like another example of what I wanted to do is,
Wes Bos
The body is not usable.
Wes Bos
is, first of all, you can read the how big the file is you are downloading in in total. That's with a header. So you just say headers dot get content length.
Wes Bos
A progress indicator for six forty eight on standardized server JavaScript, you know, workers bun, in the browser with, like, a video API, not something I've used a whole lot, and I've I've been starting to use it a lot more as I've been Getting into streams, I needed to download the syntax m p 3 as well as 2 other pieces of audio that I needed to tag on to the end.
Wes Bos
your 1%, 2% done,
Scott Tolinski
Yeah. The body is not usable.
Scott Tolinski
or something inside of SvelteKit for actions. So I made a little middleware. That's what the 1st time I hit that issue where it's like, oh, you consumed the response already.
Wes Bos
That's awesome. Wow. Next one we have here is download progress.
Wes Bos
Then you can loop over
Wes Bos
I used one of them to actually server JavaScript,
Transcript
Announcer
the craziest,
Announcer
Open wide dev fans, get ready to stuff your face with JavaScript, CSS, node modules, barbecue tips, get workflows, breakdancing, soft skill, web development, the hastiest,
Announcer
the tastiest web development treats coming again hot. Here is Wes, Barracuda,
Scott Tolinski
syntax is presented by Century at century.io.
Scott Tolinski
Wes has been doing a lot of fun stuff with fetch and, he wanted to share some of his hot tips on the fetch. Before we get going,
Announcer
Monday. Monday. Monday.
Scott Tolinski
Welcome to Syntech. On this Monday, hasty treat. We're gonna be talking about 8 it. Awesome, fun, neat tricks to using the fetch API in JavaScript.
Async/await with fetch
Wes Bos
And then you can use an await to get the syntax that you do want. So it is totally fine to mix and match
Wes Bos
logic code because it messes the flow up and messes the scope up. All of that stuff is really annoying. Yeah. It's annoying to write.
Wes Bos
response response that JSON, Right. Like you look, I just want the JSON. I want it in one line. You can tag a dot then You need to log in 1st, and they send you to their HTML login page when you're expecting So I'll often do this where I'll run a fetch and then I'll tag a little dot then
Scott Tolinski
Yeah. I do I I like that a little bit better myself as well. And and try catch is one of those things that just straight up feels
Wes Bos
await and the chaining dot then promise syntax. Especially, in my case, I'll often use await to get the data, happy path, And I'll use dot catch to actually catch any error. I find that to be a bit nicer than having to wrap the whole thing in a try catch.
Testing JSON responses
Wes Bos
and maybe you can run that text through.
Wes Bos
Like, if you JSON parse HTML, you get, like, the open angle bracket error. It says, hey. I was not expecting open angle bracket. Right? And we'll try to await that promise, and then it'll wrap it in a try catch. And then that function will either return and that gives you sort of like a handle. And then when you have a fetch, you pass it a signal, Could be progress, could be any number of things that is being sent from the server.
Wes Bos
The actual text that's resolved, then you can say Parse it or wrap yeah. You can wrap your
Wes Bos
So what you can do JSON to come back.
Wes Bos
Then, you know, Okay, something went wrong All of those are like, yeah, it's all fetch. It's all based on fetch, Specifically,
Wes Bos
So one little trick that I've used in the past, do different things with them. You can even Zip them up with the gzip stream, A fetch request, you simply just run controller. Cancel in all the server things. But the way that works is you create a abort controller, If you want, in many cases, you want both the error and the happy path data result In the browser rather than just sit there and look at a loading indicator. And right now, what I have to do is I have to go to the dev tools and look at the console logs of the server. Right? await and that is decode that stream essentially, you say, alright. I wanna try to run this function. The next line, I wanna check if there's an error. And if there is, I'll do something. Otherwise, that would return And or you might be simply getting, like, a 500 internal error or whatever.
Wes Bos
Because you don't want the thing to choke I can calculate And if it doesn't parse,
Wes Bos
HTML. That's the classic.
Wes Bos
JSON parse in a try catch.
Piping streams
Wes Bos
So you can use different
Wes Bos
30, 60 seconds. And it that should probably be thrown into a queue, and it probably will. But as I was just sort of debugging it, You have to do dot
Scott Tolinski
And without even
Scott Tolinski
looking into that request, you could stream it directly to a ZIP That is being downloaded
Scott Tolinski
Yeah. Exactly. And and zip it as it's being streamed in. That's wild. Yeah. And that's that's how the
Wes Bos
rather than just sitting there sort of what you like. So I have and you're not sure which one will be faster. If one finishes first, then cancel the other one. Interesting. Next one we have here is testing thing if JSON is returned. So one funky thing about JSON And waiting for a loading indicator to finish.