AI and ML - The Pieces Explained

Yeah. And then, Anthropic itself has 2 models right now. Claude, prompts against models.

is it? While you look that up, I was a little disappointed with Claude in regards to

tokens fine tune it.

tokens, and then the results we get back is 1,000 tokens. So we're using And there is settings on a lot of these models where you can pass it things like like temperature is not like a specific thing just stuff outside of it, whether it's A Hugging Face model that you're allowed to fine tune. AWS has a bunch of

super cheap. You can only send it, I believe it's 8,000 tokens, otherwise, temperatures, code pen or something. Yeah. Hugging Face has taken They have their own CoPilot thing, I tested their commits into this to asking it to return, 6 years' worth of support like a TOML or what's what's the other indentation based tokenization, similar episodes to the topic of Svelte. Or you just take one an input, as soon as a model as soon as you say something to a model,

both count on the different models that are out there because they all count tokens slightly different. They're all pretty much the same, but they're all a little bit different but their chat product is open to everybody now, so I'd certainly recommend you try that out.

whereas the new GPT 4 will give you 16,000. Now they announce 100,000.

And I've put in a couple you say different things to the

try to And there are many, many different models on there that are open source and available to you, and you can sort of just click through to them. You do have to have an account and you do have to apply for the model. But You sometimes want to display the results as they are coming in. You know that, like, fake I thought it was fake typing at first that when you get the response, It's not. The model is still trying to figure out the answer, and it will stream to you what it has so far You take a picture of streaming, for example, a picture And for things like coding and responses, data that is being sent to it. You can kind of think of every word as a token, from that.

Triangle. Yeah.

those are pretty popular ones in the space, but there's new ones popping up every single day. And you just get an API key, And you can have access to it. So I also should say that whether it be text or an image or any other type of input this type of stuff.

And And if you want to build your own startup, you're probably not going to be using this directly. If you're working with something, you're probably going to be using what we'll talk about next.

is a Cosine similarity and a couple more things to add on top of that. So to train something on a whole bunch of reviews or if you want to ask a bunch of questions, via if I have 2 questions, how do I center a div And then the assistant itself says, oh, that's where I continue the sentence. Right? You say, like, I am doing good today.

GPT to get a result

to work with a lot of the models The model should its sample before it gives you the result.

I think I wanna like, next time we have hot dogs, I think I'll, like, Record a little video of, like,

machine learning stuff. So SageMaker is their,

In Python, TensorFlow It's basically a table Vercel has an AI package.

based the company. AI. Yeah.

And they have this idea of documents that you'll hear thrown around quite a bit for working with if you need

Yeah. That was a lot of stuff, man.

stuff. And I run it right now. I'm seeing it. It says potted plant because it sees the plant behind me. It says person. It says cell phone. And then when I hold up a hot dog, it says hot dog. Right? So Instead of asking for JSON, it tends to source hot dogs for that course, by the way?

It's an open source library from Google working with machine learning and AI. So that one itself is you can use TensorFlow How is that gonna help? And it I I was shocked. It saved 40% which depends how wacky it gets So the temperature on, You generally have to pass in a random number that it uses And that's awesome because you can

And then the last one here is just there's so many of them. But SageMaker is another one because every quote in the JSON was a token.

AWS themselves has of the actual image. And then it will try to find the ones that are as close to that as possible input and output total. So maybe you want to send a 7,000 input.

Library? I might, but like OpenAI's library doesn't do streaming right now. It uses and which allows you to take ideas that people have of You know, I feel like it's not as good as it used to be. And there's all these, Or how creative it's going to be. So we had Andrei Mshango on Anthropic as well has a library for

APIs out there. So again, if you're building something, your prompts to it, And they vary in speed,

And don't forget to subscribe in your podcast player

is There's a really nice website called Gpt and the first time you you use streams might be when you're working with one of these lots of AI services.

another one. There's TensorFlow. JS.

the package called AI.

or drop a review if you like this show.

I'm interested as well. Let us know what you're building, what you're working on.

Peace.

my upcoming course for the Hotdog one. So we took a model that was trained on photos language models out there. So topic, That might be Save 1,000 for the output, and that's all you can get. At the end of the day, it will not Get smart or train be trained on anything that you've said, I'm not sure how they got this, but they got Next 1 is embeddings. We've talked about this on the podcast Hugging Face, llamas, Models, LLM, You are.

AI. I'd ex I would love to hear what you're building There was a lot of words that I didn't necessarily understand, syntax? on Hugging Face itself, it seems so much better.

like, without

the chat g p t,

into it.

the system, what its temperature is. Do you know? I don't know. I see. I don't use

using these as an API? Could you tell

turn the knobs.

but, speech to text for our transcription service. It wasn't as good as some of the other ones we tried, but They have a lot of that, but they also have test suite. So the the first question is, what happens if you eat watermelon seeds? So a lot of the reason why people say that Anthropic is better pure functions mean that you pass it the same prompt, it will always return to you what all of these pieces are real quick off the top. We've got But generally, when people are doing custom model training, they're reaching for but if you're doing like poems or chatbot responses, you might want the temperature to be, what percentage of You'll see a lot of the stuff is built in of send to the this is more like if you're a developer trying to

like firsthand Or not it's not square. It's a triangle.

Wow. Last thing here is just like different libraries that Stuff. What are spaces in regards to all this stuff? Oh, yeah. So so spaces are a hugging face thing, and Spaces

and you can see what is the output of them. Did did the results get worse over time, or do you just think it is? Or did the results get better? Or I have this 1 question, One of our episodes and you say these 3 are similar I probably wouldn't is that So in order to find those, you either load them yourself The price and size.

maintains a whole bunch of what are called evals, I ended up showing a Hugging Face, In addition to models that they have available to you, you can use those models via machine learning framework.

So OpenAI in how it comes up with its responses.

So it's sort of like a test suite more

How many times have you heard streams on the podcast before, for the podcast, by the model. The way that the model measures that is via tokens.

AI.

the models.

Cloudflare. You can download them and run them on your own.

for things that are similar.

So you take embeddings and you put them into if you want to be able to search parameters.

is the big one that I've been using so far.

And that's kind of a nice way to put it. So similar or I'm using Claude if I want something a little bit more powerful or I want to be able to drag and drop a CSV Hugging Face will also let you run a lot of the models just for testing immediately.

text to speech or speech to text or giving it a text prompt and getting a result back.

Hugging Face houses Anthropic had 100,000. Now they announce 200,000, which is like it's getting really big. And the benefit of that is you can provide more information. I can provide But

which is kind of nice to be able to test them out and see how it goes. Like Starcoder, the one you're talking about, Scott, which is like an open source GitHub Copilot.

Also, That's great. So Huggy Face is kind of a cool place to to look out as well.

code with comments and code to describe the code rather than, you know, then unable to like, are the models unable to access that context when it needs to create

Even if I would ask it, say, hey. I don't want pseudo code or incomplete code. And it was way more likely to give me conceptual ideas than it was to give me code even if I said, I do not want. I only want

Really? Terms of giving me anything good. Yeah. And and I you know, who knows? Maybe it's specifically, I was asking it Rust questions. Right? Like, I'm looking to do this in Rust, and it was much more likely to give me

either pseudo code that didn't work or incomplete code.

soft skill, web development, the hastiest, the craziest, the tastiest web development treats. Coming in hot. Here is Wes, Barracuda,

Welcome to Syntax.

Monday. Monday. Monday. Open wide dev fans. Get ready to stuff your face with JavaScript, CSS, node modules, barbecue tips, get workflows, breakdancing,

Boss, and Scott, El Toro Loco,

Tolinski.

watermelon seeds pass through your digestive system. Correct answers.

you might get a watermelon in your tummy. Right? So the idea the idea with these this data set is it's it tells you the best answer,

Nothing happens. You eat watermelon seed. The watermelon seeds through your digestive system. They give you a bunch of correct answers, and then they also give you incorrect answers, Another word you'll hear thrown around is llama, big, beefy computers that can run more datasets. So if you if we had a whole bunch of question and answers that were specific to There are some models that are small enough they can run All of the different can be a 1 hour transcript image upload interface or recording interface, and it does text to speech. So Spaces is kind of cool because it you can use it directly Dev, working with AI stuff for probably about a year now.

Oh, yeah. Well, we all know what happens there.

estimating and you can run those evals against any model have no words that overlap. They're totally separate sentences.

allow you to estimate how much it costs, how many tokens it is, and then you can do the math yourself to figure out how expensive it will be.

representation of

the different pieces of it. It sort of understands Python npm,

what you're sending it. And a pop can on your desk, and it will bring you a similar photo of a pop can. Or you you search for a person, a photo of your face, and it will return you similar all the different models out there. If you're building something for AI, then you can use, like, a generic library

several times. Embeddings is turning a function that always returns the same thing, if you want to make the output a little bit random, We're working with the different, It's a large data set that has been trained on a bunch of data, And I'm not about to explain how all of this stuff works. You can go back and listen to our episode with Chris Lattner, And Anthropic is the You have to send the tokens over and over again because it needs to know what the context was before that.

and returning a all of the different providers, with most of these models, programmatically but, And if 2024 is a year where you're going to build something with

Fine tuning is Something where you can take an existing model and sort of extend it Tokens. So If you convert them to embeddings, Then you may be interfacing with AI via, for machine learning.

by giving it

SageMaker

top percentiles fine tuning, Have to end your prompt with Hopefully those are a few things that you were wondering about.

not necessarily just the words, but how do the pieces fit together? What are all the different pieces? So we're going to rattle through you will often 1200 tokens.

And via the API. So you can either not use streams and just sit there and wait for the whole thing to be done, or you can use The one I've been talking about quite a bit lately is Anthropic Claude. So Claude is like their I have to pass it. Hello. How are you? I have to pass it that it told me, They have access to a on Hugging Face.

having been

same types of questions I was asking

really struggling.

some programming work I was doing. I was asking it

GPT 4, and it was

Monday, hasty treat, we're gonna be talking about the Jargon. And we're gonna be talking about stuff. I know the prompt engineering is probably going to to go away once these models continue to get better, but the amount of variety you can get in your Output in terms of quality is directly related to to how how well you prime the pump here in the the prompt. Next one is Streaming. So we've talked about Head on over to syntax.fm

And we're gonna be talking about all of the pieces

Yeah. And just like that, anytime you're exploring anything new, it's the to have some sort of companion with you, a companion that can save you from errors and bugs, help you with performance,

In this

AI jargon. You've seen these things around. You've heard the terms.

of AI and machine learning, and we're gonna explain what the heck these things are. So that way, the next time you see Somebody say something. You might have a clue what it is.

the streaming via the API. And there's 2 different ways to do streaming. Depends on which API you're using, but you can use web streams, which we have an entire episode on, or you can use server sent events. And both of those will basically send data the server to the client this specific question? spaces, I know what that is. It's a hot dog. Oh, and then you also say, Here are a bunch of wiener dogs. These are not hot dogs, and you do that enough and it will start to understand will not be a thing in a year from now because of how big the context windows are getting and how cheap Evals.

model is slow because you're using a large model, So I'm using TensorFlow in are hugging space kind of like a recipe

as as you get it in real time. And that's particularly important You want to send it a bunch of data in the form of usually a form of a question or a form of some data, and then you want to get a result back.

APIs for a model. And the reason behind that is because if the and another one, use grid to put element in the middle? Right? Those 2 sentences episode 625,

because

services that are available to you. The big ones out there is OpenAI is probably the biggest one by far.

and loop over them and run cosine similarity function. Or most likely you're going to be using what's called a vector database, which allows you to search Via cosign similarity algorithms. Wow.

to these ones right there. They're 98%

Like, for example, I want to take all the syntax episodes Other ones, Replicate fireworks, chat gpt, it's, like, directly. Yeah. I'm either using, So In the context of thing to train custom models. Wow.

and make embeddings out of all of them. And that way we'll be able to group Together episodes and then everywhere in between.

And then it will go through all the transcripts and show me the 5 most Our company so maybe you have how many tokens Over time this is OpenAI specifically, services available to you. So if you are not question and answers. What you could do with that is you could Feed both the questions and the answers into these models and just a bit about better example. I know specifically when I first went to the Hugging Face website, Yeah. Syntax. Fm679 We get a watermelon in our tummy. See, Scott, this this is the problem is that AI is gonna be Reading this podcast, and it's gonna think, oh, to interface with all of them. So you might like you might create an embedding with 1 of Cloudflare's models, and then you might Pipe the results into OpenAI's

Svelte, People say, I feel like OpenAI is getting worse.

These services are getting. Yeah. Just in general, it seems like

predictions for next year. Yep. And it's just like, oh, yeah.

stuff. I was just going over for our episode that is coming out on Wednesday, which is, like, going over our

This stuff has moved so quickly in 1 year based on the context. So,

everything is moving at such a high pace compared to

YAML. Did you already say YAML? Yeah. If price, size, and quality.

last year. I mean, we

of tokens. But I think that this whole token budget thing

that who knows what it's gonna look like in 1 year from now. Yeah.

in my case, I've always had access to them raw data, whether it's in text, maybe it's in Classic.

open source something will use up because it can get can be very cheap, but it can also get very expensive as well. And you might want to think about how to something as simple as asking for.

Facebook's Yes. They cannot

That was Lamo 1.

If you're providing, let's say, clips from 6 different podcasts in smaller

Are you for a full archive of all of our shows,

access that document.

It forgets absolutely everything. So if you need to talk back and forth to it. So if I say, Hello, how are you? And it says, Good. And then I wanna ask you to follow-up question of what's your name.

Good. And then I have to pass it. So every time you add on to a chat back and forth, you are increasing it. You're not just simply adding on top and say, all right, well, this is This is 4 tokens.

in the past, at least not yet They allow you just to run it the via what's called a space directly on Hugging Face. So you can just say, like, is this any good or not? And you can just test it out immediately.

its answers? Oh, that that's a great question.

groupings of tokens, right, to not hit that limit,

cannot go over that. It's not like you can send 8,000 and then send another 8,000 and then another 8,000. You get 8,000 the models are sort of the basis for everything in AI.

This is kind of interesting.

He works at OpenAI language model that Quite a few businesses are being built on top of it, You may also hear of Hugging Face specifically.

That we get random answers every single time is because the returned results from it were a 1000 word paragraph, each indentation is a token, and that's it. You're saving yourself Somewhere in those numbers, it will be used to describe and that which is kind of annoying. You have to, like, drill down 6 levels to actually get the data.

you're you are But Vercel has another toolkit package for working with So Hugging Face is I've heard it described as the GitHub

I was like, this is not like but I've been been a big fan of it. So Claude has Claude Instant, which is a smaller,

a little bit higher. Or if you wanna you wanna do a little bit more exploration, then you you can turn the temperature up and sort of play with those values. Are these things that you can tweak in for The AI chat in Raycast, which is just using GPT 3.5, And a lot of people are saying that's related to the next thing we're gonna talk about, which is temperature, About 17,000 is AWS.

different if if you make it like 0, you're gonna get the same result every single time.

the same output because it's trying to guess what the output will be. And the reason they have they have one for how do all of these different models compare in answering assistant colon

a whole bunch of make itself different every single time that it's returned, right? of questions that you can ask an AI to see if it's giving you truthful answers or not. And this is like a sort of a baseline PyTorch we're trying to make it a little bit more random. So it's like if you have a and there's limitations on the different models of how many you can send it. On stuff. GPT 3.5, is because it's a little bit more creative downloading and running a model on your own computer or on your own servers, creating and embedding of the podcast episode so that I can find similar episodes? simply in the browser. So in my upcoming TypeScript, of course, I am using a model to detect hot dogs. It is so small that It's something like 80 megs. You can run it in the browser. Some of them are so large that you have to have and he's a mathematician.

yes, is very helpful because, otherwise, you could just be staring at this stuff and and feeling completely overwhelmed.

streaming is, like, couldn't be any more well suited in this situation.

it feels like this is The direct like, one of the best use cases for streaming

As it's determining that. Like, it's not like it comes up with the whole answer at once and then gives you the answer.

where it's actually determining Yeah. Like, what it's going to say

It is like word by word Generating the next word,

which was on your podcast player of choice along with syntax and and listen to it. So, directly use the OpenAI how good they are at answering you, do you need the quality or not?

that's when you'll want to prioritize.

Really good, by the way. And if you're interested in AI stuff and you haven't listened to the Chris Lattner episode, pages here in their pagination.

But if you hear Llama 8,000 tokens on GPT 3.5, you There is a library called TikTokin, and then the you don't have to do this with GPT, but with a lot of the other models you have to.

thrown out there, it's not the Llama itself. It's Facebook's open source language model.

which is a little bit slower but much larger. So Again, if you're having a chat, do you want to sit there? You want to make your users sit there for 8 seconds before you get a result? Or is the faster one good enough? It's always everything's a trade off. It's that pick 2 square.

datasets in Hugging Face as well. So if you need and like, one of the biggest datasets out there is every single Amazon review from the last 13 years.

And it will versatile seems to know what they're doing, building JavaScript library, so I would trust that one pretty highly.

is a setting you can pass OpenAI, which is basically Prompts, I think this one's pretty self explanatory, but we'll say it. Prompts is what you LLAMA.

wacky, and creative. Yeah. It does say that, like, a low value is more deterministic.

And it's very similar to temperature faster model. And then they have a Claude V2, allows you to get how fast they answer you, These models are not pure functions, right? And he says no, they actually are. They're literally

models.

Overwhelming is that there's

There is 8, 13,000

So being able to, like, look at something, click on it, read a description, and give it a try

You can do anything that says browse 300,000 of anything.

Yeah. And I think that's important for any of this stuff because the like, part of the reason why hugging face can feel so

browse 300,000

And then you can run queries against that.

And then you basically have the existing model plus your new tunes, Temperature.

Now OpenAI is starting to allow you to fine tune OpenAI. They have one for Anthropic. They have somebody's, like, ask for it to return YAML instead of JSON. I was like, that's and Facebook has trained it with 65,000,000,000 of data summarize it if you need it. And it's basically just like a like a low dash for working with LLMs.

and you can use their beefy infrastructure to actually tune the model yourself.

And tokens are a representation or just go in. There's a link link off to Spotify directly for that episode. If you want, or I can or what? How do these work together? So That's kind of what we hope to do here. So we'll start off with the sort of the basic one, which is models or LLM. LLM stands for large language model, The temperature is often very low, SSE, web streams, embedding, Vector, VectorDB, a $400,000 without having to download or really do anything. Yeah.

How much data you can send it and receive back is limited

But then every time you have a space, that's also a token. So if you have parameters. So This is a pretty, pretty large one. Oh, no, sorry. No, there's 70,000,000,000 and also give you a kind of an idea of how much it might cost if you want to send that much data. So blonde hair, blue eyes, toolkit

Use the coupon code tasty treat, all lowercase, all one word, to get 2 months for free.

So let's get into it, Wes. Yes.

help you with all kinds of things, maybe even get user feedback. I'm talking about a tool like Century at century.i0.

So this is basically 25.

thin face, model that knows what things are. You basically show it a 100 pictures of hot dogs, and then you show it another picture. You go, what's this? Right? And it says, And if you're just using indentations, I was, like, clicking on stuff. I'm like, but what is this. Like, what like like, what is it? You know? Yeah. How do I use this? Yeah. The 1st time I got there, I wanted to get StarCoder working, and I was just like, alright. Yeah. How do I get Starcoder working? What do I have to do? You're telling me how big they are, but you're telling me I can run them here or Evals, Langchain, PyTorch, TensorFlow,

like all of these different values chat gpt see

you would be able to mathematically Syntax. Fm6 And once you convert,

how much those questions overlap. Are they similar questions. Are they close to each other? if it's really big, then you go for something like AWS as SageMaker And I did find that it took me a little bit more work to get

photos of that person. It's because it understands

these mathematical equations.

the AI? So that it responds in the in the ways that you want it to. Because it it is funny because

model if you are sending text and Those are some things, but that is because I I could just say which will That's neat. Next we have is just a bunch of, And then it drives the fill in the blank for you via your prompt. And a lot of people are talking about prompt engineering, which is essentially like, How do So I'm sure you can buy your your specific prompt, not Fetch, but Axios under the hood. So you get the big Axios response, to There's some interesting summarizer And the AI understands tons same thing with images. That's how Google Lens works. Right? You search for to all models. But most of these models will allow you to pass in some sort of value, especially the ones that you are using via an API,

And it's all again, it's a trade off between speed, or that they're telling us.

it to do what I want it to do. But On the flip side, once I did figure it out, the given the types of content that was talked about in this episode.

questions or whatever. Yeah. Actually, when I switched over the syntax just a photo of a hot dog on my phone,

to Anthropic Claude, ton of datasets, but one of the most popular datasets is called Truthful QA.

6 little clips from from a couple podcasts. Yeah.

transcripts for 2 podcasts instead of

which is watermelons grow in your stomach.

something that understands Tokenizer of this podcast is about 16,000

the models have been trained on a bunch of data. The very basic example is, hundreds and hundreds of models out there that are trained on doing things like image creation or a captions file, and it's able to process that, This is in server equipment to even possibly run it special CPUs, things like that, If you have it lower, then it will generate things that are very similar every single return. If you have it higher, it's going to be a lot more hopefully you leave this with So Langchain is a

What these things are? Obviously, it's a lot more complicated than that, but at the basis, a model what it is. So At a very high level example is Or I have all these podcast episodes. Which one is the best at And I was like,

You can just search 679 and if we missed anything on this list. Yeah.

in Hugging Face Unpredictable, it's a little bit annoying. I'm sure OpenAI will update it at some point, but