Supper Club × 70,000 Serverless Functions with Kristi Perreault of Liberty Mutual

Mean, you want you you want a new, car insurance policy. Right? Like, there's a process to go through that. You know? There's a process to submit claims,

to get a quote, to do those things. So we have developers that are supporting, you know, you as the customer, on the mobile app, or on your desktop,

and then folks like myself doing more infrastructure DevOps kind of work, and then dev experience tools for for those folks. And it may it makes sense that these things need to be fast considering

man. There's so many things so many things I wanna I wanna ask.

as far as, yeah, Like, internal tooling and systems that we use internally,

for our employees to kinda do their jobs, for call centers to handle things,

developers that we have. So it it was an interesting contrast, that conference. Okay. Well, I think that's a good lead in. It it would be maybe awesome if you could give us a little bit about,

It's a good way. Like, that's the weird thing about conferences is that you show up, and you're like, I know nobody. And you just have to start making friends with random people that look like developers the power was out

And, like, in my line of work, Especially, like, I'm in I'm in serverless dev ops, so I'm really focused on that that bigger picture and, like, the infrastructure and supporting,

In the in the lobby?

who you are and what you do for your career may be in a little bit more than than that, and we can, take it from there. Yeah. Definitely.

roughly, like, 5,000

which was pretty cool. So it was kind of a a weird mix of, like, Folks that were really, like, front end React developers and then, like, a whole group of us that were just, like, very serverless focused.

but they had an extra day that was serverless in the park,

on,

stuff

Yeah. There you go. Who's that, Zach? What is that? Hey. Oh, hey. What's your yeah. No.

is the Syntax supper club.

Reactathon

database,

So you missed the key part where I was actually super creepy, and you were checking into the hotel. And I was like, So let's mock it out, kinda here locally.

oh, I think I know you. I was like, you're you're Speaking at this event too. Right? And I was like, you wanna walk with me?

The Instant GraphQL

and LightStep incident response,

Wash those hands, pull up a chair, and secure that feed bag, because it's time to listen to Scott Talinsky and Wes Bos attempt to use human language to converse with and pick the brains of other developers. I thought there was gonna be food,

I sure hope you're hungry.

the Stack Overflow podcast, which is the podcast of Everyone's favorite website, Stack Overflow,

Welcome to Citac's supper club. Today, we have a Christy Perrault here, And she's going to be talking about some epic serverless

So welcome to the show, Christy. My name is Scott Talinski. I'm a developer from Denver. And with me, as always, is Wes Bos.

the serverless in the park. And, basically, we were just, like, walking to the speaker dinner.

I was like, hey. So what do you do? And then she's like, yo, I do 70,000 serverless functions. I was like, what? Well, we we should also say that, like, Hey, everybody. Excited.

serverless functions here. I'm gonna Click bait, the title of this podcast. Yeah. Yes. We're gonna be talking big time serverless with Christie. This episode is sponsored by 3 amazing companies, Hasura, which is

This

API for your Postgres

So I was being super sketchy about it. So, you know, how to get on a podcast 101, just hang around the front desk For a bit.

where you were doing a talk

So, immediately, I was like, can you please come on? Because Scott and I have been talking about serverless for quite a while on the podcast, but, we have not talked to anybody that does it at a larger scale in a big corporation. So, Christy, thank you so much for coming on. Appreciate it. Yeah. I'm, super excited to be here.

so buckle up and grab that old handle because this ride is going to get wild.

West just added 70,000

Oh, I'm starving.

next level And in terms of how many functions and the types of coordination that's going on here. So,

where it is seriously,

intelligent on call scheduling and escalation.

And then there's some other,

Direct stuff for probably

But

You can run a CDK synth command locally, and it shows you what the template looks like. You can go through and check it and be like, yep. This is exactly what I wanted to create. Like, this is how I wanted it configured.

And then the other thing too is, you know, we definitely use multiple environments still. So

Yes. Because, you can't just Immediately, oh, the power's out. Let's go rack up a couple more servers so we can get this thing going. Right? Right. I mean,

command that you can run-in your CLI and and run your functions locally. And you just define your events Send JSON files, and say, like, hey. I'm passing it this event. This is what I would expect it to get in the cloud.

products as a service that sit on top of Amazon Lambda. Begin.com,

You can have a test environment, a non prod environment.

I don't know. Big big serverless you're not big serverless users. We don't have 70,000, but, like, we've we've tried a lot of them. And most of what Scott and I use our,

So it does help out a lot to push up to a development environment,

as part of re:Invent this past year, like, the same week.

Vercel,

Work like for these types of things. I know that's, like, kind of a question people get into with serverless functions in general. Like, what's this like, when I'm working locally with functions,

but you don't really care that much about the functions that run it. You know, if those go down, it's not a big deal.

So if you're using something like CDK and you're Finding all that, infrastructure in Python or TypeScript.

So we've had a lot of Support from the vendor as well, which has helped us out a lot. And I have found at an enterprise level, in house.

when you're just hosting a little website, you have to pay $19 a month. And people are like, well, that's not Serverless, you're using a platform as a service, and there's added value there.

spends another

It seems like it's come a long way, and and now you're now you're making me wanna try it again.

6 or 7 years since when I when I first tried it out because of how I had a poor experience with it. And and you're You're telling me about it here, and I'm looking at the CDK docs.

Is it is, like, the the local workflow Most of the times, people are contacting insurance companies. It's when something bad has happened or they're maybe not feeling great about

all of these. They sit on top. And I I had a joke the other day I tweeted out that says, like, Serverless is so cheap, he says as he cries and, I met Christy,

cool kinda tools and The things that along with, like, CDK, like, there is, like, a Python Lambda local,

and just like a normal normal flow that you guys have going on, is it is it So where does all this code live? Does it live in, like, a big mono repo? Are there Did you say? Typon. Yeah. And he said,

CDK.

And, you know, we're we're mostly an AWS shop. We still also support, you know, Azure and and GCP as well if If folks need it, we're definitely not as mature in those spaces, and the majority of our users are going to AWS.

So it it's kinda like trade off decisions in that space. But, as I said, like, 5,000 developers, I cannot possibly keep track of everyone is doing all of the time, and there is a little bit of everything going on. Yeah. I mean, what is the, like, like, local local workflow folks like me working on our our cloud infrastructure and development.

products

all button buttoned up to,

there's sometimes I'm just like, this like, I don't wanna pay Like, of course, you can you can optimize for regular use. But when people need you the most,

AWS, all of these things roll up to a cloud formation template. So it's just 1 massive, like, YAML template that shows you everything you're provisioning and how it's configured.

from all AWS if you are thinking like, oh, I don't wanna learn another thing is If you've ever used a serverless function or even a regular, like, express function, you'll feel very at home in all of the Lambda,

$19 a month for a service. And,

and kinda test from there as well too. Neat. You you have me wanting to try

we have a company wide, like, sandbox account that's kind of a A free for all. People can experiment, POC stuff, push up whatever. It gets cleaned out, like, every 90 days.

So they have. They've definitely iterated on it. And one of the things I really like about it is it is very community driven. Right? Like, it's developers coming together and saying, hey. I needed this, so I'm gonna Submit this construct because somebody else can also use this. So it's really driven by the priority of the community and and what they're lacking and what they're seeing and what they're needing.

developers? Is that how many people are are working on

application developers, a couple refills. Some of them maintain 1 refill that has a lot of things in it.

70,000 serverless functions? Like, is it Primarily just all about claims and and managing those types of, customer interactions, or is it, like you said, internal tooling? Or Is it all of the above? It's a lot of different stuff. Yeah. I'd say it's all of the above. Right? Like, there's definitely, like, customer facing applications, like, you know, On declaring incidents, automatically notifying the right teams. And you can get unified incident response by seamlessly orchestrating alerts

kinda everything, in between. Wow. So okay. So that was gonna be my next question is about, like, what does Liberty Mutual need to do with,

to,

Stuff at Liberty Mutual? Yeah. It's it's kinda crazy.

We have some contractors, but we're global. We're we're in 30 different

economies and countries, you know, worldwide, Yeah. Literally. And the funny part is, like, I so I am not a React person. I don't know, like, much about React at all, and here I am at Reactathon, pulling reports every day for stuff, and that cost adds up too. And you're gonna run some things, kinda into the ground and have and everything that's in your dynamo table, So, yeah, we do have, reaction developers, front end developers, on prem. So, we have a I'm in a really unique spot where there's some areas of the organization that are really mature and are churning out, you know, Lambda functions, API gateways, And I knew I liked hardware as well as software, And then you also have a dev account.

and we have folks supporting everything from our our claims and quoting systems to internal applications and we embrace this concept of of the serverless first initiative. So,

So we actually joke a little bit that the the company tries to brand itself as, You know, we're a tech company that happens to sell insurance.

So Liberty Mutual, you said 5,000

You don't need to write your GraphQL server by hand anymore

So there's also a number of different developer tools you can use for that, which would be AWS CDK, AWS SAM, CloudFormation,

kind of locked into using infrastructure as code. So in order to deploy any Sources, you have to go through our pipelines. You can't just go out into the console and the fancy UI and create whatever you wanna create that way. It all has to be done through code.

There there are some folks that are doing microservices.

And I think that's an important thing to touch on here for our listeners because we hit this in the What was it? The episode about the no code,

depending on how you wanna do it, what makes sense for your architecture, what you're trying to accomplish.

It just takes a moment and you can get started. And you can get real time GraphQL

you know, we'll check your code and say, like, hey. You know, you're trying to push up this s 3 bucket or this I'm role, and you don't have this permission on it, or you haven't tagged this properly so we know who to reach out to, TerraForm, and those can all be Be deployed up into 1 application if you wanted to, or you can separate them out into separate

setting up these services, if you're just clicking buttons to configure something,

head on over there right now and give this a try. Thanks so much for Hasura for sponsoring.

So it it's big on the security front, on the tech debt front, And kind of on the the management and governance part, so that's why it's kind of a nice balance where we're gonna allow you to use whatever tool you're comfortable with, a platform is not gonna cover our use cases that we're looking at developer tools more than anything, but I highly recommend it. I've had great experiences with it. I found it's really easy to get started with. If you try something like there's Lair, I think it's literally called cdkworkshop.com.

You're gonna have a bad day when that gets accidentally wiped out because you're not gonna remember what buttons you clicked and what knobs you turned and

is one of the most, review and well architected framework. So we're making sure that your products are, kinda

Typically, you know,

is that

You would define your infrastructure based on, a stack. So, you know, like, we've done, we can add checks and certain balances and security to some of those things. So, like, we have a process where,

You can even have your own GraphQL API sources bringing into Hasura

API within minutes.

free tier.

we're trying to get away from that model.

So head on over to hasura.infoforward/

stacks. We have, like, applications that have Your Lambda functions as well as maybe like a dynamo table for your storage or an s three bucket for your storage, The biggest the the bigger thing is that we're, so making sure that we are hitting all those points when you're architecting your applications.

So there's a number of different tools.

Like, committed to your repo, you have a description of all of the infrastructure. Is that right? Is that what infrastructure as code is? Yeah. And I I think the the big thing for us as a company and a large organization too is, by by pushing developers through our pipelines,

mini repos? Is it is this the type of thing where everybody's kinda working on the same The more or you you just have a flow of

being able to plug in. So it's a bit of a mix. Like I mentioned earlier, there are some teams and things working On monolithic applications, you know, typically when you're using,

There are some folks that are just maintaining

and you'll get 3 months of Hasura cloud standard free tier. Only the first 100 people get this, so please,

the Serverless framework.

Keys you selected from a drop down, but, like, a lot of people are like, yeah. Like, serverless is supposed to be cheap, but

and whatever language you're comfortable with. But, like, you know, no matter what you're using, you still have to make sure that you have these policies, these tools, We'll see things in place. Oh. Mhmm. Never thought about that way. That's that's interesting. Okay. This episode is sponsored by Hasura, which is the instant GraphQL for your Data that's using any Postgres or Postgres family of database.

Free trial. You can use the coupon code, try Hasura,

It's time consuming, and it's difficult. And with HESARA, you no longer need to do that. You can save half of that time That you would be spent building your API and maintaining it, and you can put that into actually doing real work. So sign up for HasuraCloud's

infrastructure as code, like you said, that means that you literally have,

Maybe some things need a lot of uptime, like, you know, maybe you're really worried about the storage,

You have granular authorization

baked in so you can secure access to all of your data sources, databases, or any other external GraphQL

Because let's face it, building and maintaining your own GraphQL servers

what

Because all you have to do is point Hasura to your data source and let it auto generate your GraphQL

rules Down to the individual row and column level, you can also simultaneously share GraphQL and REST APIs

when there's tech debt, when there's a security patch, you know, if we notice you're spiking something.

So that's, like, a huge part of why we like To go through and use infrastructure as code and kind of force folks into our pipeline service.

and create Styles of APIs from one configuration to support new and existing architectures.

instantly.

we're not used to thinking like that. Especially developers that have been around for a while, we're not used to putting that responsibility on someone else, And that that is a scary kinda risk for a company and a business as large as we are. But for a developer,

what I'm learning from that is, don't move to Canada. But, So I I like to joke because I feel like every time somebody in tech themselves, they're like, oh, I'm from a nontraditional So, sorry. I kinda know what I'm talking about, I guess.

you know, it's one less thing that we have to worry about or maintain, And we can kinda put that trust in the cloud. It's there when you need it too. Like, I'm thinking a couple weeks ago, we had a major almost a tornado roll through here, and,

And that's almost a a downside in terms of,

Yeah. Right. Right. But, I mean, you it's true. Like, you have these massive cloud companies. Right? And they have Multiple locations with different data centers. So, like, if if a tornado does hit a data center, you know, there's backups, there's Edge locations. There's other places to kinda default to and to fall back on. So there's some of that fault tolerance and reliability in place But, again, you don't have to think about or worry about. Yeah. That's pretty amazing. So okay. So we have a lot a lot of functions being are they all Mostly the same language that developers are writing here, or is it a number of different languages? Are different teams able to kind of work in their own, but in terms of an enterprise as large as we are and as many things as we're covering,

know, you're gonna have issues with your physical servers and stuff, and you don't have to worry about that going down all the time. Right? Like, you can put that burden on someone else a little bit there.

people from Idaho.

for people single sign on or, You know, folks,

And some I'm sure there are some,

You know, like, I'm I'm sure that exists somewhere within those 70,000. Right? So so the it takes, what, like, 30¢ or or 10¢ So run that once a year versus maintaining a server to run that app year round, is that the kind of thing that you're Exactly. Looking at here? Yeah. I mean, that's definitely part of it. And then some of it's the reverse side of Things right. Like, we have, you know, a catalog or a library being hit constantly, like, Scott and I,

I never heard of that. Yeah. That sounds bad.

Python? the the own way they want or or or, like, what are we talking about for and then add complex authorization

well,

Like people wanna pick this up for the 1st time, CDK workshop? Yeah. Yeah. Start with the docs. Start with CDK workshop.

you will have to do it in TypeScript. So it's it's the Okay. The one that people are rallying around. And Python is a close check-in. That one's that one's near and dear to my heart. That's my favorite. So So a lot of folks don't realize yeah. Yeah. A lot of folks don't realize that all of our Our code, everything is done,

And then constructs are just kinda those building blocks that'll help you build those things up, if you do wanna start a little bit more bare bones or aren't seeing something in the patterns that make sense. Oh, that's awesome. I'm just looking And then I did, like, a programmed for it, and I was like, I did not realize how much art was involved here, and I do not have this kind of vision.

CDK constructs And CDK patterns are also 2,

contributed from by the community. So, patterns will get you started with, you know you kinda know a basic structure of what you're looking for, what you wanna do. You can browse there and start with a pattern right out of the box.

synchronize it. Yeah. TypeScript is probably the most popular CDK language. And, actually, if you do wanna contribute to that construct hub, sustainability,

great spots to kinda get started as well, and those are, popular ones, and Python's a really close second.

Walks you through a hello world So kinda in DevOps, and to focus on the developer experience, you know, I like to say that my job's a lot easier when, I can just support what the developer wants to do. So we allow them to kinda gravitate towards what the skill set is on their team, what makes sense for their group.

super easily. I was just gonna ask you. Yeah. Is that your recommended,

buckle up because it's going to be, like, 8 more things. And I I love it when TypeScript is deal with the output and then re like, they have TypeScript on the CDK workshop right out of the box, which I love. Like, it's always With a lot of these things, it's just like, here's here's a JavaScript example. And then if you want to use TypeScript,

You just combine the 2 of them. Type on.

Some of those other, ones that you mentioned are are great products and might be good kinda more One offer individually, We see the most, TypeScript, kinda stacks or groups, We do support and allow for all of those. I would say that our most popular that has bubbled The top over the last year or 2 has been AWS CDK, because you can define that infrastructure in TypeScript or Python rather Then like a YAML or a JSON. Okay.

Typon.

at least CDK has been a really great tool to kinda utilize there.

I was like, what is Typon? last week. So very exciting stuff.

and actually you guys are getting a little bit of an exclusive here. I was Just recently announced as an AWS serverless hero as well,

to now we're focused on, kind of the well architected

Yeah.

creating kinda starter patterns to get teams going for to consulting with teams 1 on 1,

kinda get to the cloud as as easily, as cost effectively as possible, But we did have and still do have some, data centers up and running that that we maintain. We do have some applications that are still, I I don't know what I'm trying to get at here, but, Like, what what does the local development actually look like in this type of workflow? Yeah. So that's another reason why CDK is a pretty popular one because there are some really good tools there to to Help, with local testing, local development. So, like, Yeah. It definitely has. And I mean, even, we they actually released CDK v 2,

So I am a principal software engineer at Liberty Mutual Insurance,

Focusing on those pillars of cost, of performance optimization, of reliability,

you know, kinda getting there. We're gonna try serverless right off the bat, see if that works with your system, with your architecture. Have those conversations,

and then we kinda Help you get to that point. So I do everything from

But, I'm focused specifically on, so I work on a 4 person team. We do serverless DevOps. So we help support our

5,000 or so engineers,

hopefully. And in most use cases, it does,

architect like, a building architect.

You can get the full context on your service health active alerts and who's on call. LightStep incident response immediately pinpoints issues And groups alerts with machine learning, reducing your time to respond. And this means less noise and more personalization for precise notifications where you can take control API.

background, and I I feel like I'm nontraditional now for going to school for engineering because, like, everyone else? I was like, this is cool. Like, why That's why I'm asking. Before code, and I'm just like, no. Yeah. No.

An incident triage with on call scheduling across Slack, Teams, Zoom, desktop, and mobile. So what you'll want to do is head on over to light step.comforward/

Actually built an autonomous vehicle with a team and competed with it What? Across the country. Yeah. It was pretty cool.

I'm I tell people, like, it's not a big deal to

syntax, and you'll get a free 30 day trial. Listeners of syntax will also

LightStep instant response is built On the ServiceNow platform, which is used by over 6,000 I think that can be a bit of a barrier to entry for some people with serverless.

and you can move code from 1 to another, which is always why

If you've used Netlify functions or Vercel functions or any of these, they're all bay they're all built on top of the same signature,

Can can I ask what you went to school for? I I was just reading your,

hero thing here. I'm very impressed. So can you tell us about that? Yeah.

Receive a free LightStep incident response T shirt after firing an alert or incident.

So when I I went to college, undergrad for for Computer engineering. I actually thought I wanted to be a,

So I just gravitated towards, like, the math and science.

So check it out. Are you in need of incident response? Check out lightstep.comforward/syntax.

lock into

AWS

you pay for the services you actually use, not the number of the people on your team, which is important because you can scale your incident response team without adding to your bill. You get intelligent on call scheduling and escalations.

companies,

Netlify functions or Gatsby functions or Next. Js functions because you it's just JavaScript at the end of the day, and very rarely do they have Specific stuff that you wouldn't let you move somewhere else. This episode is also sponsored today by LightStep Incident Response, The all in one incident response platform for DevOps and SREs.

code as well. Because for days at at all over Ontario, and and, like, everybody was filing insurance claims. And, like, the 2 websites that you need the most, the power website and the insurance websites, a lot of them went down, and they were just out at the time that you need them. It's like, well,

so I went for this computer engineering degree. And in my undergrad, I actually focused quite a bit on, like, circuits and,

And then there are areas of our company that are still Really on prem, still working on monolith applications.

working with AWS services.

Man, there's so many things so many things I wanna I wanna ask here. So before you had serverless functions, Netlify,

So we have a lot of legacy code. We we've acquired a lot of companies and inherited tech debt, and some of those things too. So, You know, we have folks that are really experienced in in cloud and serverless and are super excited about the latest and greatest and then folks that are still kinda Trying to get on board with this this cloud idea. Wow. So what's if for those people, like, why why are you using serverless functions? There's probably people listening Right now, I say, okay. I'm building my application as a regular one. Why would I want to use serverless? Why are you at Liberty Mutual using serverless? Yeah. So some of the biggest Things for serverless that we really like. 1st of all, pay cost. Obviously, we're an insurance business. That's very important to us. So you only pay for for what you need and what you're using. Funny enough is also the Liberty Mutual tagline, only pay for what you need to be. That. That's that's it's so Oh, familiar.

I'm assuming you had

times before that, I was kinda lucky. Oh, yeah. Got to skip a lot of that and jump right into the cloud and and to roll this work.

people and lots of on-site hardware.

Mainframes are huge servers and and, like, lots of on-site like, you think about the the scale that you're at.

Is that how it worked? Yep. So we did have several data centers, I think in the time that I've been here. So I've been with the company for about, 5 years now.

I interned with them, and it was my 1st job out of college too. So, I I will admit I don't I I don't know a ton about the the pre, programming languages across these functions here? Yep. So that's a great question.

We tend to acquire a lot of companies than we have over the past you know, We've we've been around for, what, a 110 years now.

Doesn't mean that there's no servers completely.

Whereas with serverless, someone's maintaining that for you.

So I think that that's an important

is managing that for you.

That's great.

It's just that someone else and usually one of these cloud providers

And it also helps with, you know, just just time and resources. You know, that's less that we need to worry about. That's less that we need to think about. That's less that we need to manage, and we can focus Way more on optimizing business code.

It's amazing. Yeah. So we're doing it, like, even bare bones. Right? So With with serverless, you only pay for what you're using. Right? Like, if you're have an on prem solution, like, you're still paying to manage, to house all those servers, to have them, to maintain them.

So that helps a lot with cost. That helps a lot with, performance as well. So it will scale up and down as you need things.

That's my favorite lead up right there.

differentiation, like, you know, serverless

Makes that that makes that makes total sense, especially, Whatever for this every this thing, like, service is supposed to be cheap. And I have stayed away code that only gets run once a year or something like that. Right? Like like, would it would it make sense to To run a server for 365 days a year so that 1 guy in accounting can get a report on