Ep. 6: Rethinking How We Build AI Agents – with Braden Ream, Voiceflow

Welcome to Bespoken AI podcast. Press one to start the show.

All right, welcome to the Bespoken AI show. I'm the host here, John Kelvie. I'm joined today by Braden Ream, uh CEO of Voiceflow. Uh and uh yeah, it's great to have you here, Braden. Uh thanks for joining us.

Yeah, I mean, we've known each other probably like 6 years now. Uh honestly, that might be short, so uh excited to to be on the show and uh chat about everything enterprise AI and wherever else we want to go.

Yeah, so I was thinking back to when we first met, and I think it was probably 2018 or 19. Um somewhere in that time range in Chattanooga at one of uh Bradley Metrock's Project Voice events. I I guess actually uh this was wasn't my intended first question, but now that I think about it, one of the conversations I think that we had was about You and I were both deep, and I should preface it. You and I were both deep in the Alexa Skills ecosystem. Shout out to that. Uh and we were talking about Alexa conversations.

Yep.

And and I actually think it's somewhat Alexa conversations is is was such an interesting thing. And I I mean, you could almost argue that it's come to fruition now in the sense of like MCP, cuz that was sort of some of the idea behind Alexa conversations.

Yeah, I mean, I I think you I think you could argue like every generation now has thought that this is going to be the time it worked. Like, you know, there's like Siri probably thought that they were going to become Alexa.

[clears throat]

And then Alexa thought that, you know, that it was going to work with them, and you know, you honestly even with like GPTs on ChatGPT, like I don't think people have figured out the kind of like connected [clears throat] ecosystem approach yet. Uh it's kind of turned into a new version of the internet versus more of like the walled garden approach that folks trying, but yeah, it's it's it's certainly working far more than it was before.

Yeah, and and actually that So I mean, what do you think about the explosion that's happened now? I mean, when you think about 2018, 2019, did you conceive of the moment we're living in today as being a possibility?

Oh, of course. Yeah, I mean, you know, that's why we started the company in 2019 and had to wait, you know, 3 or 4 years, right? Uh No, we we um I think, you know, like many in the space who kind of got in at that time, it was pretty obvious that conversational AI would be big, specifically I think voice interfaces would, you know, would get there. Um but it wasn't clear exactly what the path looked like. Uh and there were certainly like a dark, you know, period, right? Like I think 2020 like COVID helped out uh a little bit with just sort of the acceleration of automation in the contact center space.

That was like a lifeline for I think a lot of folks while, you know, the traditional uh sort of area got a big boost in adoption uh while the tech wasn't there yet. Um and then yeah, for ChatGPT to come out when it did and to be as big as it is uh is is crazy. Uh in fact, I actually think it's created um we see a lot of consumers like when we uh work with partners who are launching like automated IVRs or chatbots, uh you can see consumers still have a lot of like scar tissue build up. Uh and immediate hang up on the agent, right?

As soon as they hear a robotic voice, cuz even though this thing is incredible, right? But they're expecting it's going to be like press one, press two. This is going to be such a pain. Or in the chatbot, they don't even want to try it cuz it, you know, it's it's a chatbot in the bottom right corner. And even though the back end is a Ferrari, it looks exactly like what they, you know, were trying in 2014. That was not a Ferrari, right? So, um yeah, it's been it's been really interesting to see and also the uh chat interface is becoming a primary interface. I didn't expect that. Um, I knew that voice interfaces would be a big consumer interface, but to see huge like billion-dollar companies be built off of a chat interface is is Yeah, it's it's unexpected.

In a good way.

Yeah. Yeah, I mean I would say for my part I mean I I mean obviously I hoped it for a moment like this. But it was never clear what what was going to happen in the uh in the future at the

going to be. Yeah.

You know, our business plan was sort of build stuff for Alexa, question mark question mark dollars. I mean we always liked the ecosystem. Uh I definitely believed in voices and interface and uh loved it as a technology, but I I didn't see it and I will say so I mean you and I think have gone through some of the same journey where we ended up pivoting into the contact center. I think we did it even a little later than you. I was clutching on to Alexa uh really had to pry my fingers away from it. Um, and then

If if you want to

and then going into the contact center, you know, I did see it as like, well I I've I've been sort of condemned to this very unglamorous, but reasonably lucrative business. Uh not anything that anybody wants to hear about at parties, especially, you know, testing contact center things. I mean people's eyes are glazing over so quickly. And so then like when 2025 and the investments started again and then the customers have really picked up. I was like, well I I did not expect to see you know, us so close to the center of things.

Yeah, I mean I I was going to say I I'll give you some interesting lore by the way on the I think [clears throat] pivot from Alexa into the contact center. Um I think given where Bespoken is in the stack, you're a little bit later, right? Uh you're going to be leaning towards you know, companies that are uh at scale, and so that they hit the problem of, you know, uh either needing QA before they launch because they know they're going to have scale once they launch, or they're already at scale and they need QA, right? And I I know I know you guys do much more than QA, but uh you know, just make it sort of easy.

Um we were at the hosting layer, and so we saw the usage of Alexa. Uh and for us, we had such a wide uh number of sort of Alexa skills built on Voiceflo. I think it was like 10,000, right? Which was you know, if you list, you know, look at the reports from, you know, back in the day, I think that was like 10% of all the third-party apps, right? Um and we saw that the average usage was going down over time. So, not like the overall usage kept going up as they were just shipping more and more devices, more skills, but the usage per skill was going down. And that for us was the sign of uh it's not catching on consumers. Um and then I remember I had an investor meeting once where they asked me what my favorite Alexa skill was, and I said, "Oh." You know, like that's what I realized. I was like, um

[laughter]

I I couldn't answer that for you, right? Uh and so that's when we started looking uh at where's, you know, really proven value. Because I think, you know, I I got to meet Michael Seibel once, who was I think at that time he was the president of Y Combinator. Uh and we were chatting about this exact problem where Voiceflo had product market fit. We were the biggest Alexa skill building platform. Uh but yeah, you know, it wasn't hand-over-fist growth. Uh and he goes, "Well, product market fit is a chain." Uh and crypto has this actually a lot where um you can provide tremendous value to your customers, but if your customers are not making money and can't pay you, like it has to go the product market fit has to go all the way down the stack, right? It can't just be your little corner of it.

And so then I, you know, that was a another unlock moment of, okay, so we're seeing the huge usage of Alexa skills go down over time per, you know, sort of like the average skill. And by the way, I'm sure Amazon has much better data and like the biggest developers like Volley, like they've done incredibly well. Uh they were not built on Voiceflow, right? Like you were we were a lot of sort of like mid-markets and there's SMBs and things. Um but um that value stack uh just didn't perforate all the way down cuz very few developers were making money on Alexa at that time. So, yeah, there's just a little bit of lower uh as to how you know, you said that decision arrived later, it's probably cuz you just got the data later.

Yeah, I mean, we definitely look at trailing signals and um yeah. But, I you know, I mean, either way we uh you know, it's it was interesting to pivot into the context. I don't know and I think it's it's been amazing to see how much it's picked up. I mean, and you know, for us the adjustment with everything that's happened has been really challenging. I mean, how have you found that? I mean, how much have you guys had to redo your architecture? I mean, how have you guys met everything going on with generative AI? Uh I mean, how has that gone for you guys?

I I for Voiceflow, it's been it's been amazing. Uh I think the it's like the promise of, you know, when you're a builder tool at the end of the day, right? Um very similar to that product market fit stack I was talking about before, you can be the best, you know, uh builder tool in the world, but if what people are able to actually build and ship for customers isn't uh you know, isn't great, um you know, as NLU and intents, like there was just such limitations in the fundamental technology, right? There was only so much you could do for the customer experience that um, you know, we built a Ferrari of a builder tool and then, you know, when you actually took it on the road you had that, you know, 20 mile per hour speed limit. Um, I think LLMs have just blown that off, right? You know, it's like the uh uh it's been completely uncapped in what in terms of what you can build now and I think folks are going from thinking about even just like press one, press two uh to you know, to NLU to now to LLMs.

Now they're suddenly thinking, hold up, this isn't even about the contact center anymore. This is actually like a core product line for us. Like this is our uh AI concierge that's able to work with customers not just on their first call but on uh subsequent calls across multiple channels. Like a lot of this stuff used to be sort of a collective uh tech fantasy that we all had of where we knew where it would go but it wasn't possible. It's now actually uh so possible that it's like a ton of work to try to get there as fast as possible, right? Uh putting all the pieces together. So, I think it's been really exciting. Um, you know, I think you know, before we started the recording we were talking about uh the pace of innovation is is like relentless uh and I think that the curve between where businesses are and what's actually possible is only widening. Um, you know, most folks are not thinking about marketing to agents, right? You know, as we were chatting about before.

Uh but you know, that's very much possible and that's sort of the, you know, the a lot of the bleeding edge now is a agent-led go-to-market, right? Uh and so I think, you know, that's more on the business side but like applied to customer experience, there's so much more possible now than than I think what folks even realize and sometimes as a as a vendor it's actually difficult to balance like building on the bleeding edge and building and marketing what's, you know, most folks are thinking about. Like hallucinations in my opinion is very much a solved

[clears throat]

solved problem done well. Uh, you know, as long as you're building things properly, like it really doesn't come up that often, but it's still one of the main things we hear from enterprise buyers uh because that's where their head at head is at. Uh, and then you have other folks who are talking about red teaming and blue teaming, you know, and like as a term I only learned, you know, 3 4 months ago. Uh, and like that gap is just widening between that, you know, bleeding edge and the um sort of where most folks are.

Uh, so you touched on a lot of stuff there that's really interesting.

I I touched on a lot of stuff. I tendency to ramble it is a Friday afternoon.

No, no, it's I mean I I think it's it's fantastic. So, um Okay. Uh, there's actually a few things that if I remember I'll follow up with you on in just what you said there. So, hallucinations as a solved thing. Um, I think that's a great observation, but then whether or not it's solved, how you know, how do you see people approaching governance here? How are people overcoming that hurdle? And I'll just I'll quickly share that, you know, what we see is I mean obviously it's a challenge and there's back and forth on it, but we also just see people are like, you know what? These things are great. And you know, the fact is they're as good as the human beings if not better, so what the heck?

Um, because I did I you know, I was skeptical people would ever put this into production. I you know, we did see people seem to kind of hold back and being very afraid to get into the voice out of things and then you know, uh, the gates have just opened. Um, but what are you seeing? I mean, how do you think how are people approaching governance? How do they get comfortable with those questions? Yeah, I I think um it depends on the customer, the use case, and the industry, right? Like it you know, the core idea of of AI and working with agents is everything's about context and the business case is the same, right?

So, um if you are in a I'll I'll give a very uh tangible example. Uh I'm friends with the CEO of a company where uh they work in a sensitive area and they put a chatbot that was built not on Voiceflow now on Voiceflow. Um and the bot had hallucinated with the customer who had had a problem and the bot then went said you know, the customer's trying to get this this issue resolved. Very sort of sensitive area and they said, "What should I do?" And the bot actually said, "Well, you should sue the company." The company like I you know, I mean like it it it actually walked them through how to sue the company and that was a $10,000 settlement for that for that one customer.

Um those are the types of hallucinations that really scare people, right? You know, there's so many examples of like I think it was Chipotle who just put out like a raw endpoint. You know, I think it was people calling it like Chipotle GPT. Like there was like a day where everyone found out that you could just basically use like their uh their cloud and sensor their customer support chatbot. Like um you have to do this stuff well and I think that, you know, it's more important than ever before for uh really strong governance and this goes this this goes so far beyond even just conversational AI. We're just talking about AI in general, right? Like this isn't even customer experience anymore.

The faster that things move and the faster that things become possible, you're seeing far more upstream dependency attacks than ever before around the security side. There are so many corrupted packages because they know that uh a lot of, you know, vibe coders are just going to pull in some malicious code uh and it's a you know, you're you're seeing that attack factor just get wider and wider. Um and that's a result of people being able to move faster and ship more and etc. And so, if you take that analogy and you squeeze it down to like the the customer experience space, you have to be uh QAing and monitoring at a at uh more rigorous level because you can now ship updates faster than you could before, right? Um and to your point, I think you have to be comfortable with a little bit of risk. At the same time, too, I think, you know, if one in a million conversations um you know, goes astray, that's probably better than the human rate, right? Like, I think sometimes, you know, people are almost more okay with human mistakes, right? Uh but for an AI to make a mistake, you know, it feels a little bit more disastrous, and I understand why that is, but uh ultimately, um you know, folks do have to have a level of risk tolerance, for sure.

Um and if your agent's not uh making any mistakes, you're probably not automating as much as you could, right? Like, there's a there's a healthy balance there. It just depends on the type of mistake.

Yeah. Yeah, I mean, I I agree with that. I you know, somewhat related to that, I would say, you and I were talking about determinism before we started recording.

Yep.

And uh I thought you had some interesting thoughts on that. I mean, what what do you think is the role of determinism in these systems as we go forward in balancing between the probabilistic Yeah. Not just probabilistic, deterministic, and I would also throw in there, you know, sort of the guided, directed paths that may still be relying on an LLM, but, you know, are not so unstructured.

Yeah, I mean, so, you know, when we talked about where voice was benefited through the transition from um probabilistic to deterministic or the transition from, you know, sort of the old ways to the new way, at first, we were, you know, really like, "Oh, boy, this is going to be a ton of work, right?" To transition. And we were debating, you know, seriously, like, dropping our, you know, full stack. And then, the more that we actually played with these LLMs, we went, "Oh, wow, they actually need a context store. They actually need state." And suddenly, we went from thinking we were at a huge disadvantage to, "Oh, we've been you know, we've run billions of of state changes through our dialogue manager.

It's exactly, you know, like companies are now being bought that just produce dialogue managers, right? And so we had the benefit of spending so long building a robust state manager. Uh and you know, a little bit of a fun fact, we'd gone so deep in state management because we were trying to solve a lot of the problems that NLU had. So our dialogue manager was like far more robust than it needed to be. But now that's actually proved quite useful on the on the voice flow side. So even to talk about voice flows architecture a little bit, a lot of folks you know, a lot of competitors that we'll we'll face off against are typically like you know, agent frameworks or mono prompts. You know, it's it's a prompt based framework though, right? Um you know, there's like prompt chain changing and all that stuff.

Um voice flow is actually a node graph under the hood. It is our the dialogue manager that we built like six seven years ago. It's the same the same fundamental engine, but now the nodes could be agentic nodes. And what that allows us to do is do like prompt splicing, really different prompt compilations cuz they're just state changes, right? Hey, like we're now running this prompt with this prompt under the hood or now it's this prompt for these instructions and it actually is the same technology that we had way back in the day just applied to a a different use case and and so that's been a huge huge benefit for us.

Now, that brings us after that little weave, that brings us to the determinism and probabilism today. Um from our data roughly 30% of uh customer support conversations are about one maybe two messages, right? Like very short conversations. Um and if you look at most greeting messages, a lot of folks are using like PhD level frontier models to send a single greeting message. You can cut like 10 20% of your cost right there by just having a hard-coded scripted message with some variants. You know, this message for you know, this type of customer and this type of customer except you know, etc. Voice that we built that because that was our original. We had to build this type of variety for the original scripting. So we have some benefits there.

You can save a ton of cost. Now, for really sensitive tasks, that's where you're seeing like for really sensitive tasks have a lot of nesting. Sometimes I look at prompts and I go, this is just a flowchart, right? And you know, it's you'll read the prompt and it's essentially pseudo code, right? If statements, it's like if customer says this then go do this and if they say that then go do that, etc. And the more like a core thesis of making really robust agents is you want to try to eliminate all these little micro decisions, right? And so that's where what you want to do is instead of having one agent with 10 nested conditions, you'd actually want to take those nestings out and actually just have pure code branching, right? We know if the customer is a VIP or not a VIP. Why do we have to have the agent think about that? That just adds another layer of potential mistake. Instead, do that branch right there and now we serve this prompt or this prompt. The other benefit of that is now you're serving half the prompt. You're serving half the tokens.

You drop cost, you increase reliability. You also increase increase speed, too, right? So there's all these little benefits to doing, you know, we call it prompt compilation, but to mixing determinism Yeah, mixing determinism not only in the responses, but also even just how prompts are compiled. Like the actual business logic of the conversation. If you look at your prompt and you have like three or four nested statements, you now have to jack up to a frontier model to make sure it doesn't make mistakes and you have a ton of stuff you don't actually need in there.

Yeah, I I what you said there. I mean, the prompt compilation um I I'm like almost wildly enthusiastic about that idea. I saw because I have been, you know, you know, our valley with being testing, it's like it has to be deterministic to some extent. Something has to be repeatable. It doesn't mean there can't be some variance, but you need uh some element of it to be a known thing. Uh and so and I and I will say there was a few years there after and I tended to keep this to myself when ChatGPT first launched cuz I didn't want to be, you know, so out of fashion and also just seen as negative about it. But I was just really delighted when I saw this paper come out from Amazon where one they basically documented I I like that way of saying it. They had a very different terminology. It was like

Yep.

you know, a symbolo- symbolic [clears throat] sym- symbological uh approach to LMs. But it it was basically talking about, you know, uh write your policy in plain English and then they would turn it into code. And

Yeah, yeah.

And then use that as guardrails on the system and then they had even also taken the time uh this research team to then show that this actually was far more effective in terms of handling false positives uh as well as false negatives. Like far more effective. And I was like, this is exactly right. I mean, let the agent have fun, but then have policy guidelines uh that can bring it along. And actually, I think what you're saying is it's not just, you know, you can have a lot of the framework as well and the sort of padding done by that sort of uh you know, uh pa- you know, prompt compilation.

And it's it's such a great architecture, I think. It makes it testable. And then the last part that I like, I I don't know why, but I I've done a few different startups now, and one of them And so, I'm always grateful when something comes along that makes something I spent, you know, 5 or 10 years of my life on relevant. I spent a lot of time on business rules, which was exactly this promise. Plain English, turn it into code. And it was an idea that just died. It was branded as AI. It was really pretty lame under It was pretty unsexy and really unsophisticated. You don't need to cover this. But, the idea that write your policy in plain English, and then uh apply it to your systems at scale, you know, I it seems like that's coming into fashion.

I'm tempted I'll I'll tell you about a uh project that we have, like a little R&D project we have at Voiceflow, cuz we've proven that it works, and it's quite cool, but we haven't productized it yet. Um

[clears throat]

So, we we've been playing a lot with embedding models. Uh everyone's focused on LLM models. Um But, at the end of the day, it's sort of like it's the same family tree of models, right? And so, embedding models have gotten so good now with uh as as LLMs have, you know, needed to get more and more robust. Um you can now run really small embedding models that are quite accurate. Um Now, why am I why am I talking about this? Uh the concept that we've been playing around with on our sort of research side is uh this idea of uh called progressive hardening. Um And what it is is you try to move the cost uh cuz what you're talking about is moving the cost of intelligence. Do you run it at runtime, or do you run it at author time, right? Um because

Right.

you know, the most efficient If you have limited scale and volume, you should run it at runtime, right? Like if you have, you know, only a couple thousand conversations. But, if you have millions of conversations, it actually makes sense to do the upfront investment and use the intelligence at author time so that you then have you know your built out deterministic tree to run really cheaply and reliably at run time. Now that's sort of how we think about this Voiceflow. Like where do you use the intelligence to maximize efficiency?

Um this concept of progressive hardening is a middle ground. It is because at author time you still have to know and this is the big challenge you still have to know what customers are going to say. That was always the big problem with conversation design the traditional approach, right? If you know exactly what customers are going to say great everyone that's a no-brainer everyone would put it author time. But because you don't know you can't predict it. And so you can't really move all the cost over there. And so this middle ground concept uh we should TM this or something but um is what you do is every single time an agent serves a response you can essentially cache that response.

And then the next time at run time you can use an embedding model cuz they're so fast and say do I have a response for this already? And so over time you're caching actions and generic responses and you basically have to decide where do you use this progressive hardening and not cuz there are going to be things where it's sensitive you know you know of course you don't want to you know have like whack responses anything so it does require some human supervision. But over time the more volume you run the more it's like Google's caching the more queries you see you're caching those results and that actually allows you to over time build a deterministic system with a LLM based run time. Uh which is is quite is quite a a neat concept, right?

[snorts]

Um and so we've we've proven it out that it that it's able to work technically and then the question for us is how do you productize it? Um and also if people are updating the agent a ton right? Like you basically have to drop the hardening back to zero every time, so it has to be like a stable state technology, but there's there's some pretty cool stuff that I think you'll start to see on that mixing of of runtime, author time, and and stuff in the middle.

Um so as a company that makes its uh its living off of testing and verifying these systems, you know, I mean that I want to stand up and applaud that idea. I I will say like when this stuff was first getting started, I was like, would be people just do that instead?

[laughter]

Well, I mean

I like I talked to a friend of mine when I was mentioning about the rules-based system, we did we'd work together and built a company around underwriting.

Yeah.

And mortgage underwriting, and I was like, I uh and hopefully without sounding like a Luddite, but I was like, I I hope people never will turn an LLM just loose as, you know, just to underwrite mortgages and assign credit scores just on the fly. Instead, let it generate the policy. It'll do a brilliant job. A much better job than humans will do. It'll work through all the quirks. It'll come up with edge cases that you never could. There's so many advantages, things it could do. But then, don't have it be the runtime. Um

It It's just quick and easy, that's why people use it for runtime, right? It It's It's easier, but you do pay the cost.

And I And so And the cost in terms of the tokens, but then also, I'm sure you you know, I mean, what about on the performance side? I mean, I'm sure customers are coming to you. I mean, they want it agentic. They want it to be fully agentic, but then they're like, oh, what's up with the latency? I mean, so I would think they'd be very excited about something like that.

Yeah, I you know, I think It's funny, there's some stigma still left, and you know, people want agentic systems, and I think that most folks, when they think agentic, they think runtime. Uh and I think that there'll be some education as to that intelligence component is a tool. It doesn't have to be the whole system. And it's like where do you apply that, right? Um so you know, even on the testing side, the biggest challenge when you're running a probabilistic system at runtime is to queue you can you basically have to battle scale of probabilities uh at runtime with scale of probability at QA time, as you know, right? Like the only way to truly battle test and like QA an agent is to have a thousand agents run against it, right?

That's you just need you just need scale, right? Uh because you're you're ultimately just playing a numbers game. Um and so yeah, it's like it it introduces all different types of uh flavors. Now, my co-founder, I'm I'm super bullish on, you know, progressive hardening and there's a reason that he's the head of product and you know, we we haven't, you know, pushed a lot of this stuff and his argument, which I I think is a very valid argument, is uh the cost of intelligence is coming down at the same time, too, right? So at what point are we actually, you know, some of this discussion is actually optimizing a local local minimum, right? Or a local maximum. Um and as models get cheaper and intelligence gets better and all that, um does it not matter, right? You just run at runtime. Uh you don't need to build all these systems and sometimes I actually, you know, I really think about that because I look at all the technology we built at Voiceflow uh around NLU and because the NLU tech was so brittle, we built such a deep runtime, right? You know, we had to have full state management and cursives, you can do like a Fibonacci sequence on Voiceflow if you wanted to, like it had to go so deep to make up for the technology and LLMs blew that away.

Um and so at what point can you just put all this stuff at runtime? You know, it's we'll have to see, right?

Yeah, I mean, I I um certainly the performance, the cost, it's easy to imagine that that kind of goes away. Um I will say from a testing perspective, I you know, it's hard for me to let go of the idea that some of it uh is not going to be deterministic cuz you know, when you get into things, you know, we work with a lot of enterprise customers as you do as well and just to have that comfort. But, you know, that could just be me holding on to a different era. I mean, a lot has changed. Um

Yeah, you know, I I will say I um I think open source models, what's so interesting about them is it it does create some stability in the actual model itself because like we've noticed uh as as we use frontier models, every time they deprecate an old model, someone had built a whole system and that prompt, it's not like code, right? It's not like when you update from like one version of node to another. Uh it like you know, it's not like your whole system breaks, right? You know, some things might mess up, but like if you go from like Sonnet 4.5 to Sonnet 5, your prompt, which is essentially code, right? Like system prompts are essentially programs that and LLMs are computers, um it could break the whole thing. And so, you don't have stability of the models and the models themselves uh degrade, right? I don't know if you've noticed this too, but like they do change the reasoning behind the scenes to optimize and compute and um you know, yeah, you get some stability with open source models, right? Like we're doing a lot of our model inference now on um uh inference providers because we just know that that's going to be a stable state model for at least at least a couple years, right? So, we don't have to worry as much about model degradation uh and model switches like we do when we work with the frontier labs. There there's so there's yeah, there's so there's so many layers here.

I I think that's a great point. I mean, that's one that I don't know that our customers are taking into account. Um But that goes So in terms of you know, what is what is the level of validation that you see from customers? I mean, what is it that gets them comfortable? I mean, when we started seeing these things coming on the scene you know, in my head I think well, you you've got millions if not billions of test scenarios. But then also having spent a lot of time in the space, I know people don't have the appetite for evaluating all that. Yeah. Um We've always tried to keep it simpler.

Try to steer people towards you know, have your golden utterances, your sort of golden paths in the system that you're verifying. Um and we found that that works pretty well. People don't seem you know, there's definitely we see this anxiety around launching the systems a lot of times. But then once people you know, one have some assurance that it's working. Two, see that it gets out there and it doesn't terribly embarrass them. They're not on the front page of the Wall Street Journal. They're good. And they relax. And they you know, it is these things are incredible. But what do you see you know, what is the sort of validation level that people are looking for?

I'm going to make I'm going to let the secret out that I'm Canadian here. I'll talk about BBM for a second as a quick analogy. Um I remember that back in college uh everyone had like it was all BBM, right? And like yeah, everyone had their BlackBerries cuz they had to be on there. And that was like the tried and proven. I remember like the first time seeing an iPhone. I was like, oh, like that's interesting and cool. It has like you know, apps and stuff, whatever. Um And once it was like a third of the folks had an iPhone, like it was a pretty slow tick up. Suddenly everyone had an iPhone.

Uh and I think So the reason I bring up that analogy is I think as consumers get used to these chat systems, these are systems. Um you're going to suddenly going to feel tremendous pressure because you were not an early adopter anymore. You're actually a laggard in what you offer to customers in your in your customer experience. And so I think the acceleration that you and I were chatting about before the show and like it's really picked up the sort of past year in terms of folks getting comfortable, it might not be them getting you know, you could get fired if you're the first to do something and it and it hallucinates horribly.

But if everyone is doing it, uh that's just like the risk that's inherently in in the technology, right? So I think there's this level of industry-wide comfortability, which is pushing it to accelerate faster on itself. As more and more people are launching these things, suddenly it's not "Ooh, that's risky." It's actually it's risky now to look like a laggard and look like you're a company that does not care about the customer experience because these systems do perform better, right? Like when you actually look at CSAT, when you look at resolution rates, like we're seeing resolution I'll I'll give I'll give an example of of a customer.

Um they are better than humans at resolving customer queries sometimes, right? But that's a pretty insane statement to make, right? That a bot could actually outperform a human. So the example that I'm thinking of is a customer sent us a transcript where they were mind-blown and we were mind-blown so reading this where a customer had come into a book it's a very very large book retailer we work with. A customer had asked for uh the Bible from a like a it was like an '80s show and they said I want this exact model from this show. Uh and the agent said, "Oh, you mean this show? Well, actually that looks more like a prop, but based off that time period da da da da da, I think this example would be good for you." And they end up buying that and like a couple other items. And I sit there and I go, "What are the odds that your contact center staff A cares enough to like go Google the show and then go look at relevant SKUs and like all in a second, right?

Uh and so that was a big lightbulb moment for me of these things can perform better than humans and it's not because they have more empathy, it's not because they're smarter, it's actually just because they can ingest so much context in real-time. Like that is the ability to to consume a million words in a couple seconds, that is like foundationally the big advantage. And so the more context you can give it, uh the more you can start to leverage that advantage to actually deliver a better customer experience. So all of that being said, I give you two different stories between the you know the BBM and iPhone and uh and the sort of the book uh book retailer. Um I think it's now people are realizing it's better customer experience and there's enough of them out there where there's pressure to adopt uh because if you don't, others are going to be offering better customer experiences. So it's starting to self-accelerate.

Yeah, I certainly agree with that. And I will also say, I mean throughout my entire career, I've heard typically consultants saying this that you know, get in early because even if it's kind of useless now uh or there's not like a lot of new use cases that you're actually winning on, you want to start learning the technology. And I think almost every other time I've heard that it's it's kind of been BS. You know, I mean it hasn't paid off to get in early. But I do think you know, the most common use case we do see today is the sort of paving the cow path, you know, let's take what we already have, let's move it to an agentic experience. It's more conversational, but at the end of the day you're kind of exposing the same functionality as you currently have.

Yep.

In general, I would say you know, as a technologist, I'm not in love with that, but in this case, I do think it's the right approach. I mean get onto that foundation because there's obviously so many opportunities to innovate from there, once you have done that.

I I will say on the migration piece, the funniest not funniest, the most challenging thing we see when we we we have a bunch of customers now who started with us pre-agentic and then now have launched agents. It's such a big mental gap in terms of how these systems work. And at the enterprise levels, you know, you've seen there's often like 200 intents, right? Uh you know, on an NLU-based system. There's 200 intents. Each of these things has a, you know, hopefully a voice you know, voice flow diagram, but you know, it's a flow chart, it's a Visio, it's whatever it might be. Um and then you look at what it it gets replaced by, if it's done well on the agentic side, it's like it's like 1,200 words. It's just it's a big prompt, you know, it's it's got its playbooks and its workflows. Like, you know, there's some sophistication there, but uh often like a big FAQ cell system, which is like maintained by a large staff of people, you know, like you know, it takes weeks to go and update a single intent. Um there is no concept of intents anymore, right? Um there's concept of tools, right? Which I would argue are kind of the successors to intents and how they're used. Um but for the most part, like you're not thinking about every possible FAQ. And actually, you get into a really bad um mindset if we've seen customers where they take all their previous FAQs and they basically put those into the prompt. And you start to overfit um specific examples and A, you pay more cuz you don't need to have the FAQs inside of the prompt. You need to think at a higher level. Like, what is the binding thread of all of these FAQs, these 200s? Like, where does the data live that has the answer? And then what are the rules in terms of how you engage, what you say, what you don't say? And that might just be a paragraph.

And that's what that have paired with the data source. And I think like that's a big mental gap for people to like move up a layer, right? Uh and not think about the specific It's even like by coding, like you shouldn't be thinking about the specific example as much as the system. That's a really hard mental shift. Uh and we've seen a lot of enterprises struggle to feel like it's not a toy and feel like it's robust without it feeling big and expansive because it it might just be a couple sentences that does what used to take, you know, a year to build.

Yeah, I mean that part is amazing. I will say letting go of intent-based thinking, I mean that is uh I think that is tough for organizations. Um

It's hard too cuz it's baked into the business process. Intents like people built their QA and their BI dashboards around intents. This intent has this CSAT and this deflection and this whatever. And like now you kind of have to do that at the agent level. Like it doesn't have the granularity of what it it used to be really nice to have a dashboard that had like all of your intents and how much money went to each of them and how was the each of them performing and we can plan our sprints and our, you know, our uh our actual PM process around shipping new intents, right? Like it was a very modular system that was really nice in terms of how it worked from a business process side as well. So that's also like that's gone too, right? There is no How how do you split that?

I mean I would think I I don't know. I mean for customers coming to you, I would think it would be advantageous that you can easily take them through that journey cuz I, you know, uh they are used to thinking in intents. So let's map it out in that way that we're accustomed to. And we And then over time we can take away all that sort of craft and we can let it be more open-ended. But to get started, you know, so that we don't feel like we're just kind of totally revolutionizing things.

the issue is there's totally

of the training wheel approach.

They're so different though. It's like uh let's say I want to go A to B, right? You know, I want to drive from LA to San Francisco. Um it's like saying, "Well, we used to take horses, and that's a good training wheels to, you know, to to learn how to drive a car." Like, yes, the same end goal, but two fundamentally different approaches. Like, building an agentic system is so different in mindset that I actually think it's really harm- it it I've seen customers who are net new to this space have so much more success so much faster than those who are taking the old approach cuz they're trying to find how do I take all of this knowledge I've built over the past 10 years and apply this to this new system when they're just fundamentally like they're incompatible. Um Yeah, like if if you're thinking about intense, you're not thinking at the system level. You're just thinking way too low down.

Yeah. Well, I love that take. I mean, that that that will be a good blurb, Greg.

[laughter]

I I hope so. Um

That was a good, uh So, uh we're almost we were talking before here. I I do like to give people a chance or if you want to make any predictions. And I guess from this conversation, it kind of naturally lends itself where I see, and sort of my prediction for what I think is going to where systems are going. And I don't think I'm saying anything that most people or at least hasn't been said by other folks. But, I do and it kind of ties together what we talked about. I mean, I think it's going to be agents plus I'm going to borrow your terminology, the the prompt compiled policy

Yep.

and databases. I think that that's to me sort of the golden triangle that if I was I don't know if that's what I would roll out today. I might do the intent-based thing if I was CTO of a big company. But, that's where I would be going towards. That's the sort of vision that I would have. Um and you can unlock so much capability, I feel like, with that.

Yeah, I mean, it in terms of predictions, it depends on what layer of the stack you want to talk about, right? Like I think the whole stack is going to be reinvented. Um you know, you can do like HTML streaming and stuff now. Uh you can have like uh completely generative front ends on the fly. Uh I think you'll see that uh be used with uh agents. Uh like that's on the interface layer. Uh I think you'll start to see the edge be used a lot more. Uh like you know, we we've we've just done so much R&D on different layers of the stack. Uh and I can tell you about some of the stuff we've done maybe offline at some point, but um if I was to give you like a big thematic prediction that's not like on any one particular layer, uh it's probably on the customer experience side, it's going to be um uh people are going to start thinking about light about uh their agents and their agent platforms more like CRMs and their agents more like life cycle uh uh products more than they are single-point products. Like I think back in the day it was you get a support ticket or a call and you solve the thing and you forget about it, right? Maybe you update some records and stuff behind the scenes. I think it's going to start to become someone signs up for your product or your service or they buy a thing, the agent goes ahead, reaches out, "Hey, I'm your agent. If you need anything, let me know." They give it a call, they remember what they bought.

Like it's it's essentially a a life cycle approach to customer experience. Um and I had a, you know, uh I had a conversation with um uh we we have a customer who's a very large car uh rental service. And I met the CEO and he goes, uh you know, "We don't want to wait for them to come to the contact center. Like that's already a poor customer experience. They had to go fiddle through the app or and, you know, find the number. They'd go Google some stuff. Like I want it to be the second that like the key drops in their hand, like we send them a text message and say, 'Hey, if you need anything at all, I'm your AI agent. I'm totally here for you. You can call, You can text me, whatever it might be.'" And he was like, A, that's a better customer experience, and B, I've now directed the customer to the lesser cost channel, which is SMS, versus a phone call, right?

Yeah, that is a great idea.

Yeah, and so like there's all these, you know, and that just comes from like a first principles approach of like instead of like the the view of we get a million calls and how do we build a bot to solve the million calls? It's we have a million customers. How do we build uh a AI-enabled experience that spans the whole life cycle where support is just one part of that, right? Uh how can these things actually be revenue creative uh for us, so that'd be my where the space is going to go over the next year on the customer experience side and on the agent side, I mean, if we had another 3 hours, I'd you know, we could dig into all the different layers.

Yeah, I love that though. I I mean, I think that that's a fantastic customer experience and, you know, also just knowing something about companies' goals, I I think that would be a huge win. I'm I'm actually As soon as you say it, it seems like an idea that everybody should do, so

It's obvious, yeah. Move people to channels that are more convenient and lesser cost.

Yeah, but that proactive part, I and I like that I mean, the other thing I'll just say there is I you know, I one of the things around having to pivot into the contact center, uh some people say, "Well, this is a cost center, not the best place to be." So, ideas like that that to me, you know, I mean, cuz I'm sure that your customer there is thinking about it. If you have that proactive approach, you can start thinking about upsells, and all of a sudden, you're driving revenue and you're not just a cost center. And that is that's great, right? I mean, that's that's great for customers and it's great for for vendors as well.

Yeah, I'll I'll leave you with the one-liner is, you know, everyone uses AI for the bottom line, but, you know, half the battle of the bottom line is the top line, right? Uh it's not just about reducing cost. Yeah.

yeah, I agree with that. Well, Braden, uh it's an absolute pleasure. Uh it's great to catch up with you and uh yeah, it's been a lot of fun.

Great. Thanks, Sean. Appreciate it.