Ep. 7: The Evolving World of AI Agents: Autonomy, Reliability, and More – with Alan Nichol

Welcome to Bespoken AI podcast. Press one to start the show.

Welcome to the Bespoken AI show. Uh this is our seventh episode. Really pleased to be joined today by Alan Nichol, who's the CTO and co-founder of Rasa. Alan, thanks for joining us.

Yeah, thanks for thanks for having me on, John.

Also, as folks who have both spent a lot of time in the space, I guess the first thing that I really want to throw out as a topic for conversation is just you know, what do you think of what has happened with voice AI and you know, chatbot-based AI? And did you at all anticipate this coming in in the sort of way that it has?

Yeah, did I anticipate it coming? That'd be a silly thing to claim. You know, I think circa you know, 2018, 2019, I was moving in a lot of NLP circles and it was kind of in vogue at the time, the kind of in vogue snarky comment was, you know, Google and others and OpenAI and stuff, they kept training all these bigger models and it was kind of very cool to say that that was a very uninteresting thing to do. Right? And you're just like, oh well, you train a bigger model, you spend more money and you get a marginally improved machine translation model or whatever else, right? And I had some friends who took the opposite view and they're like, well, that is interesting, right? That's an empirically interesting question. If you keep training bigger models, do they keep getting better? And does it ever flatten out? And does anything else happen, right? And then all of a sudden this step change came about of this sort of new qualitatively new set of capabilities that emerged and caught everybody's imagination. And then this was just snowball effect, right? Of just building more momentum because the excitement was there and more investment and more ideas and more scaling. And I wouldn't rule out that there's another one of those on the horizon, right? And I mean, like the progress is so quick, it kind of feels like there's one every month, but I really mean like a disruptive, like big big step up.

And on [snorts] the voice topic, there one that's sort of front of mind for me is and and it's not just me, I mean, everyone's thinking about this is when do we genuinely move away from a text-to-speech, reasoning, speech-to-text I've said that the wrong way around, but you know, like three components versus just having a model that's working speech-to-speech in that modality. And when do we come up with a an architecture that gives you the right level of control over something like that. And and you know, what are the benefits and downsides to it, right? And of course, you're I think there's a lot of analogies there where when people were first first playing with ChatGPT and and GPT-3, 3.5, those kind of models, it's hard to imagine being able to steer these models enough to really believe that you could build on them and that weren't that we hadn't figured out the right primitives yet.

Jev I think is a great example of that, right? Everyone's excited about Jev and for very good reasons. Because it's a much better building block than what people have been doing so far, which is use a model that can generate arbitrary text, hope that it generates some JSON and the schema that you care about and then parse that on the other end, right? But just restrict the model to producing valid outputs. That's an obviously good building block, right? And it is What is it? It's Q4 2026, right? And ChatGPT was, you know, years ago. And only now did we kind of make this in hindsight extremely obvious step for many applications where we're just using these as a as a piece of software.

And so, we kind of discover what are the right LEGO bricks or what are the right sort of building blocks for these applications over time and like sometimes there's quite a big delay. And I think similarly, when we look ahead and we think right now if I if I'm toying around at the weekend and I'm playing with a speech-to-speech model, there's all sorts of things I'm like, yeah, but you know, how are you going to do this and how are you going to do that? And maybe that the answer is to just sort of plunge headfirst in and figure it out along the way. Um and and we're going to have a similar kind of situation where we have a whole bunch of duct tape solutions for a long time, right? And I think about all the crazy variants of rag that were invented that everybody forgot about, you know, there was a gentic rag and I don't know, co-agentic rag, I don't know, I can't even remember what they were all called, but

a long time ago now, right?

Yeah, right, but these were all sort of little, you know, offshoots from the main path, but they were just we were just exploring what are the right ways to do these things and they some of those feel silly in hindsight. And then like reasoning models coming in and saying like, "Oh, actually we're going to generate we we had the chain of thought thing come in, right?" And chain of thought was interesting then it was like, "Well, what if we just what if we just do that like intentionally, right? Do that at training time and do that at inference time." Uh and then maybe we'll find some quality things. So, there's an awful lot of exploration in the last few years and definitely the dust hasn't settled yet on even on the pure text modality, what are the right building blocks?

And I think it's it's going to be very much the same on on the speech side, right? And in some of the speech providers now do a very nice thing, which is that they they do the speech recognition and along with that as they're streaming that to you, they're also telling you about their confidence that the person has done talking.

Right.

Right, and that's for a nice natural experience, a really dumb approach where you say, "I'm going to wait for half a second of silence and then I'm going to start like generating an answer." It's a pretty it's pretty janky, right? And you have these sort of, you know, you could have an intermediate model that sits there and there are a few very variants and versions of that, but then that's more plumbing for everybody to do and and you know, sort of language dependent and and all the other pieces. And there's also some context in there, right? If you're just asking the user a question, like you're asking them for their phone number and they they they give you three digits and they pause to think, then you know, they're probably not done, right? So, there's some like semantic information in there and like what they've been saying and whether they're going to continue they're likely to continue or not. So, baking that into the speech recognition model is clearly the right answer, but it's not as if every, you know, that's become like a standard part of a of an ASR or a speech recognition engine yet.

And so, the the dust still needs to settle on all of these pieces.

Yeah, I I would strongly agree with that. So, you made a lot of great points there, but the first thing you touched on there, the speech-to-speech models. I am really interested in that.

Yeah.

Um I think they're they're obviously amazing. At the same time, how do you actually deploy those safely? I mean, we still have yet to see any customers, I think even seriously considering putting those into production. At the same time, I mean, you can open up ChatGPT on your phone and and the experience you know, I mean, it's it's science fiction or, you know, would have been science fiction a few years ago.

Yeah.

Um what do you think, you know, when do you think we might see speech-to-speech models? I mean, how do you get guardrails into the speech-to-speech models?

Yeah, I mean, that that's an obvious obviously difficult question because when you've got audio input, it's this like continuous signal of anything, right? And it is very easily potentially poisonable with some like malicious input. And it's much harder to to protect against than pure text, which you can just kind of scan and understand exactly what's in there and remove the naughty bits, right? If there's something in there that looks that looks untoward, that's that's much more difficult with just an audio input coming into something. And if you are in one model's reasoning space, right? In like parameter space, whatever you want to call it, latent space, you have both the understanding and the reasoning happening and then this the the generation of language, right? Like I mean, there's of course the, you know, even even totally benign but just drifting the voice like drifting into a different register or a different accent or a different pacing or all these things.

And some of those are good, right? I mean, that's a truly natural conversation. People do, you know, mimic each other. They do start to echo each other's vocabulary. They do even start to match accents. And so, some of these things are natural, but then some of these things are, you know, maybe undesirable or or not things that you can control or want to control. So, there's all sorts of things there, but I think it to your point about having it in the ChatGPT app, I I think we sometimes lose track of a very simple and basic thing, which is when we use the word agent in the sense of it being an agent, like a representative, is a very different proposition if I am working with chat GPT or Claude on my machine and it is my agent doing things for me.

Yeah.

versus I'm an organization and I have an agent representing this organization to an end customer or a citizen or an employee or whatever it might be. And the the burden and the onus of of sort of consistency and not doing strange things and accountability are just very different in those use cases. So, I think for sure we will see it um and that will create a similar pressure the same way that every CEO in 2023 was playing around with chat GPT shouting at the team that built their company chatbot saying, "Why doesn't [laughter] ours work like this?" Right? I think we're going to get the similar thing.

And I I mentioned this before we started the show, but for me, what I saw in the market was when OpenAI did that Scarlett Johansson demo, that was you know, that was definitely not Scarlett Johansson, but it was definitely Scarlett Johansson's voice. That was really a catalyzing moment where I saw in the immediate aftermath of that a lot of executives all of a sudden believe. And so, voice and uh and and you know, telephony, automated AI that picks up the phone and answers and deals with deals with requests and resolves problems end-to-end, that was on everybody's road map and only very few like technically savvy teams were were really doing it.

And now it's everybody's doing it, right? Like every big company we talk to is is doing stuff there, is investing there heavily and and believes that it's possible. So, I think it can be a great catalyst when people have those experiences in their own lives to create the momentum to say, "Oh, actually no, we are going to do This is possible and we are going to do it."

Yeah, I think that's probably certainly the case. The drive we've seen for people to get these systems into production, I mean, it's on like anything I've really seen in my career. And I will say also, as someone that works in the contact center, which is we started off working with Alexa and Google Assistant. We thought that was very exciting and sort of sexy and then felt like this is not quite as cool. And the contact center is often under invested in. They're deploying things at a rate and with a degree of innovation that I haven't seen. I started my career with the internet sort of revolution and that you know, I would say board is in that. If they're getting the best you know, and most innovative new technology, I think it says a lot about people's appetite you know, for investment in this space.

I would I would almost argue it's almost as tail wagging the dog argument where the the the the drive to do something with generative AI is huge. And then you can look around and you go, all right, where's some obvious ROI for us to do this? So, so that A, I can brag to my board and leadership team that we're doing something here and B, have real dollars to show for it, right?

I very much do think it was the tail wagging the dog cuz I was listening to podcast all through 2024, all through 2025, and everyone's just stating it like it's a truism that CX has been turned over to AI. And it's like, well, I can tell you, okay, yes, chatbots and FAQs were turned over to AI. But in terms of we have a very good set of customers, financial services, healthcare companies, people who are handling the biggest contact centers out there. They had not remotely turned it over to AI. Now, I'm not I I these people are obviously looking ahead a little bit.

They're also probably in a sales role. I understand that completely. But to some extent, I do think that all those sort of prognostications where people are like or you know, their statements that this is truth, it did bring it about cuz people are like, well, okay. I mean, I just listened to the Bloomberg podcast.

Obviously.

Yeah, the CTO of Citibank is saying that their CX has been completely AI-ified.

Mhm.

Why isn't ours been? You know.

Yeah. Yeah. And Mark Zuckerberg just yesterday I was talking about their their new model, right? Muse.

Muse.

It's Spark. It's Glimmer, Spark, and Wonder. Muse, Wonder. That's the name of the big model. Trying to remember it. He He made that point explicitly. He's like, "We're in this sort of strange place where on the one hand you've got models kind of solving open problems in mathematics and having these sort of moments of brilliance and yet not enough common sense to just generally solve customer service for everyone, right? And he like you know, he's he's now a you know, claiming to be a that of a frontier lab and and he's pointing that out as an example like the the the lag between kind of moments of brilliance and consistency on more banal simple things is is is very real, right? And and it takes a long time for what is theoretically capable to become actually implemented and and delivering value.

Yeah. And it's a I think that I I'll have to see that interview or I'm interested to see what Mark Zuckerberg has to say about that, but but with you know, it gets to a question I have that I think you probably have an interesting perspective on, which is the balance of deterministic and non-deterministic. So, I mean obviously to some extent you know, people are using it for advanced math cuz there may be more appetite for innovation there than there is for a call center. But also the call centers have a lot of regulation they have to deal with. They have to deal with sensitivity towards bad publicity.

You know, and a whole host of their own internal policies. They don't want to go bankrupt by say underwriting loans that that uh they shouldn't be underwriting. So, a lot of that stuff I mean, I see you know, people are saying guardrails. You could just call it just programming. Amazon wrote a nice paper symbolic reasoning. Whatever term you prefer, but most people now seem to have accepted and I'm I'm personally excited about this that there's going to be a combination of these different technologies and people are going to find a way to melt it. And I know that you guys I followed you guys for a long time and and I've used Rasa and I think you guys have been balancing this for a long time. So what do you think about that and how have you guys approached that and also talked to customers about it?

Yeah. I think what do we think about the value that determinism brings? Right? One is just consistency. Right? [laughter] And the other is like scalability. And you know, there's so many things that we use in our everyday lives that we just rely on working consistently. You know, like a lift or an elevator, you might call it. Or the landing gear on an airplane, all these things, right? They're very banal, but they they better they we expect them to work consistently every single time and you know, any model-driven thing is inherently probabilistic and there are cases where that's totally fine and there are cases where you know, you might not be okay with improvisation and you might say, "Well, actually, you know, in this particular case, for this particular point, this is not a suggestion. You need to kind of follow this follow this precisely." And the other aspect, the other value you get from introducing some determinism is scalability.

And my VP of engineering always says, "The reason we write tests is to go faster." Maybe this this analogy is a bit, you know, old school from when we used to write our own code, but but the point was like, "The reason we write tests is to move faster because that means that when we get to the next sprint, we can focus on shipping the new features and not fixing regressions that we introduced in the last release, right?" And so how do you build on something that com- that composes together and that you can build things, right? And I think there are a lot of naive approaches out there that sound very cool, but ultimately they are doing nothing more than just making a smoothie of a bunch of different prompts and then crossing your fingers and hoping the right answer comes out.

[laughter]

Now, I will say that that as as models have improved, there's more value to to be had from actually giving the model more leniency to think about things and reason things. And really reasoning models were the piece that that unlocked this, right? And so, if you're thinking about a sort of GPT-4, GPT-4.5 class model, right? If you give a lot of decision-making power to the model, you get a a a loss of consistency for not a lot of benefit. So, the way I would have thought about it at the time was like, you're not getting any brownie points for laundering your business logic through an LLM and having it approximately follow it. If you have some business logic, you should just write it down and should just be followed, right? And And And something that we worked on quite a bit was how do you marry the ability of an LLM to have a fluid, fluent conversation around different topics with just deterministic business logic that gets followed and executed. I will say that, you know, the technology we developed for that era then didn't let you capture the full value of the kind the moments of brilliance of those that laser crop of models, right?

The like the GPT-5s, the the Sonnet 5s, the the Fables, like all these kind of models that can do qualitatively different things from like a GPT-4 class model. And so, we started kind of rethinking about rethinking through that problem and saying, "Well, how do we actually approach it from the other side?" And we say, "Well, we can start by giving the model total freedom to reason and show us its brilliance. And then just in the places that we really care about, we can just layer in bits of determinism." So, we can say, and and there are other people who have made this argument, and I think there are other approaches to it. There are a few frameworks out there and platforms out there where they make the same argument, and then the solution is that when you get to one of these high-stakes critical things, "Hey, you need to check for potential fraud, or you need to at this point you really need to call this API and check if this user has been verified before you take the next this next step, right?" Those kinds of things.

That the solution to that is injecting at that last minute last moment in time just a little bit more text into the prompt at the bottom. Definitely, definitely, definitely, please do this, and definitely never not do it right or fleeting like injected right at the end right and then you know and then you come up with a toy data set and you show that that solves a problem right but but that's not real determinism so the way we think about it is the Rasa approach is the agent has the freedom to reason about the problem and make it the right decisions right but in the places that it's really high stakes and you want things to happen a certain way you have some conditions that need to be met or something that has to happen at some point during the conversation or something that here three steps that have to be followed that in those moments could Rasa just takes the reins and just says here and here there's some some code being invoked and it's saying we're just going to do this and we're not going to ask the mobile for its opinion we're just going [laughter] to we're just going to progress through these bits but that it's not so kind of like all or nothing that it's not the sort of stark division of conversational ability and business logic but that you just focus on the places where it matters and you get started with sort of like total freedom so the way you you build skills in Rasa now is you start with just a plain old skill doc that you would use for teaching your Claude or your chat GPT locally to to do something for you and you just write it in prose and in plain text what it is you're supposed to do and then you can just in the places that it matters right like hey at this point we're going to actually I don't know check for fraud right and you can just in those places add like sprinkle on a little bit of color right a little bit of determinism right to just say like hey this to will go this has to happen or this can't happen before this has happened or these three steps need to go in exactly this order so you can just kind of layer that on top right but ultimately you know you still have something that in its simplest form is is fully just a SKILL.md file and it's compatible with that sort of agent skills standard and that it each skill just lives in a folder and is fully portable across projects so you just have a self-contained folder here's my skill for handling refunds or whatever it is or authenticating a user or whatever it might be and here it is and like any agent that you're shipping whether that's your voice agent or your in-app chatbot or like your agent assist in your contact center or like an employee kind of thing, whatever it might be.

You must drop that into any of those projects. So, you can build a kind of shared library of of skills. And I think that's really timely because we are a few years on now from chat GPT rag build some prototypes, POCs, exploration mode. And every enterprise at this point like this is never greenfield, right? Every enterprise has a bunch of initiatives, they have things in production, they have a bunch of different things and for very good reasons they kind of let all these teams kind of chart their own course. And now they have this very like disjointed thing and they're kind of looking at themselves thinking, well, it can't go on like this.

But I also know that I can't just force everybody onto one platform cuz there isn't one platform that meets everybody's needs. And it would be ridiculous to put all our eggs in one basket given how quickly the the the market's moving. But how do we get to some version, like what is the path from where we are to some version where we can do this federated development and people have the freedom to do things they want, but we managed to do like a centralized version of governance? And I think some organizations have done that by building an internal LLM gateway.

That's a good start, but then when you want to move up the stack of governance into actually the applications, then you want to start thinking about well, how are these agents people are building how do they make it in production? Right? What what is the pipeline for shipping it? Do we have a standard for how we want to do evals? Do we have sort of reusable skills and capabilities and tools and integrations that so that, you know, the next team comes along who wants to ship something that they can kind of reuse that and they're just assembling some pre-built parts.

So, I think it's sort of a formalization or an operationalization or whatever you want to call it of of people building these things because yeah, we're we're past the point of, you know, can we do it? Does it have value? Do we have the capability? And now it's like, how does this become a core function of how we operate?

Yeah. Uh so, I'm I'm glad you touched upon or actually said the word governance circle cuz that's another area that I am very interested in in talking about. The governance question, when you start to back away from just the implementations themselves to how organizations are approaching this as a whole, how are stakeholders getting involved, I have found that to be Well, I So, frankly, I should out myself as someone in 2022 that was just going, "Well, I I don't know that we need to pay that much attention to this cuz governance-wise, I know these companies. I know what the stakeholders are like. This is never going to fly. Have you seen how often these things are wrong? Just What are you getting so excited about? Yeah, personally, I love it, too, and I I use it for everything, and I wouldn't I'm not going to go to a actual doctor again for the rest of my life. But,

[laughter]

the insurance companies are never going to use it. Yeah, yeah, yeah. But, I

[laughter]

But, I mean, my Well, maybe maybe try actually, but but not it's not a smart thing. But, I was like, "This is never going to fly with the companies we work with. We don't have the sort of importance at least yet. Uh though, I think you you opened a door. I liked what you said I might use it in our marketing for a standardized approach to evals across the whole enterprise. And And that's how we get in into the door to the CEO. But, we haven't figured that out yet. But, what I've been interested by without seeing the conversations is the stakeholders have gotten over it.

They've They seem to have come to terms with the hallucinations. They haven't tried to boil the ocean. I expect a lot of people to come to us saying, "Well, we need to run 10 billion test cases to make sure this really works. Cuz how else can we let it in there? We can never let it give people a wrong bank balance or an incorrect idea about what their payment terms are or incorrect information about what, you know, their copay is on a surgery." And I And I'm not saying it's doing all those things today or that it's completely delegated to for all those things, but we've seen it delegated for a lot of stuff.

Yeah.

And what we've seen from teams we're working with is I I think a very practical perspective. Cuz even though I'd like people to run hundreds of thousands of test cases, and I think that would be exciting.

Yeah.

I know people aren't going to do it, and they're not going to know what to do with the data, etc., etc. Um So, instead they've been practical about it. Oh, let's have some golden utterances, run through 500 test cases, we'll see how it does. And we won't freak out if some of them don't work.

Yeah.

Like a completely sensible Something I've almost never seen strangely in my entire career. People being just sensible and practical around security considerations and you know, issues around errors.

I think on both the individual level and on the enterprise level, we have seen a lot of movement on the I guess like utility versus risk curve. And to your point of, you know, uh talking, you know, getting medical advice from an AI assistant, I mean, I doubt that's one you have running locally, which means you're sharing a bunch of medical information with some third party, right? And I think uh you're obviously getting a lot of value from that, and that's a trade-off that you're making. And I think we're seeing a lot of people make those calls.

I think, you know, Instinct is a good example. I you know, I love this thing. I mean, it's sort of my little personal assistant. I'm still a little sheepish about, you know, what I really give it access to, but I know plenty of people who are giving this thing, you know, epic access to everything, same thing with Muse, you know, giving it full read-write access to everything. Here's my credit card, like go do stuff for me. Mhm. And and both those organizations buying those assistants might might give you pause for various reasons why why that might not be, you know, uh totally kosher. But I think similarly on the enterprise side, right? I think the the value both real and perceived of of doing some heroics and and and putting some of these things live and innovating are so big, and I I everyone just kind of accepts that, you know, there's some risk involved and that there's some these problems haven't been solved. The cynic in me says that hallucinations didn't get solved, but people just got bored of worrying about them. So, they just moved on and just got on with their lives and and just wanted to ship things. But, I think more more truthfully, uh they just see there's so much value to be had that it seems silly to to Yeah, people just become very pragmatic about it, as you say, and not worry about what's the what's the worst case, what's the worst thing that can happen.

I had one insurance company put it to me very bluntly, and they said, "Well, yeah, okay, but currently we employ humans for this process, and humans also err. And managing risk, that's what we do as a business. So, so long as, you know, we can do the actuarial, you know, calculation, and if, you know, it's it's it the errors it creates cost us less than the errors humans create, or at least if it counteracts it by the the automation, then, you know, that's a that's a bonus for us, right? That's a net positive."

One thing for a long time, so we've been in the contact center space, and there's a a well-defined business realm around contact center QA, which has nothing to do with what we do. We do automated QA of bots. There's There's QA on human beings that's gone on for time immemorial. Very vital field, one that we've never touched upon. And all of a sudden now we see, and it's not like the vendors are doing it, we're not doing it, and I don't know what we're But, there's this this acknowledgement that, "Hey, yeah, we're looking at all these transcripts from human beings, and we have a sense of the error rate there, and uh not so good. And then when we look at the errors in the bots, and then those are pretty good. But, either way that it's it's a similar thing. We're just keeping track of what this thing is saying. We accept the fact that it's not going to get it right with with human agents, one of the things I've tried to do with AI that I find amazing is, let's try to train these people quickly, which it just brought to my mind something that's rather obvious, but a lot of people who sell technology to the contact center never worked in one. But, it's like, "Yeah, if you put yourself in the position of the people operating these things and trying to deliver great customer service, they're putting bodies into seats with a very short amount of training.

You know,

Yeah. High turnover, short amount of training. Yeah.

And they're expected the people we're talking to on the phone you're talking to somebody about your insurance policy, you're talking to somebody about your bank account. Their knowledge on these subjects it may have been close to nil 5 days before. And they're now expected to deliver expert-level advice on these subjects that frankly, you know, if you studied it for weeks and weeks, you would not be able to get. Right. So, I mean, I do think that that argument

Yeah.

I mean, to me that has been very, very persuasive for everyone.

Mhm.

And only again, not something that was obvious to me when this was coming out cuz I was really adamant. I wish I recorded the conversations uh that I have with people, but

There's my own natural tendency

people I checked in with where I was like, "Come on, you know, give me

[laughter]

And it's like, "No, everybody got over it." And I think for that fairly straightforward reason. [snorts]

Yeah. Yeah. No, absolutely. And and I think to your point earlier about why have we made more progress on mathematics? Not to be slept on that doing proofs is a verifiable task. And so, it's much easier to hill climb and make incremental progress on a model if you know every time it does something whether it was right or wrong. And I mean, this is very much something you think a lot about, but if we could get a 10x jump in how well we can judge the quality of conversations, that would be a huge unlock. Right? That would be a huge benefit to the quality of what we could build because right now, we get these evaluations and no shade on you or anyone else, like they're noisy, right? And we we have to do a bunch of manual effort to align them, right? If you have an LLM judge or something like that, you have to do a bunch of effort to align them with like what your perception of good is and and what you value and what you appreciate.

And even then, right? It's not as if you can point it down to a number with three decimal places. How good was this conversation, right? It is a bit of a like finger in the air thing. And I mean, that's a very noisy signal to try and improve from, right? So, if if you get a score out of 100 every time you run your evals, and that goes up and down by five points every time you run it just because the nature of things, then, you know, it it it's much more difficult to do incremental improvements and know that you're you're you're making it better all the time. And I'm sure you're going to tell me that I'm I'm wrong and ill-informed, but

No, no, well, I was going to say that, but No, no, with that particular thing, that is one where I've been uh I don't want to say violently cuz that doesn't make me sound good, but the whole thing of like

passionately

Pe- people are going to enjoy looking at a array of probabilities as a way to evaluate what's happening in their systems. I You know, I've worked in this space for a while. They're not going to do that. They are passionately not in favor of looking at sets of probabilities or even one probability, much less a set. Now, we as our we got pulled into that as a product, but then I have felt somewhat vindicated as that we've had customers now that have been doing evals for some time and have been looking at broader arrays of data across many different factors.

They're now saying, "Gee, do you think you could roll this up into a thumbs up, thumbs down type of deal? Cuz that's really

Yeah, that's really the same that's the thing we can act on, right? Yeah, yeah.

And if we have five charts, and even if four of them or 10 charts, and nine of them are green, that one red one, like the business immediately kind of goes, "Well, what's going on there?"

Yeah.

So, maybe we could just kind of move that somewhere else.

[laughter]

Um

Yeah, yeah.

And uh

Reminds me of my my misspent youth working in science was, you know, my supervisor told me at the time sort of there's only there's only two or three stories you can tell with a chart, right? Like, my my curve is better than your curve, right? This one goes up and then it goes down. Like, there's only a handful of points that you ever want to make with a graph. And the same thing with yours is like, what is the actual value to the end user when they look at those charts. It's like either thumbs up or everything's green all systems green, right? We're doing great or I we've made huge improvements and you should give me a promotion because of the great work that my team has done or hey, there's something really here. I need to fix it and you need to give me more budget so that I can fix [laughter] it. I think there can be three following conversations from from looking at that data.

Yeah, but I I I really agree with that. Well, I think you have a hard stop in a few minutes so we should wrap up but Alan it's been an absolute pleasure talking to you. Thank you for joining us and uh

Yeah, thanks for having me on John. This was a very fun conversation so I appreciate that we managed to connect and do it.

Yeah, but thanks for joining us. Have a great one.

All right, thank you.