Brian Bell (00:01:18.562): Hey, everyone. (00:01:21.663): Welcome back to the Ignite podcast. (00:01:23.664): Today, we’re delighted to have Tarush Agarwal on the program. (00:01:26.725): He’s the co-founder and CEO of Sakura, (00:01:29.006): which is building the infrastructure for self-improving voice agents, (00:01:32.527): the verification layer for the world where agents talking (00:01:35.449): to your customers are increasingly written and rewritten by AI. (00:01:39.390): Thanks for coming on, Tarush. (00:01:40.431): Thanks for having me, Brian. (00:01:41.491): Really appreciate it. (00:01:42.332): Yeah, I’d love to start with your background. (00:01:44.053): What’s your origin story?
Tarush Agarwal (00:01:44.954): Definitely. (00:01:45.494): So yeah, I grew up in India, as you can tell. (00:01:48.756): I spent most of my adult career there. (00:01:51.498): So I went to college to IIT Bombay. (00:01:54.660): I did major in computer science. (00:01:56.341): So if you know about India or IITs, it’s really one of the premier institutes in India. (00:02:01.544): Following my graduation, I very quickly moved on to work at a quant firm in London. (00:02:06.307): I spent A couple of years in London, I started working on low latency quant strategies. (00:02:11.731): So I ended up as a quant developer there, (00:02:14.172): ended up working in a team which works on really low latency quant strategies. (00:02:20.096): It was quite an exciting time. (00:02:21.316): Definitely, I think I learned a lot. (00:02:23.338): It was my first proper job out of college. (00:02:25.679): I spent time with engineers, like really great engineers, right? (00:02:29.421): Half my team was from Citadel, Optiver. (00:02:32.683): And yeah, half the team went to those places after. (00:02:35.648): So it was a really great team. (00:02:36.970): Some of the smartest people I’ve worked with. (00:02:38.752): I got presented with an opportunity two years into it to move to Chicago. (00:02:43.179): And that was really exciting. (00:02:44.380): It was a very small team of us who were starting the Chicago office. (00:02:48.566): And that came as a very big excitement for us. (00:02:52.330): Mark that. (00:02:52.911): Yeah, it was a big opportunity. (00:02:54.553): I was quite excited because it was really small team. (00:02:56.855): And that means naturally a lot more ownership, a lot more learning. (00:03:01.120): And yeah, as you can guess, I took the first opportunity to go there. (00:03:04.324): I spent a year there. (00:03:06.265): It was some of my favorite time spent, right? (00:03:09.330): Building and writing code from scratch. (00:03:11.913): What I also realized over there is, okay, like I really enjoy ownership. (00:03:16.240): I like building things from zero to one and hence I decided (00:03:20.005): to take this place I decided to quit my job and started applying, (00:03:25.008): started hacking, and we got into Y Combinator with one such idea, which is Sakura. (00:03:30.170): And yeah, the two years after that has been a journey, ups and downs.
Brian Bell (00:03:34.572): I bet being a founder, is it harder or easier than being a quant dev?
Tarush Agarwal (00:03:39.053): I think there’s pros and cons, I wouldn’t lie.
Brian Bell (00:03:41.474): The kind of stuff you’re building is super technical, right? (00:03:45.456): To build quant systems at a hedge fund is just, (00:03:47.757): yeah, it’s got to be some of the most technical stuff you can do.
Tarush Agarwal (00:03:49.498): I think we were not even thinking milliseconds, we were thinking nanoseconds. (00:03:53.340): Really, that was the margin. (00:03:55.061): We were looking at 0.1 bips of movements in the prices. (00:03:59.383): We were looking at 9 to 7 nanoseconds of latency. (00:04:03.526): Wow.
Brian Bell (00:04:04.246): So everything’s down in like C, (00:04:05.768): you’re down in like almost like the compiler layer trying (00:04:08.833): to like get this more efficient.
Tarush Agarwal (00:04:10.175): It’s definitely a lot of that. (00:04:11.236): It’s also a lot of architecture, right? (00:04:13.159): So do you use radio waves or do you use microwaves, right? (00:04:16.484): There’s also laser coming up. (00:04:18.307): So a lot of architectureSubstack architecture, a lot of like system design. (00:04:22.292): And of course, like the low latency on top of that, like, how do you design your algorithms? (00:04:26.679): So it’s, it’s really like the full thing. (00:04:28.862): Like re you have to really understand that it flows end to end, right? (00:04:32.828): You can’t just be sitting and touching And that’s what I really loved about my job as well. (00:04:37.851): I would touch like quant strategies. (00:04:39.791): I would go and look into order books, (00:04:41.552): understand how the orders are flowing in, (00:04:44.993): where we’re doing well, (00:04:45.873): we’re doing poorly, (00:04:47.013): go dig deeper into that data, (00:04:48.874): find that place where we can improve that one nanosecond, right? (00:04:53.255): And that’s really what excites me about my previous job. (00:04:56.856): And yeah, I think I still miss parts of it, but I love what I do currently, so I can’t complain.
Brian Bell (00:05:02.173): So you said you applied to YC with one idea, but now you’re working on another. (00:05:05.518): Tell us about that kind of journey.
Tarush Agarwal (00:05:06.780): Definitely. (00:05:07.241): So like everyone, right? (00:05:08.903): When I quit my job, I didn’t have a straight up, I want to do this. (00:05:12.528): I didn’t have this year I come moment. (00:05:14.151): I was like what most people do where I start. (00:05:17.095): We started hacking on projects. (00:05:18.277): And at that time, if you remember, agents were really older age. (00:05:21.441): People were building agents for healthcare, mortgage. (00:05:24.245): We picked one such vertical, which was legal, right? (00:05:27.330): And we were building voice agents for personal injury law firms. (00:05:30.935): What was really... (00:05:32.137): Exciting was we were able to get clients on that as well. (00:05:35.021): We were really automating a process and you could see the value, (00:05:37.745): but we really struggled to test our agents that we were building. (00:05:41.531): And we just ended up founding ourselves, (00:05:43.794): calling our agents, spending three hours after dinner Just talking to it. (00:05:48.580): And as an engineer, as a developer at the core of it, right? (00:05:52.063): It didn’t feel right. (00:05:53.164): What any engineer would do, we started thinking about how do we automate it? (00:05:57.167): And we came up with a very primitive version of Sakura, (00:06:00.249): which essentially was an internal tool for ourselves. (00:06:02.852): That’s really where it all began. (00:06:04.793): We were still like applying for YC and I think we were interviewing in the process. (00:06:08.556): And while all of that is happening, (00:06:10.157): we’re also showing this internal tool to our friends who are in similar spaces as us (00:06:14.581): and they all seem to love it. (00:06:16.242): We get this natural pull for our idea and almost instantly, (00:06:20.786): we feel like we are attracted to that as well. (00:06:23.689): Working with developers, (00:06:24.810): working with product managers, (00:06:26.451): head of engineers, (00:06:27.512): that was really exciting for me, my co-founders, we’re all engineering background. (00:06:32.557): And compared to law firms where we might not have enough context, right? (00:06:37.801): Hence, we just decided to make a switch full time to this. (00:06:41.724): And I think it was the first week of YC. (00:06:43.706): We just got in, I think two days into it, we were like, yeah, we want to do this. (00:06:47.189): We talked to our group partner, Nicola, and he was like, guys, do this. (00:06:50.812): Like, what are you even thinking about? (00:06:52.293): And yeah, that’s how we ended up at Sakura.
Brian Bell (00:06:54.255): Yeah, and there’s lots of famous examples of that, (00:06:56.197): of the first, you know, week or two pivot in YC. (00:06:59.239): You know, you kind of You’re a strong team. (00:07:00.921): You get into YC, obviously. (00:07:02.123): It’s hard to get into YC. (00:07:03.385): And then you get there and the partners are very supportive, right? (00:07:06.549): They’re like, hey, yeah, like this is a better idea. (00:07:09.133): You guys are uniquely suited to go after it. (00:07:11.316): And it’s something that people, (00:07:12.779): you know, (00:07:13.339): it’s a hair on fire problem for people just like it is We started with YC, (00:07:15.743): I think we were one of the fastest growing companies in our batch, (00:07:18.307): both in terms of number of customers, in terms of revenue.
Tarush Agarwal (00:07:30.624): It was really an exciting time. (00:07:32.825): I think those were the three most intense months of my life. (00:07:36.727): I learned a lot. (00:07:37.527): We built a lot. (00:07:38.347): We made a lot of progress. (00:07:39.808): And that just set the pace for us as a company, right? (00:07:43.069): It set the pace for what was to follow for the next two years. (00:07:46.291): We ended up doing our seed round of two and a half million. (00:07:50.003): I’m glad to be working with you as part of that as well.
Brian Bell (00:07:53.649): Fair disclosure. (00:07:54.410): That’s why I’m being so nice.
Tarush Agarwal (00:07:55.511): After Seed, I think it was pretty much heads down, right? (00:07:58.055): We didn’t go and be like, oh, we should start expanding our team to 20 people. (00:08:02.261): It was more rational. (00:08:04.004): We focused on Our customers, we spoke to a lot of them almost every day. (00:08:08.868): And I love to say it like my full time job is essentially being a customer success person. (00:08:14.253): I speak to a lot of our customers. (00:08:16.675): I speak to them all the time. (00:08:18.456): I get feedback from them. (00:08:20.298): I see how they’re using the product. (00:08:22.379): And we just try to iterate on it as fast as possible. (00:08:25.845): There were days like when customer would be on a call, (00:08:28.148): they would be complaining, (00:08:29.189): I would finally be messaging the team, oh, hey, this is a bug. (00:08:33.055): And before the call ends, we would have fixed a ship for that. (00:08:37.081): That’s really what this intensity brought. (00:08:40.867): We’re not just doing it for the sake of doing it. (00:08:43.510): We really enjoyed this. (00:08:45.453): And that’s how it went on, (00:08:47.156): I would say, (00:08:47.636): for the next six months, which is when we felt like, okay, now we should expand. (00:08:52.784): We ended up working with some of our best friends, friends from college. (00:08:57.531): We also ended up meeting people in our network. (00:09:01.096): And that’s really the team, Sakura, that you see today. (00:09:03.940): And all of this came alongside the customers We work with some of (00:09:09.408): the biggest enterprises, two of the 10 top CCAS players, two of the four big fours. (00:09:15.337): These are like system integrators who work deploying voice agents (00:09:18.902): for massive enterprises across the globe. (00:09:23.968): We love them. (00:09:24.848): We love working with them. (00:09:25.969): We love the feedback they share. (00:09:27.470): They move really fast. (00:09:28.630): And you really see what is at the edge of voice AI today. (00:09:32.512): And yeah, that has been our journey. (00:09:34.273): And we would want to do this for the next 10 years.
Brian Bell (00:09:37.594): Yeah. (00:09:38.115): What is the state of voice AI? (00:09:39.875): I mean, you integrate with all the big ones, right? (00:09:41.476): Vappy, Retail, Blend. (00:09:42.597): We’re investors in Blend. (00:09:43.697): LiveKit, LemLabs, all of these. (00:09:46.710): What is the state of Voice AI today?
Tarush Agarwal (00:09:48.753): Voice AI is moving really fast. (00:09:51.197): I think if you know the market, (00:09:52.719): you know how fast some of these companies have grown over the last two, (00:09:56.645): three years, right? (00:09:57.806): I remember when we were launching, Bland had already done their Series B. (00:10:01.692): And all of that in a span of one year after making the voice AI product, right? (00:10:05.698): Go from seed to series A and B and lose all the rage. (00:10:09.523): Part of that is how valuable it is to people, right? (00:10:13.910): If you go to any big company, they have a voice agent. (00:10:16.854): In the future, everyone will have a voice interface. (00:10:19.618): Everyone will have like support agents, sales agents, agents that do all sorts of their work. (00:10:25.367): And a big part of those agents would be interacting over voice. (00:10:28.371): And that’s really what we see today. (00:10:30.194): There’s definitely like a There’s a lot of small problems to solve. (00:10:34.360): Is the transcription accuracy good enough? (00:10:36.904): Are the LLMs fast but also accurate? (00:10:39.868): Is your speech quality natural enough? (00:10:43.073): There’s tons and tons of edge cases and that’s really where You can not go wrong (00:10:48.821): in picking a platform or a framework like LifeKit or PipeCat. (00:10:52.406): It’s just there’s a lot you can do on top. (00:10:55.671): And to make sure you’re doing accurately, you need to be able to measure that. (00:10:59.517): When I say measure, you’re not just measuring, oh, is your agent accurate enough? (00:11:03.142): Is it following the instructions? (00:11:04.785): But it’s also about latency. (00:11:06.567): It’s also about interruption handling. (00:11:08.250): It’s also about how your agent performs in accents, background noises. (00:11:12.656): When there’s crosstalk, (00:11:14.058): when there’s sound artifacts like cuff, Why is your agent dealing with all of this? (00:11:18.404): Because trust me, the real world, it’s crazy. (00:11:23.572): Some of the calls I’ve heard, I still laugh today. (00:11:26.315): But imagining a human will have problems understanding that. (00:11:29.660): Imagine doing that with a voice agent. (00:11:31.703): And that’s really what we want to solve for these people, right? (00:11:34.225): We want to give them a simulation environment where they can launch their agent (00:11:38.929): and get a real-world feedback of how it would perform. (00:11:41.752): And then they go live in production. (00:11:43.854): We learn from where they’re failing. (00:11:45.856): We help them improve their simulation environment and essentially close the loop.
Brian Bell (00:11:49.579): Yeah, I mean, you can’t manage what you can’t measure, right, is the old management phrase. (00:11:53.542): And you guys enable... (00:11:55.324): The measurement of voice agents, (00:11:57.788): whether they’re being successful, (00:11:59.170): whether hallucinating, (00:12:00.412): what are kind of the top three or four or five metrics as I deploy Sakura (00:12:04.778): into my voice agents? (00:12:07.262): What are some of the kind of the measurements that come out?
Tarush Agarwal (00:12:09.265): I’m glad you’re asking that. (00:12:10.386): And it does depend. (00:12:12.469): That’s one piece of advice I would add is if you’re working (00:12:15.474): in healthcare versus you’re working in finance versus you’re working as (00:12:19.079): a customer support agent, it does depend, right? (00:12:22.063): As a customer support agent, you don’t just want to measure your escalation rate. (00:12:25.248): You also want to measure your user experience, right? (00:12:28.430): As a healthcare receptionist, (00:12:29.951): you do want to measure, are you able to empathetically handle the users, right? (00:12:34.814): Are you compliant? (00:12:36.575): Are you handling all the emergency cases well? (00:12:39.918): If someone is calling and saying, (00:12:41.639): I am having a heart attack, you can’t be a patient voice agent.
Brian Bell (00:12:44.881): I’m sorry, you’re having a heart attack, you know?
Tarush Agarwal (00:12:47.272): Exactly. (00:12:47.713): You really have to handle that, right? (00:12:50.817): So that’s one piece of advice. (00:12:52.039): But broadly speaking, you, of course, want to measure latency. (00:12:55.904): Nobody wants to get a reply five seconds later from an agent. (00:12:58.808): You definitely want to measure how your agent is handling interruptions. (00:13:02.333): Because even when me and you are speaking, (00:13:04.777): right, you’ll see I would give some opinion in between your sentence. (00:13:09.344): But you’re rational enough to know when to stop, when not to stop. (00:13:13.289): You need to make sure your agents do the same. (00:13:15.533): That’s only then as a natural conversation. (00:13:17.235): There’s also transcription accuracy. (00:13:20.197): If you’re transcribing incorrectly, everything downstream fails. (00:13:23.218): There’s voice tones and clarity. (00:13:25.340): So is your agent speaking clearly? (00:13:26.960): And lastly, of course, instruction follow is a big, big one.
Brian Bell (00:13:30.142): What sounds like where you’re kind of having the hair on, (00:13:32.984): you’re finding the hair on fire problem is really in compliance, right? (00:13:36.306): In finance, medicine, where you just can’t be wrong, right? (00:13:39.627): You have to be 100% accurate, right? (00:13:42.712): And you can’t get to, (00:13:43.673): nobody can get the six or seven nines of accuracy when you’re talking (00:13:47.238): to people via voice, but what does like good look like? (00:13:50.283): And is that really the case where that’s where you’re kind of seeing the hair (00:13:52.806): on fire problem there?
Tarush Agarwal (00:13:53.547): Definitely, (00:13:54.649): as I mentioned, right, (00:13:55.410): my example, (00:13:57.532): So what we see (00:14:03.016): in real world, and we work with a lot of these people, right? (00:14:07.559): We work with 200 plus customers. (00:14:09.321): We do like millions of simulations. (00:14:11.482): What we really see is you can get these agents to be accurate without compromising (00:14:16.746): on the quality in terms of latency and everything. (00:14:18.848): We’ve worked with people who have deployed agents to the point of two digits of nine. (00:14:23.071): Of course, six digits of nine You need more data. (00:14:25.754): You need more evolution of the voice AI space. (00:14:28.858): And if you see, (00:14:29.880): honestly, (00:14:30.701): a lot of model improvements in the last two years have been around reasoning (00:14:35.187): and you cannot reason. (00:14:36.249): You cannot reason for 10 seconds in a live voice call. (00:14:39.113): You’d be surprised I mean, (00:14:39.894): I guess you could tell somebody if it’s really important, (00:14:42.997): like if you’re doing proper model routing, (00:14:44.198): maybe you’re kind of saying, (00:14:45.559): okay, this, this is a question I can’t instantaneously answer.
Brian Bell (00:15:05.236): Can you hold on for 10 to 20 seconds while I research that kind of thing? (00:15:09.902): Do you see people implementing that sort of workflow?
Tarush Agarwal (00:15:12.626): Definitely. (00:15:13.027): I think people who are on the edge of this are doing that wrong. (00:15:17.594): And all of that is possible because now you’re They’re measuring how it is (00:15:22.441): on accuracy without compromising on latency. (00:15:25.245): They see these issues and think of problems to solve. (00:15:28.509): I work with companies (00:15:29.931): who have done really beautiful work around how they handle transcription issues. (00:15:34.898): Transcription, (00:15:35.960): it’s largely solved for English, but real world presents so many different cases. (00:15:40.807): Someone who’s speaking Spanish and English, (00:15:43.330): there’s multiple languages, (00:15:45.093): there’s background noises, (00:15:46.355): there’s interruptions, which all of that affect how you accurate your transcription. (00:15:50.380): And people have moved beyond from just relying on the model to do everything (00:15:54.404): to making sure their hardness does capture those. (00:15:57.887): And that’s really the beauty of my work. (00:16:00.008): Honestly, (00:16:00.429): I see all of these amazing engineers, (00:16:02.570): developers, (00:16:03.391): product people coming up with solutions and solutions which are very impactful (00:16:07.995): in real world. (00:16:09.436): A 0.1% accuracy in a healthcare domain means day and night for a person (00:16:14.160): who is having an emergency.
Brian Bell (00:16:15.361): Right. (00:16:15.851): Yeah. (00:16:16.752): And even getting to two nines with voice AI is pretty impressive. (00:16:19.535): So your own research has shown that in a single turn, (00:16:23.660): a system prompt attack succeeds almost 20% of the time, (00:16:27.344): but across multi-turn conversations, it jumps to roughly 92%. (00:16:31.772): What are the implications of that?
Tarush Agarwal (00:16:33.014): It’s big. (00:16:33.695): I can’t name them, (00:16:34.516): but we worked with some people and you’d be surprised in terms of what we found. (00:16:39.904): We got a customer support agent from one of the biggest providers to give us (00:16:43.970): a $150 discount code. (00:16:46.574): Just by saying, oh, you’ve promised me this before over multiple turns. (00:16:50.579): And that’s really the basis of our article and like the big research we have. (00:16:56.146): And it’s also a part of our product where we help red team your agent essentially (00:17:00.372): against all of these realistic real world scenarios where a person might try (00:17:05.398): to Get discounts from a customer support agent. (00:17:09.201): Trick of healthcare receptionist to give medical advice. (00:17:13.067): All of that is just not acceptable. (00:17:15.290): And that is where we like to help (00:17:18.254): the people we work with.
Brian Bell (00:17:19.156): So evals has become a favorite industry.
Tarush Agarwal (00:17:22.901): I think everyone talks about evals. (00:17:28.569): I don’t think most people know how to do it correctly. (00:17:31.854): That’s my honest take. (00:17:33.356): There’s also not one single way of how to do evals. (00:17:37.021): That’s a caveat to that. (00:17:38.462): What I feel is you need to build your own evals. (00:17:42.743): You need to own your evals, right? (00:17:45.364): If you’re working in healthcare, (00:17:46.745): you definitely have the most context of what your agent should do (00:17:50.926): for your customers. (00:17:51.806): The infra around it is the tricky part, right? (00:17:54.887): Let’s say you know you want A test case where the patient is asking about an emergency, right? (00:18:01.842): Yes, you have that scenario. (00:18:03.664): That is the piece you own. (00:18:05.046): You need a reliable infra to make sure that scenario actually plays (00:18:09.092): against your agent completely. (00:18:11.114): You don’t want like The eval itself to misbehave, right? (00:18:14.857): Because otherwise, how do you tell what is right, what is wrong? (00:18:17.438): A failure doesn’t really mean failure anymore. (00:18:19.619): And that is really the tricky piece with eval. (00:18:22.700): It’s not just the context, it’s the infra as well. (00:18:25.881): And what we love to help with our customers with is essentially we bring in (00:18:30.483): the infra, you bring in the context, we make sure you are successful. (00:18:35.045): Right. (00:18:35.466): And you cannot measure a conversational agent which is booking an appointment (00:18:39.472): for you by asking three different questions, (00:18:41.775): negotiating on a time by doing a one shot LLM message. (00:18:45.480): Right. (00:18:45.921): That just doesn’t work. (00:18:47.323): You cannot have a single message. (00:18:49.546): You have to. (00:18:49.947): You have to have a simulation and that is how you eval conversational agents. (00:18:53.689): If you pick coding agents on the other hand, (00:18:56.030): the way you eval them is you give them a unit test and some problem statement. (00:19:01.332): The coding agent solves the problem and you run the unit test to verify is it working or not. (00:19:06.535): Similarly, (00:19:07.135): the way you eval a conversational agent is you run a fake customer, (00:19:11.637): fake patient against that and let a full conversation happen. (00:19:15.399): Domain specific evals are a thing. (00:19:18.235): Coding agents, (00:19:19.036): conversational agents, (00:19:20.417): and people will come up with more and more ways to eval their agent. (00:19:23.860): And I think that’s really what we see. (00:19:26.562): If you think about 24, everyone knew about evals, nobody was using it. (00:19:30.826): 25, it became a bit popular. (00:19:32.987): 26, evals are everything, right? (00:19:36.030): If you don’t have evals, (00:19:37.471): if you don’t have good evals, (00:19:38.952): you probably are not going to be able to ship much faster.
Brian Bell (00:19:42.440): Speaking of evals, (00:19:43.281): maybe there’s another belief in the space that you kind of hold near and dear (00:19:47.708): to your heart that people push back on or vice versa. (00:19:50.793): Something that other people believe that is wrong that you push back on.
Tarush Agarwal (00:19:54.177): I definitely would say people think voice agents I think that’s (00:19:58.243): the thing I have most experience with. (00:19:59.986): People think if they just test the agent logic over text, that’s really it, right? (00:20:05.614): It’s surprisingly incorrect. (00:20:07.897): Think of a case where your agent is asking me for phone number, right? (00:20:12.543): All of this is not possible to replicate over text. (00:20:15.027): We have seen so many different failure modes that surface over text. (00:20:27.405): Only in voice. (00:20:28.406): And that is definitely something I speak to a lot of my clients about. (00:20:32.589): It is not to say you should not devalue a voice agent over chat, right? (00:20:36.692): You definitely should. (00:20:38.253): When you’re testing your agent workflows, (00:20:40.255): start with chat, (00:20:41.055): run a lot of simulations, (00:20:42.216): make sure at least in a controlled environment, which is chat, you are getting 100%. (00:20:46.799): Once you’re there, (00:20:47.800): Move on to voice, run that additional test, see where you really stand in voice. (00:20:52.084): And that is definitely a bit of advice I share with all my clients. (00:20:55.827): And I think something we have learned over the last few years.
Brian Bell (00:20:57.969): It’s really fascinating. (00:20:58.790): So most and I think, (00:21:00.691): you know, (00:21:01.192): the way voice agents started, (00:21:02.853): it was text based, like basically reading from a from a text output. (00:21:07.277): But now it’s become much more voice to voice, right? (00:21:10.059): Rather than voice to text.
Tarush Agarwal (00:21:11.915): I disagree, actually. (00:21:14.351): You’d be surprised most people still use cascading, (00:21:17.804): meaning there’s a transcription, (00:21:19.306): then there’s an LLM, and then there’s a text-to-speech. (00:21:23.292): So yes, the input is still a text. (00:21:25.775): The output is still a text. (00:21:29.100): But the challenge is when I’m inputting a text and I interrupt in between the text, (00:21:34.849): how does your harness handle that? (00:21:36.491): Does it pass the full thing? (00:21:37.733): Does it pass the half? (00:21:39.075): Or does it re-ask the question? (00:21:41.999): That’s the reality. (00:21:42.880): Yeah.
Brian Bell (00:22:06.936): So I would imagine when a customer joins your platform, (00:22:10.459): you guys are able to have a bunch of evals right out of the box (00:22:13.441): because you have all these failure modes that you’ve built up (00:22:15.742): across these probably millions and hundreds of millions of minutes of conversations. (00:22:22.487): Would that be a correct statement?
Tarush Agarwal (00:22:23.968): Yes, definitely. (00:22:25.070): I think as a platform, if we are not learning from what we’re doing, we are doing a bad job. (00:22:30.898): So we have learned over the past two years what are really the different failure modes. (00:22:36.326): We have built an environment, a simulation. (00:22:39.451): Right. (00:22:41.914): We, when you come to us, you probably haven’t deployed a lot of voice agents. (00:22:47.082): If, for example, (00:22:48.304): a massive company comes to me and they ask me to eval their voice agent, (00:22:51.809): they probably don’t have enough data of what real We, (00:22:57.173): on the other hand, (00:22:58.033): working in the voice space for the last couple of years, (00:23:00.854): we have that data and we make sure their simulation environment represents that. (00:23:05.056): If you ask me two years down the line, what would it look like? (00:23:07.537): I would say it would be personalized to your domain, right? (00:23:10.798): Again, coming back to the examples of domains, (00:23:13.179): healthcare, people calling are in a slightly distressed state sometimes. (00:23:17.460): Sometimes they might not be in the best of locations versus customer support. (00:23:21.802): It’s slightly different. (00:23:22.842): People are frustrated. (00:23:24.124): People are more profane. (00:23:25.726): And that’s really the difference between these environments, right? (00:23:28.791): And we do want to represent that. (00:23:30.533): Today, you already get some version of it. (00:23:32.997): And then we measure what is right for your specific environment. (00:23:37.784): We highlighted the key differences on what to measure and our platform, (00:23:41.529): like we do handle all of that. (00:23:43.271): And we definitely want to expand on that because we have seen this working really well.
Brian Bell (00:23:46.915): Yeah. (00:23:47.295): So what is true today that wasn’t true, (00:23:49.879): you know, (00:23:50.399): when you got started, (00:23:51.260): you know, two years ago and how’s it changing in the next two years, you’d say?
Tarush Agarwal (00:23:56.841): Just not good when we started. (00:23:58.563): If I’m being frank, we were seeing latency.
Brian Bell (00:24:01.047): You’re running emails and it’s coming back at like 60% and you’re like, oh, yeah.
Tarush Agarwal (00:24:05.213): Exactly. (00:24:05.714): Voice AI was just not good enough. (00:24:07.376): And the big reason is there’s so many moving parts. (00:24:10.320): So many moving parts. (00:24:11.842): The transcription, the LLM, the text to speech. (00:24:15.384): There’s also a turn detection model in between, right? (00:24:18.965): Knowing when to speak, when to stop. (00:24:20.886): There’s also your harness. (00:24:22.467): This is just on the model layer, then all of the infra.
Brian Bell (00:24:25.308): What’s a harness for people who don’t know what that is? (00:24:28.030): I don’t, but...
Tarush Agarwal (00:24:29.278): A hardness is essentially how data flows across your agent, right? (00:24:33.605): So when, for example, it’s a voice agent, right? (00:24:37.450): Someone is calling in, (00:24:38.612): you probably have a telephony, (00:24:40.575): telephony passes that data as web packets onto your Speech-to-text gives out (00:24:47.245): an output. (00:24:48.206): You start streaming that to your LLM model. (00:24:52.592): While that is streaming, (00:24:53.534): there’s a turn detection model, (00:24:55.717): which knows when to let the speech-to-text model run. (00:24:59.382): When to ask it to stop, right? (00:25:01.926): And all the downstream processes, right? (00:25:04.029): Like the LLM generating output, speech to text, text to speech giving the final voice itself. (00:25:10.637): And then forwarding all of that voice to the telephony back. (00:25:14.102): And that is just like the basic audio. (00:25:17.006): You can also divide your agent then. (00:25:19.189): So you mentioned there might be a smaller, much faster agent giving you that fast experience. (00:25:25.899): But on the background, you’re running a more bulkier agent, more bulkier LLM. (00:25:31.647): The reasoning through every turn, right? (00:25:34.531): And that is really all of this, all of this architecture, hardness is just a fancy word for it.
Brian Bell (00:25:39.018): So what’s true? (00:25:40.039): How do you think this category unfolds over the coming years?
Tarush Agarwal (00:25:43.083): Definitely. (00:25:43.965): So yeah, as I was saying, Voice Air was just not good enough. (00:25:46.688): And we’ve seen it progress to becoming really good in production. (00:25:50.674): I think there’s a lot of room still, and a lot of that will be solved, right? (00:25:55.501): As transcription becomes better, (00:25:57.685): as voices become more natural, (00:25:59.667): as limbs become smarter, (00:26:01.550): And also when speech to speech comes out, we have been measuring this. (00:26:06.134): If people don’t know here, (00:26:08.355): we did launch our public benchmarks for all the voice providers and how they perform (00:26:12.779): in a default setting. (00:26:14.400): So we have been measuring this. (00:26:16.482): We want to measure more such agents. (00:26:19.624): We want to measure more of these speech to speech agents and essentially Help move (00:26:25.416): the industry forward, right? (00:26:27.679): We see latencies come down from four seconds to sub two seconds now. (00:26:32.285): We have seen the accuracy really improve from those 60% nightmares to really 99.99%. (00:26:40.809): I think that is still not good enough. (00:26:42.752): Think there’s billions of conversations, (00:26:44.975): phone calls happening every day, (00:26:47.219): just on customer support, just on customer support, nothing else, right? (00:26:51.184): All of this will be done via voice AI. (00:26:53.307): And if you take 99.99%, That’s still not good enough. (00:26:57.073): You’re doing millions of conversations, which are still bad. (00:27:00.238): And that is what we need to improve. (00:27:02.100): We really need to get to those five, six digits of nines.
Brian Bell (00:27:05.245): Yeah. (00:27:05.666): What does success look like for you guys long term? (00:27:07.789): You know, if you kind of put yourself, you know, eight, ten years in the future.
Tarush Agarwal (00:27:15.450): I think 10 years down the line, I would firstly still want to do this. (00:27:19.165): I would still want to build. (00:27:20.567): If we have probably solved voice end to end, we’ll move beyond voice. (00:27:24.774): We’ll do video. (00:27:25.795): Build to physical AI, right? (00:27:27.878): That’s really what I want to. (00:27:29.521): I don’t want to stop building. (00:27:30.762): But if you ask me if I solve those billion conversations happening every day so (00:27:35.489): that I don’t have to go wait 30 minutes in a queue just to speak to a person (00:27:39.796): and resolve A stupid issue with my order, right? (00:27:43.959): I don’t want to do that. (00:27:44.779): Nobody wants to do that. (00:27:45.900): If I can make that possibility a reality, I think I would be quite happy. (00:27:51.944): And I want to get there faster. (00:27:53.545): I think 10 years is still too slow. (00:27:56.407): Yeah, I
Brian Bell (00:27:56.807): love that. (00:27:57.287): Well, yeah, (00:27:58.288): I think people, (00:27:59.428): you know, (00:27:59.749): they overestimate what they can do in a year and underestimate what they can do in (00:28:03.711): a decade. (00:28:04.131): So you’re starting to hire, right? (00:28:05.392): How’s that going? (00:28:06.052): I have my notes that you’re starting to hire... (00:28:08.826): So if you know me, as I mentioned, I’m an engineer at heart. (00:28:10.949): So all of the last one, and a half years, I did do a lot of products.
Tarush Agarwal (00:28:24.489): We did hire our engineering team last year. (00:28:30.858): So we are a team of 12 engineers now. (00:28:33.743): And very recently is where we are transitioning out from founder, (00:28:38.349): let’s say, We feel like we’re at a point where we’ve built something repeatable. (00:28:45.416): We’ve learned a lot on what are the pain points that really resonates with our customers, right? (00:28:51.201): What does a sales cycle look like? (00:28:54.364): How to align them on success criterias? (00:28:56.987): And that is really where we feel like, okay, now if we expand this team, we can expand. (00:29:03.044): What’s a metric that you guys obsessively watch? (00:29:05.967): What’s a metric you guys obsessively watch? (00:29:21.124): Usage would be one. (00:29:22.084): I think, yes, revenue is good. (00:29:24.185): Yes, where people are finding us. (00:29:26.986): Yes, how much usage is there. (00:29:29.788): I think usage comes with a caveat of good usage, right? (00:29:33.009): I can build a really shitty voice agent, (00:29:35.070): deploy it for hundreds of thousands of calls, but that doesn’t mean anything. (00:29:39.934): I want to make sure if my customers are doing a million conversations (00:29:43.319): and I’m telling them 10% of their conversations can be improved, I really mean that. (00:29:48.306): So usage on how they’re using us, how it is scaling up and the quality of that usage. (00:29:54.215): And yeah, the day it dips, I’m sad. (00:29:56.697): The day it increases, I’m quite happy. (00:29:58.779): Currently, I’m quite happy because yes, a product is largely used by agents now. (00:30:04.724): So Claude is writing the code. (00:30:06.566): Claude is testing itself. (00:30:08.147): Claude is doing the evals via Secura. (00:30:10.389): And one thing is it’s definitely a lot easier to let Claude use our product and (00:30:15.114): that usage just keeps going up. (00:30:16.475): Right.
Brian Bell (00:30:16.795): I think people fail to realize that, (00:30:19.036): you know, (00:30:19.596): when AI starts kind of going into these other parts of the economy, (00:30:25.098): it’s 10x larger than software, right? (00:30:27.319): All of software is like 700 billion, but all of like services is like 10 trillion, right? (00:30:31.581): And then maybe more than that. (00:30:33.001): And so like voice agents, I mean, (00:30:34.762): you think about the sheer number of people who need customer support every day. (00:30:38.683): It’s billions of people, right? (00:30:40.524): Spending billions of minutes every single day. (00:30:44.668): Like needing help on stuff, right? (00:30:46.689): Setting appointments, you know, rescheduling my dentist appointment, like whatever it is. (00:30:51.572): It’s just, there’s just a ton of that. (00:30:53.934): And, you know, (00:30:54.534): it’s kind of like the way I think about it is like, (00:30:56.795): you know, a datadog exists in kind of the software world, right? (00:31:00.518): But Sakura is almost like this datadog for every, (00:31:03.019): like voice agents, (00:31:04.180): which, and if you think about it, that’s all of the interactions that we do. (00:31:07.862): Not just the software interactions, but all the service interactions. (00:31:11.504): So the opportunity is much larger. (00:31:12.905): Yeah. (00:31:13.793): And we’re just we’re still like in the first two or three years of this journey, right? (00:31:18.360): I mean, voice AI platforms didn’t really exist five years ago. (00:31:20.922): I mean, it kind of existed, (00:31:22.303): but it was like, (00:31:23.043): you know, like very simple rudimentary speech to text or text to speech, right? (00:31:27.006): It wasn’t what we have now, agentic voice. (00:31:29.127): I’d love to wrap up with some rapid fire questions. (00:31:32.950): What’s the best advice you ever got that you initially ignored?
Tarush Agarwal (00:31:36.272): I think focusing on your problem, right? (00:31:38.994): We could have very well gone and built a voice agent platform. (00:31:42.796): We could have gone and built models. (00:31:44.417): We could have done simulations for everything. (00:31:47.439): We could have built a generic tracing platform, (00:31:50.123): but what we really wanted to solve for is conversational agents. (00:31:54.189): And that’s the best piece of advice. (00:31:56.773): We just kept our focus on working really closely with our customers, (00:32:00.799): knowing them, knowing their problems. (00:32:03.062): Knowing how they use the platform and building from there. (00:32:06.183): Not coming up top down and be like, hey, let’s go build this, right? (00:32:10.886): That just doesn’t work. (00:32:12.487): And coming from like a job, right? (00:32:14.608): Where usually you’re told what to do versus having to figure out (00:32:19.211): from ground up what is working, (00:32:21.072): what is not working, what really even needs to be built. (00:32:25.234): And that piece of advice was definitely one of the key things I hold near and dear to me.
Brian Bell (00:32:31.178): Speaking of advice, what advice do founders constantly repeat that you think is wrong?
Tarush Agarwal (00:32:35.964): I think one of the things is it’s very popular today that you don’t need (00:32:39.910): to hire more people, just keep more agents. (00:32:41.933): I strongly disagree with that. (00:32:43.455): I feel we can do so much more, right? (00:32:45.698): We can be solving 10 problems for our customers and moving our product that much faster. (00:32:51.547): There’s a reason teams grow as they grow in the problems they’re solving and (00:32:56.233): as they grow in terms of like the revenue and the stages of their life cycle. (00:33:00.780): What do you imagine, you know, we’re really in the far, far future?
Brian Bell (00:33:21.851): You know, IPO got acquired, you’re dead or, you know, whatever. (00:33:26.832): And people look back and like, (00:33:28.053): what do you think, (00:33:28.693): what kind of impact do you hope to have left both with Sakura and for yourself?
Tarush Agarwal (00:33:32.174): For myself, I think learning is the key metric, right? (00:33:35.815): I constantly want to build. (00:33:37.135): And I think when I build, I learn the most. (00:33:39.175): And yes, that would be my personal goal. (00:33:41.316): Like, I know this world very closely. (00:33:44.300): I know the things I’ve built make sense, make, add value to the world, solve real problems. (00:33:51.070): In terms of Sakura, (00:33:52.191): I definitely want Sakura to be the only thing that I do in my life from now on, (00:33:56.498): right? (00:33:57.079): I want to keep doing it till the day I die. (00:33:59.082): I want to make sure it goes on beyond. (00:34:01.986): As I said earlier, I want to solve voice. (00:34:04.750): I want to make sure every voice agent that we deploy, (00:34:07.314): voice agents that people speak to for customer support, (00:34:10.058): at hospitals, (00:34:11.380): when they And then from there, (00:34:12.581): I want to solve more problems, (00:34:14.124): not just voice, let’s do chat, let’s do avatars, right? (00:34:16.387): People want personal companions. (00:34:27.022): I want to solve all the agents and I want to solve robots. (00:34:31.284): Really 10 years down the line, (00:34:33.405): I definitely, maybe not 10, like maybe just five years down the line, right? (00:34:37.786): Maybe three years down the line, physical AI is going to pick up. (00:34:41.067): And I want to make sure when you’re interacting and talking to those robots, those make sense. (00:34:46.609): Those robots are doing the right actions. (00:34:48.850): They’re taking all the control. (00:34:50.431): And that’s really the impact I want to leave with Secura.
Brian Bell (00:34:53.432): Love it. (00:34:53.832): Well, I really enjoyed the conversation. (00:34:55.133): It’s always good to catch up with a portfolio founder. (00:34:58.015): Where can people find you online and find out more about Sakura?
Tarush Agarwal (00:35:01.198): Definitely. (00:35:01.698): I think do visit our website. (00:35:03.820): It’s sekura.ai, C-E-K-U-R-A dot A-I. (00:35:08.103): I’m also really active on Twitter. (00:35:09.704): So just go on to Twitter, look me up. (00:35:12.827): It’s Tarush Agarwal. (00:35:14.328): That’s the handle, basically. (00:35:16.490): And yeah, honestly, just shoot me an email. (00:35:19.293): I’m always happy to talk to people who are interested in the space, (00:35:22.637): who love building and everyone.
Brian Bell (00:35:25.281): Thanks so much, Tarish. (00:35:26.202): Really appreciate it.
Tarush Agarwal (00:35:27.343): Thanks a lot, Brian. (00:35:28.284): Enjoyed talking to you. (00:35:29.646): And yeah, always good to see you. (00:35:31.488): Yeah, no question.
