Listen on
Overview
When AI can produce a plausible ten-page analytics report, spin up a data pipeline, or generate a credible-looking infrastructure assessment in minutes, the question every services firm is quietly asking changes. It is no longer “what should we charge for this.” It is “what is the client still paying for at all.” The honest answer has almost nothing to do with AI capability and almost everything to do with three things the model does not provide: whether the client can tell the work is right, whether anyone can recover when it goes wrong, and whether what is being built compounds on a foundation that holds. Where the answer to all three is yes, AI displaces the work. Where any of them is no, expert services get more defensible, not less.
In this episode of Eventual Consistency, Ross Katz (Principal and Data Science Lead at CorrDyn) and Jason Bradwell (founder of B2B Better, host of Pipe Dream) come at the question from two sides. Ross from data services, Jason from marketing agencies. Ross opens with a three-test framework for where AI displaces expertise and where it does not: verifiability, recoverability, compounding. Jason maps it onto the agency pyramid he sees collapsing from the bottom up. AI removes the junior rung, forces the middle to retrain, and pushes the future of services toward smaller specialist boutiques that use AI as leverage rather than as a replacement.
The conversation moves through the trust problem that quietly got harder in the AI era (the deliverables clients used as trust signals can now be prompted into existence in minutes), why hourly billing has stuck around despite the value-based-pricing hype, what the iron-triangle illusion is hiding, and why “I can build it now” is the worst single reason to build something. It closes on whether the heavily subsidized pricing of today’s frontier AI tools is a real future risk or an overblown one.
Key Takeaways
Verifiability, recoverability, compounding: three tests for what AI does not displace
Ross’s frame: figure out what AI does that services firms no longer need to do, then look at the residual. The residual sits at the intersection of three questions. First, verifiability: can the client recognize whether the work is right? A brochure website passes that test, because the client can see whether the result matches the brief. A data pipeline does not. Second, recoverability: if the work is wrong, can the team recover? A website typo is easy to fix. A court filing, a production data system, a piece of legal correspondence — those get one shot. Third, compounding: does the work need to sit on a foundation that holds for years? Most homeowners do not pour their own concrete. The services work that survives AI is the work that fails at least one of those three tests, and that is where the buyer should be willing to pay for outside expertise rather than prompting a tool.
The agency pyramid is collapsing from the bottom up
Jason’s read on the marketing-agency model: it has run for decades on a wide base of juniors doing the grunt work, a middle rung managing them, and leadership at the top. AI is removing the bottom rung. The middle rung now has to retrain to do what juniors used to do, with AI as the leverage. The output side of the equation still looks the same to clients. The shape of the business behind it does not. The future Jason sees is fewer large generalist agencies and more specialist boutiques: small shops with one strategic edge, using AI to handle the work that used to need a junior team. The work where the human still has to sit in the middle is the strategic, point-of-view, voice-of-the-firm work that AI cannot do without the firm doing the work first.
Specification cost is exactly where domain expertise lives
The more specific the requirement, the more deeply someone has to prompt an AI to meet it. The gap between “I gave the model a brief” and “the model gave me the thing I needed” is where expertise lives. A general blog post on a generic topic: AI can write it. A post that captures a company’s particular view on a topic-specific problem from a specific competitive angle: someone has to know enough to spot what is missing in the draft. That is the verifiability test from earlier, showing up again. Seeing the gap requires knowing what the ideal looks like, and years of doing the work is what builds that knowledge. AI is not going to shortcut it because the inputs it would need are the same inputs the expert needs.
Trust signals depreciated faster than trust itself
The deliverable used to be the trust signal. A polished proposal, a well-produced assessment, a thoughtful nine-page document — clients used those artifacts to decide whether to engage a firm. Those signals have depreciated fast because the artifacts can now be prompted into existence in minutes. Clients increasingly probe past the artifact: is this you, or is it the tool? The trust gap moves into communication, process, and relationship. SaaS platforms have spent two decades trying to productize trust. The services firms that survive are doing the opposite, showing every step of the work so the buyer can see the standards behind the output.
The iron triangle did not break; the cost moved into the future
Fast, cheap, good, pick two. AI has convinced a lot of people the rule no longer applies, because the output now looks right at a speed and price that used to be impossible. What the speed hides is the debt: the integration layer nobody designed, the maintenance burden that lands eighteen months in, the rebuild that becomes inevitable once the business tries to grow on top of something that was not built to carry the weight. Ross’s house metaphor lands the point. AI can build a beautiful-looking house this weekend. The owner will not know whether it stands until they have lived in it for two years. The cost is deferred, not avoided, and the foundation question is exactly the one that the buyer cannot answer for themselves.
Hourly pricing persists because hours got more productive and accountability is what clients are buying
Despite the steady prediction that AI forces a move to value-based pricing, the shift has not materialized at scale. The structural reason: consultant hours have become more productive. The same hour now produces more research, better analysis, and more thorough recommendations, so time-based pricing still broadly captures the value being delivered. The deeper reason is that clients are not just buying output. They are buying the assurance that a qualified human is responsible for catching the errors AI will not flag and for standing behind the work when something goes wrong. Value-based pricing works where the firm can predict both the cost basis and the value. For most consulting engagements, neither is predictable enough.
“I can build it now” is the worst single reason to build something
AI shifts the build-versus-buy calculation, but it makes the question harder, not easier. The buyer still has to weigh total cost over the asset’s lifetime (build is cheap on the front end, expensive on the back end), strategic differentiation (build only what differentiates), the scarcity of expert attention (a top builder cannot build everything well), and the customization premium (which used to favor build but now also favors SaaS, since vendors are adding AI-driven customization layers on top of standardized products). The trap is letting “I could not build it before, now I can” become its own justification. That argument used to be one input. AI has made the build option cheap enough that it now drowns out the other inputs, and the future-cost half of the iron triangle starts catching up about eighteen months later.
Related: AI Strategy | Managed Data Team | Technology Strategy | Episode 21: Acceleration Without Stabilization
Full Transcript
Jason: There’s a question that’s been nagging at a lot of people running service-based businesses right now. If AI can spin up a data pipeline, generate a plausible infrastructure assessment or produce a 10-page analytics report, what exactly are clients still paying for? In this episode, I’m joined again by Ross Katz, principal data science lead at CorrDyn, and we cover that question from two different angles: Ross from the data services side and me from the marketing agency world. What we find is that the answer isn’t really about AI capability at all. It’s about three things: whether a client can tell if the work is right, whether they can recover when it goes wrong, and whether the work compounds on a foundation that actually holds. We also get into what’s happening to the agency model, why the old pyramid is collapsing, why boutique expertise is becoming more valuable, not less, and why the trust problem for service providers has quietly gotten harder, even as the tools have gotten better. Plus, value-based pricing and whether AI subsidies are a ticking clock or an overblown concern. Let’s get into it.
Jason Bradwell: So Ross, on this topic, when AI makes good enough available to everybody, what’s left for experts like you and I that are offering professional, management, and technical services?
Ross Katz: Yeah. So I think before we determine what’s left for us to do, we have to identify what do we mean when we say good enough? Basically, at the highest possible level, my viewpoint is that what AI does that consultancies no longer do is a product of roughly three ideas. The first is who’s doing the verification or how verifiable or legible is the thing to the client or to the person who’s doing the building. So, the build me a brochure website for 25 grand agency, that’s dying because you can build a website by yourself over a weekend that’s as good as the website that one of those companies would have done five, ten years ago. But the reason for that, one of the reasons for that, is that as the client, you’re the one looking at the website and saying this is good enough. This is what I can tell that this is what I want or I don’t like this, I don’t like this, I don’t like this, I don’t like this. Can you make this a little bit different? And if you’re able to do that, then you can already tell ChatGPT or Claude or Gemini to do the thing and it does the thing for you. The second thing to consider that I think impacts my business as a data science consultant, and then your business also as a podcast production, forward marketing agency is how recoverable is the error that is inevitably going to get introduced into the thing? So, using that website as an example, if there’s an error on the website, it’s not like unless you are saying something that is just far beyond the pale from which you can never recover from a PR perspective, like if your design or your branding is a little off, nobody’s really going to notice. You’re going to be the one who notices first typically. So you can recover from that. But if your data pipeline fails, if your production software system fails, if your legal document that you submitted to court is way off base, like you cannot recover from that. You get roughly one, maybe two shots at it and you got to get it right. So you can’t afford to experiment. And what these AI tools are basically doing is making it easy for you to experiment and iterate, which you can then verify what you’ve gotten back. So, if you’re not able to recover from an error, then that’s really good for people who do consulting type work because they’re the people who have the expertise to avoid the errors on the first try. And then the last thing I would think about, in terms of good enough is whether the work compounds on itself, and there’s two ways to think about this. Do you need to build on a solid foundation that accrues value over time and you need help to solidify that foundation and ensure that it is the right foundation for you over the long term? Well, you might want to invest in the expertise that ensures that foundation is really great. I don’t see a lot of people — some people do — but even people who are Mr. or Mrs. Fix It in their homes, not a lot of them are pouring the concrete or standing up the entire edifice of the house itself. They’re fixing things around the edges. What you want in that case is someone who you trust, who has the expertise, who’s going to do it right, because if your house falls down, once again, that’s not recoverable. And maybe you don’t know what a cracked foundation looks like, but I can tell you that somebody who’s poured a million foundations knows exactly what a foundation looks like when it’s been done wrong. That’s how I would classify good enough. And I think if you think that through, it draws the boundaries of where people are going to in-house a lot of stuff using AI and where they’re going to continue to seek external expertise to complement their teams to deliver something that’s really good on that first try.
Jason Bradwell: Yeah, that’s really interesting. And we’re, as you mentioned, coming at this from similar but different angles. You and CorrDyn from the data services side of things, and from my side, as a marketing creative agency.
Jason: I know I can talk to the kind of boundaries that exist within that kind of creative field, but I’m not sure that’s all that interesting to this particular audience.
Ross Katz: I’m interested in it. Will you tell me what inside of the marketing agency field, what are you seeing in terms of where agency work is continuing to grow and where you’re seeing it shrinking and dying away and how do you think about that from your perspective?
Jason Bradwell: Yeah, absolutely. I think about the future of the creative marketing agency model like this. Historically agencies have been built on this idea of a pyramid. You’ve got leadership up at the top point, then a wider rung of middle managers, and then this large rung of juniors and new entrants doing all of the grunt work. That has borne from it agencies that grow to 20, 30, 40, 50, 100, 250 people, and those agencies have been able to survive and thrive because you’ve needed that quantity of people to deliver that quantity and quality of work. What AI is now doing is more or less removing the need for that bottom rung — which is a whole other podcast episode in itself and what that says about the workforce and who’s going to replace us when we all retire. It’s somewhat forcing the middle rung to retrain and adapt and figure out how do we now do the work of that junior rung but not actually have to do the work, but have these tools create leverage for us, and then leadership running the show at the top. The future of the marketing agency model is where we have far fewer large agencies in terms of head count, but far more agencies in terms of volume of shops, and those shops become uber specialist — micro agencies that dial in on one area of strategic expertise, whether that is content strategy, sales enablement, or pick your marketing channel or function of choice. And they are then leveraging these tools to execute on the grunt work. To put this into practical terms, think about a blog writing agency. The process would typically go: brief comes in from client, we need brochure X. Step one, step two. Marketing assistant goes away and performs a bunch of research off the back of that brief — what’s the industry saying, pull some quotes, pull some stats. That research then gets handed over to a copywriter in step three. That copywriter writes the post, hands off in step four to an editor to do the review. The review is then approved, step five, goes to client, step six, everything is finalized and signed off. AI can do all of that, but where it shouldn’t be touching is the secret sauce — step three, the writing bit. AI can take and analyze a brief. AI can run research against that brief and prepare a report. It can then hand that to a human copywriter to create something that is human, net new in terms of thought leadership, perspective, point of view. And then AI can pick up the bat once the copywriter is finished to help the writer do the editing and then do all the handoff and approval process with the client. That is where the boundaries start to become clearer in a world of AI — they are for the people who can bring that strategic expertise, which touches on your three points: verifiability, recoverability, and creating work that compounds. You want someone who’s been around the block who can bring that to the equation, but who also provides cost efficiencies, time efficiencies, and improvement in quality of work by leveraging all of these tools.
Ross Katz: Yeah. Just hearing you talk, it brings up an idea that I’ve been chewing on as well. I love your thesis about the re-emergence of boutique expertise and the ability for that to outperform the large generalist firms. I would say that is as true in data services as it is in the marketing space because the value of the expertise goes up as the generality of the work goes down. It’s worth thinking about that from the perspective of the large language models themselves — these things have been trained on the entire internet. So if AI has seen millions of examples of the exact thing you want to do, it’s going to do a really great job at it. But if it hasn’t seen the particular tailored, language-specific, topic-specific, domain-specific blog post that you want to write from the point of view of your organization, that’s not something AI is going to be able to do for you, unless you basically write the blog post for it. Which is the second idea: the specification costs. The more specific you need to be, the more deeply you have to prompt AI in order to give you the thing you’re asking it for, which comes back to the verifiability idea we had before — you need to be able to spot the gaps in what it’s given you because you understand what that ideal state looks like. And if you do this over the course of years and decades, you’re getting that defensible domain expertise that AI isn’t going to be able to take away from you because ultimately you’ve delivered outcomes, and all AI can deliver at least right now is the output you’re asking it for. Now, this takes a lot of craft-oriented attention to detail, cultivation of expertise from the perspective of the data services consultancy or the marketing agency or the professional services firm, but I also view that as an area where, over the long term, agency type work is going to continue to retain its value even as AI continues to improve.
Jason Bradwell: Yeah, absolutely. I talk about this concept with my team — the technical proficiency gap — and how for the entirety of our careers, if we as marketers have wanted to create a website or design a really great looking brochure, or do something that required a certain degree of technical skill, you would need to go and find someone with that skill. You’d need to hire a freelance developer, bring someone in house, train that skill set internally before you could actually do the thing. Whereas now that technical proficiency gap, at least in the marketing world, is getting smaller and smaller. A lot of marketers I speak to see this as a moment of fear — AI is coming for our jobs. I see it as a moment of opportunity, because as long as you have a creative mind, as long as you have taste, as long as you have expertise in the fundamentals of marketing — storytelling, positioning — you don’t need to be technical in any of these skills in order to make an output a reality. You can prompt something into existence. You can take that ball of clay and put it on to the board in natural language, then just mold it into what you see as the ideal crystallized output for the end client. And that to me feels great. I think what the challenge then becomes for us as service operators is, with everything moving so fast, how do we evolve our offerings and our businesses to meet what are growing demands from clients? I talk a lot about this idea of the iron triangle. You can have things cheap, fast, and good, but you can only have two of those things. And I think these tools that are becoming ubiquitous in our day-to-day professional lives are conditioning clients to at least now think that they can have all three of those things all the time. How is that changing, from your perspective, what you’re seeing clients coming to you with when they’re coming to embark on a service from CorrDyn?
Ross Katz: Basically, what people are asking us for hasn’t really changed, but I think that’s also because of the nature of our work. Our work is so deeply connected to AI and the kind of work that people do. People still need the same data infrastructure, data foundations that they needed previously. It’s just that they see the value of building that up sooner in their organization’s journey, or more comprehensively. That’s why we’re seeing more demand for our services, but it hasn’t really changed the nature of the conversation. What it has changed is the bar for trust that people have. It used to be that when you make a mistake, people understand — you’re doing the best you can. You get one mistake, and just don’t make the same mistakes twice. But now with the availability of AI, people have this illusion — to your point about cost, quality, and speed — which basically says that even though we’ve given you something of very high quality at very high speed at a reasonable cost, if there’s anything wrong with it, there’s always the question of: would AI alone have done it better? What’s happened from my perspective is that the cost part of this triangle is really where the illusions are. Because if you prompt Claude or ChatGPT to build you a house, it might build you a great looking house — I want the sink over there, I want the toilet over there, I want the oven over there. But you’re not going to know until you live in that house for two years that there are fundamental issues with the house, that you basically have to rebuild it from scratch to fix them. The speed with which everyone’s moving to build, build, build, build hides the cost in the future. It seems inexpensive right now, but the bill will come due at some point. My challenge as an expert services provider is to help people see what the risks are, because fundamentally we’re in trust-based businesses. The reason they come to me rather than going to ChatGPT or Claude or Gemini is because we have built up trust with our existing clients and we have built up a portfolio of assets that attempts to demonstrate that we are trustworthy to new clients. All of the signals that people used to use to understand whether you were trustworthy basically came down to: is this output that you’re showing me correct? People always used to ask me to send them an example of the document I produced from the assessment engagement I did on somebody else’s data infrastructure. But the value of that document is rapidly approaching zero because I could have prompted a plausible document into existence. That shifts the burden on services providers like us to find other means of demonstrating our trustworthiness — through our communication, through the processes we engage in with our clients especially when we’re just starting, through the relationships that we build, through the way that we’re able to speak intelligently to their businesses. Fundamentally, this has been something that SaaS platforms have been trying to do for as long as SaaS platforms have existed — everyone’s trying to productize trust. Everyone wants to become the automated platform that people trust, because then you don’t have to worry about the relationship and you don’t have to worry about the processes and you don’t have to worry about the people problems that inevitably crop up. But fundamentally, at the end of the day, we trust human beings — especially experts — to have their incentives aligned with us and to be with us for the long haul. That’s what we’re trying to architect CorrDyn to be: that partner to our clients that demonstrates every step of the way that we’re with you for the long haul and builds that trust while showing you this is how we’re building for the long haul rather than doing the expedient thing in the short term.
Jason Bradwell: Yeah. The idea of trust is so important, and certainly in a marketing context, coming back to where we’re seeing shifts — the delivery of content production is becoming increasingly commoditized, particularly the written word. I definitely sense when speaking to clients or prospects for the first time this probing mentality around: is what you’re talking to me about, the results you’ve delivered or work you’ve delivered — is that actually you, or is it these tools you’re using? But then in the same breath, there’s the expectation that you are using these tools because they’re looking for cost efficiencies, better quality of work, speed of delivery. What I’m feeling is still this interesting point of tension where prospects are expecting you to use these tools, but they’re also wanting to feel like all the work isn’t being delegated to those tools, because then the question is, well, what am I paying for? Why don’t I just do it myself? Trying to rationalize that tension and continue to build trust with prospects who maybe haven’t heard of you or don’t know of you or haven’t come in through a referral — that’s something I’m still trying to figure out. I don’t know if you have any thoughts on that.
Ross Katz: Our outlook on this is basically: deliver value every step of the way. From that first meeting, we’re trying to help people understand the nature of their problem. And if you walk away from that first conversation with two ideas — one is I understand my problem better than I did before, and two is I’d really like to have another conversation with those people because they seem to know what they’re talking about and they can probably help me reduce the amount of time and resources I have to invest in this process — that’s a reasonable way of approaching it. But fundamentally, as a services provider, you can’t skip steps. What I try to do with our clients and our prospective clients is show our work. Because AI is making us more effective. I believe that AI tools are helping us to do more on behalf of our clients. That hour I used to invest was a less productive hour than the hour I invest now, and I think our clients are seeing the benefit of that. Is the only benefit a reduction in cost? No. Because fundamentally what we are doing is investing our time and attention to get our clients the outcomes they’re looking for. What I think we’re getting is more value — higher quality outcomes at a price that justifies those outcomes, with more value going to our clients than we ever have before. How to do that is, at least for me right now, more art than science. Yes, I produced this nine-page document that was AI-assisted, but rather than having me at 10 p.m. pounding keys to get stuff out, what you got was me reviewing that document 10 consecutive times to make sure we were bulletproof on everything we’re recommending, on why we’re making these decisions, with a more comprehensive set of criteria that we’re using to evaluate it and better research inputs that allow us to evaluate more tools than we ever would have been able to evaluate previously. The work still speaks for itself, but only if you tell the story of what you’re doing and why. That’s the art — what we’re trying to practice, but that we’re not always perfect at. We’ll be better next week than we were this week.
Jason Bradwell: I had an interesting conversation with a CEO leadership group that I’m part of in marketing agency land. A lot of the founders across the table from me who are running these marketing agencies started them with time-based pricing — give us an hour, we’ll give you an hour of account manager, blog writer, and assistant, and it’s going to cost this much and every extra hour is going to be the same. The conversation we had a couple of weeks ago was: is this new landscape forcing a move from service operators into a world of value-based pricing? At B2B Better, we’ve always operated with that kind of model. It’s always just made sense to me, coming from being an in-house marketer my whole career and then setting up my own agency for the first time. But I know it’s not the norm for a lot of service providers. Your thoughts on that and how the commercials will evolve for service operators over time in this new world.
Ross Katz: It’s a really hard problem because clients understandably are trying to get the most value they can from the money they’re investing with their services providers. I acknowledge that — that’s exactly what they should be doing. Fundamentally it depends, from the agency perspective, on how repeatable your work is and how well you can estimate in advance how much work you’re going to be able to do for each client and how much value you’re going to be able to drive. Some of the things that we develop — I can point to case studies where the ROI was some absurd number, like 10,000X. You invest 10 hours of our time and what you get is something in the millions of dollars. But could we have said in advance that we were going to save you millions of dollars, and could we have charged you $1 million for the $2 million that we saved you? I think the answer is maybe not. From our perspective, the research I’ve seen out there is basically that there hasn’t been a lot of movement toward value-based pricing yet. I think that’s because what’s happening is the hours that consultants invest are now more productive hours, so the economics just kind of works out — especially because, as we talked earlier, you’re trusting the institution to have the people in the seat who have the expertise to do the verification, to ensure there aren’t any errors that are unrecoverable. I think hours-based pricing will persist for a long time. I think it’s the right model for us because our work is diverse and it’s hard to predict in advance what the cost basis is going to be for us and what the value basis is going to be for the client. But to the extent that you’re in a services business where you can predict the value and you can predict the cost basis, value-based pricing makes a lot of sense and is objectively a better model. There are a lot of people who would rather be in different business models than they’re in, and it’s easier to brainstorm the potential for new business models than it is to actually change your business model — there’s a reason why business models are sticky. Once you have something that works, you don’t want to risk everything trying to change the way that people pay you. And it’s an education game too. If you’re charging people $10 grand per value, then they’re asking themselves: how much is this services organization actually spending to deliver that $10 grand? And if they’re only spending $250 to get there, am I a sucker for paying for that service? Could I just do that for $2,000 rather than them doing it for $250? It’s always a fun game with services businesses. A lot of people have been burned by services businesses, so it comes back to what I said earlier: does your model reinforce trust with clients, or does your model actively get in the way of building trust? We do all time and materials practically. We’ve almost never done a fixed-price project because what we’re trying to do is demonstrate value very quickly and give people the ability to walk away at any time, because we don’t want to have clients who aren’t getting value from our work. That’s why we work the way we do. But if you can get value-based pricing to work, you should definitely do it.
Jason Bradwell: Final question for you Ross, and I want you to give me the answer from two slightly different angles. For a service operator who’s been listening to this episode and is trying to figure out where they stand in this new AI landscape — what questions should they be asking themselves in terms of preserving their longevity and growing their business? That’s the first part. The second part is: as a client coming into a service operator, what questions should you be asking that firm about their business model, the impact AI is having, and the value it can deliver? Let’s start with the service operator themselves. What questions should they be asking as they navigate this new AI-driven landscape?
Ross Katz: I think the question most people are asking is: how do I restructure my processes around AI to make them more effective and more efficient? Fundamentally, you can’t take the process as it existed previously. You have to reimagine the process with the capabilities that exist right now, and you have to build time into your workflow for experimentation in order to determine whether AI can actually do the things that it’s marketed as doing. Because there are a lot of vendors out there that will sell you the dream of what you wish AI could do, but ultimately it’s on you to suss out and ask difficult questions about whether AI-driven workflows can actually do this thing for your company and whether it can be integrated into your business in a way that makes you more efficient and effective and doesn’t increase your volume while decreasing your quality. It’s very easy to drown yourself in 20-page research reports that nobody has time to read or outputs that nobody has time to check. The real challenge is where do you devote your precious expert attention to verify and validate that the things AI is giving you are the things you need. And if you’re going to build in house, you need to build in time for that experimentation and iteration to continue to improve the way you’re doing things in a way that morphs with the demands of the business and the demands of your clients. That’s high level where I would think from an agency perspective.
Jason Bradwell: And now from a client’s perspective — let’s look at it through the lens of someone who’s coming to speak to a data service provider like CorrDyn, who’s maybe already tried to start building some of the solutions themselves internally and is now coming to an expert like yourselves to help either carry them over the line or build something that lasts. What questions should they be asking you as a service operator about your use of AI and the workflows you’re developing?
Ross Katz: Asking how we use AI is a reasonable question — just understanding how it integrates with our workflow. Is it push-button end to end, client gets output, or is it more of a personal enablement, or something in between? What are the automated workflows we have internally? But fundamentally, the question I want potential clients to be thinking about when they come talk to us is: what is the value you expect to get from data, and what do you know about your existing ecosystem that’s either enabling you or preventing you from getting that value from your data? There’s a reason we start with a conversation and not with a survey where you put in all of your information and we spit out an AI-generated recommendation. The iterative question-asking aspect of what we do — to fully understand your landscape, your domain, your strategic requirements and functional needs — that process is human. It responds in the moment. What I said earlier about demonstrating value from the first call, that’s what we’re trying to do: ask you the questions that get us the information we need to give you a set of recommendations on that first call that gets you where you need to go, whether you come talk to us when you need a data services provider or go off and do it yourself. We don’t want to do work that you should be doing yourself. We want to build a relationship with you that means, when that time comes in the future where your business is growing gangbusters and you don’t have enough data capacity internally to do what you need to do, you remember the people who gave you the right advice previously. That’s how we think about building trust over the long term.
Jason: Favorite segment of the Eventual Consistency episode, What We’re Watching. Ross, what have you been reading, watching, or learning about over the last week in data and AI?
Ross Katz: From the very beginning, and this conversation has really touched on it, it’s just been a constant evolution of the build versus buy equation. I’m constantly updating my viewpoint on what does the current state of the data and AI landscape mean for the build versus buy equation — especially in the data tooling ecosystem, but also in website development, podcast production, whatever it is. There are a bunch of things you think about when you’re doing a build versus buy analysis. There’s the total cost over the asset’s lifetime — not just the cost to build, but how much does it cost you to maintain or own it over the lifetime versus the monthly fee you pay a vendor. Building has gotten a lot cheaper on the front end — but does it have a lot of cost on the back end? Then there’s what is the strategic differentiation that the thing you’re building gives you. If more things are buildable, you might be inclined to build it. But if it’s a strategic differentiator, you want to make sure it’s built right. And if you have expert talent in-house, one of the things that’s happened is that because more things are buildable, you’re now considering building even more things. So prioritization has become key, and where you assign your high-quality talent from a build perspective is really important. You might think: this is not a strategic differentiator for us, we should buy it — even though I know that if we put our top developer on it, they could probably build the best possible thing for us. But what I want is the best possible version of the strategic differentiator, and that’s where that person should be devoting their scarce attention. Then there’s speed to value versus speed of iteration. Build is still faster if you have the talent and the attention to devote, but human hours is one of the ways to measure that speed — it’s not just raw days and months. Sometimes it’s easier to just get the vendor on-boarded and plug it in. There’s also customization — fit to the specific workflow. Customization is now cheap. Previously SaaS vendors were super standardized, but now a lot of them are adding that thin skin on top that allows you to customize because you have the AI tools. Maybe the vendor can fit even better than the thing you would build in-house. Anyway, I’m not going to go through everything I’m thinking about. But really what I want people to do, what’s been on my mind, is to not just see “I could build it now and I couldn’t build it before, therefore I must build.” That was only one consideration in the build versus buy equation. People who are earlier in their careers, who are excited about the possibilities of AI, who haven’t had the chance to build as many things throughout their career and then have to own them afterward — those talented individuals tend to be more excited about building new things. It’s incumbent on the people who control the checkbook, who think about the long-term strategy of the organization, to pump the brakes and say: AI is here, which means we can do more, but we’re still bottlenecked by our talent and by our attention. We need to be selective about what we own in-house versus what we want external vendors to own, because we can’t own everything and we can’t maintain everything. To your point about the iron triangle and people thinking it breaks — all you’re doing if you build everything is shift the cost into the future. Maybe that’s somebody else’s problem and that’s your outlook. For me, and for the people we advise, I try not to take that outlook.
Jason Bradwell: I’m curious to hear your thoughts on this. I was reading an article earlier today about the cost of compute, cost of AI, and how all of these tools we’re now using — like Claude and ChatGPT — the pricing is heavily subsidized. I read something where power users of Claude are paying $20 a month for their subscription, but actually costing Claude like $200 a month or whatever the number is. And there’s going to be some sort of reckoning at some point in the future where once you’ve got all the users in the building and they’re all starting to develop and build their workflows and internal tooling in your platform, you can then hike the price way up and they really don’t have any choice but to pay it, lest their businesses suddenly grind to a halt. Is that a genuine fear that those of us who decide to build something internally should have? How would you be thinking about that?
Ross Katz: I think it’s maybe overblown as a concern because the competitive pressures in the foundation model marketplace are such that there’s more than one provider, so they can always cost-cut each other. And then there’s open source models as well, which means that if the open source models continue to improve and approach foundation capabilities, there’s always the next entrant that can just become the hosting provider for all of these open-weight models that are out there. They’re reasonably capable — they are not as capable as the very frontier labs. And really what the frontier labs are doing right now, and I think it’s the right thing to be doing, is they’re building a market. Uber is the comp that people use — then the prices got jacked up, look what happened. But Uber was building a new market. They were proving that that market could exist, and here Uber is years later, still a good business, still used by lots of people. It is more expensive, but ultimately they’re providing a service that provides value to people. I think it’s going to be a bit of a tug of war, but ultimately these tools will continue to provide value far into the future. The question is how long these big billion-dollar investors can continue to finance new R&D that drives the frontier forward, or whether we’re roughly at a more incremental improvement phase of the models. That’s sacrilege to anybody listening to this podcast who comes from a foundation model company — how dare you question the exponential singularity we’re undergoing. But to me it remains an open question as to whether that frontier will continue to grow. I think ultimately you should use the tools you have right now to get as far as you can at a price point that is reasonable to you. Even if you have to slow down your building later, or you’re just using open-weight models to help identify an issue and maintain it later, the capabilities are here for the foreseeable future. Between the foundation model companies becoming more efficient and the competitive pressures they have, it’s probably not going to be an issue in terms of vendor lock-in — and really what the vendor lock-in is is that they have your data. But you’re not going to be any more vendor locked in than you are to any of the other platform providers that were already in your business giving you something of value.
Jason: That’s a wrap on this one. If you’ve been thinking about whether your own expertise sits in a world where AI can approximate the output but not always the outcome, hopefully this gave you a few useful frames to work with. The three questions that Ross opened with are worth keeping. Can the client verify it? Can you recover it if it goes wrong? And does it compound into something that lasts? If the answer to any of those is no, that’s probably still where you want a human with real skin in the game involved. Next episode is coming in the next two weeks. In the meantime, if eventual consistency is useful to you, share it with someone who’s wrestling with the same questions. And if you want to get in touch with the CorrDyn team, you can find them at corrdyn.com. That’s c o r r d y n.com. See you next time.
