Listen on
Overview
AT&T processes about 45 billion AI tokens a day, and open models already handle roughly a quarter of them. Andy Markus, the company’s chief data and AI officer, told The Wall Street Journal he expects that share to reach 70 to 80%, and that switching some applications to open models cut their cost by up to 90%. The easy reading is that open weights have caught up and every company should follow. Ross Katz’s read is that most of AT&T’s savings come from a smart router that sends each request to the cheapest model able to do the job, and that a company without that volume will find hosting a model costs more than it returns.
In this episode of Eventual Consistency, Ross Katz (Principal and Data Science Lead at CorrDyn) and Jason Bradwell (founder of B2B Better, host of Pipe Dream) take the AT&T story apart for companies nowhere near its size. Ross puts the break-even in dollars. One to two engineers cost $250,000 to $500,000 a year, GPUs bring a self-hosted deployment to $600,000 to $800,000 a year, and so a use case worth less than $1 million a year does not justify the effort. Below that line, the savings come from routing across shared inference providers such as Fireworks, Together, and Baseten, or from the model serving already inside Databricks, Bedrock, Azure AI Foundry, or Vertex.
Open source and open weight differ less than the terms suggest, and for a buyer the license is what matters. Open weights earn their engineering cost on high-volume extraction, regulated data that cannot leave the building, latency a team has to control, and fine-tuning on proprietary data, where CorrDyn clients have taken an eight-billion-parameter model past a frontier model with 3,000 training examples. Ross runs a Chinese-origin Qwen model on CorrDyn’s H100 and separates two risks the headlines merge: prompts sent to a Chinese-hosted API have leaked, while backdoors hidden in downloaded weights remain unproven. The episode closes on the AI bubble debate, where Ross argues that a burst would hand companies with working use cases cheap GPUs and cheap open models.
Key Takeaways
Most of AT&T’s savings come from the router
Four setups get collapsed in most conversations about open models. Closed models run on shared inference through the Anthropic or OpenAI API; open-weight models run on shared inference through Fireworks, Together, Baseten, or Amazon Bedrock; AWS and Azure sell provisioned throughput of a closed model; and a company can self-host an open-weight model on infrastructure it controls. Shared-inference open-weight models cost much less per million tokens than closed models, which is why the opportunity exists. The router turns that price gap into savings, because it chooses per workload where the cost and performance tradeoff sits. Ross reads Stripe’s acquisition of OpenRouter as a bet that the same price arbitrage will spread from enterprise to mid-market companies.
Self-hosting costs $600,000 to $800,000 a year before anyone tries hard
The open-weight models with frontier capability are too large for most companies to host. They span multiple GPUs and need heavy inference tuning to reach the reliability users expect. Even a smaller model at high throughput needs people, and people are a large share of the bill: Ross estimates one to two full-time engineers at $250,000 to $500,000 a year. Rented or purchased GPUs bring the total to $600,000 to $800,000 a year. His threshold follows from that arithmetic. If a use case is not already worth more than $1 million a year, a team should not start the self-hosting conversation.
Open weight and open source differ in what you can reproduce, and the license sets the cost
An open-weight release gives you the weights, and the weights are all inference needs. An open-source release adds the training code, the training data, and the recipe used to curate it, so you can reproduce the model, study it, and change how it is trained. For a company trying to get value from AI, the practical difference sits in the license. Some Llama licenses require payment to Meta past a certain scale. The license to look for permits any derivative work, leaves the IP with you, and does not charge for use.
Open weights earn their cost in four kinds of workload
High-volume classification and extraction comes first: large stores of unstructured documents, or logs streaming off manufacturing equipment and software systems. Regulated or personally identifiable data that cannot leave the organization is the second. Fireworks holds SOC 2 Type 2, HIPAA, and GDPR compliance, and some compliance teams still judge the risk too high. The third is latency or reliability that a team cannot trust a third-party host to deliver. The fourth is fine-tuning on proprietary data that a company will not send outside, whatever assurances a vendor gives.
A fine-tuned eight-billion-parameter model can exceed a frontier model on a narrow task
On a relatively narrow, repeatable task, CorrDyn clients have exceeded frontier-model performance by fine-tuning a model of about eight billion parameters on 3,000 examples. The small model keeps sensitive data inside the boundary, keeps the IP with the company, and improves as labeled examples accumulate. The same economics apply to document-heavy small and mid-sized businesses. Sending a million documents to Claude Opus for extraction produces an estimate that would make their eyes water, while renting or buying a GPU to do the same work is economical.
The documented Chinese-model risk sits in hosted APIs, and the weights are a separate question
Ross runs a Qwen 3.8 27B model on CorrDyn’s internal H100 to test agentic coding. He downloads the weights to a network-protected machine instead of calling the Alibaba, DeepSeek, or Kimi APIs. Data exfiltration through an API has happened: South Korean regulators found DeepSeek transferring prompts to other Chinese companies without consent. Ross has seen no evidence of backdoors hidden in model weights, and the ability to plant one has not been established. Censorship and propaganda bias exist in these models, though Ross expects fine-tuning to remove them and sees little path from them into deterministically written code. The risks that need attention are ordinary ones: observability, network controls that stop the model sending data to unknown destinations, prompt injection when third parties can reach it, and a verified model supply chain.
Start with the use case and an evaluation set
Ross asks every data leader with a large AI bill the same question. What is the use case, and do you have examples of it that you would use to evaluate an AI system? With that evaluation set in hand, expected end-user latency, unit economics, and usage pattern (steady batch work or short interactive bursts) decide the model, the provider, and whether to host, rent, or buy a GPU. Where the data already lives often settles the matter. A Databricks shop will use Databricks model serving, and teams on AWS, Azure, or Google Cloud should look hard at Bedrock, Azure AI Foundry, and Vertex before hosting anything. Ross expects small and mid-sized companies with large document collections to find open-weight use cases over the next year or two, as the capability gap closes and hardware costs fall.
What We’re Watching: a burst AI bubble would favor companies with working use cases
Ross puts the bubble debate in the same line as earlier panicked narratives about generative AI, such as the claim that it would take every job. He sees generative AI adding value at businesses now, with more to come over the next three to five years. If the bubble does burst, companies that already have a use case would get abundant GPU capacity, cheap GPUs, and cheap open-weight models. The critics he would listen to most closely are the social ones, asking whether AI belongs in schools, whether children should have it, and whether it leads people to outsource their thinking.
Related: AI Strategy | Data Cost Optimization | 11 Questions: Open LLM vs Third-Party LLM API
Full Transcript
Jason: AT&T processes about 45 billion AI tokens a day. Open models currently handle roughly a quarter of that, and Andy Markus, the company’s chief data and AI officer, expects that number to reach up to 80% in the next few years. In some applications, he cites the switch has cut costs by up to 90%. And his line to the Wall Street Journal is, they’re not scared of a token future. The obvious read may be that open weights are catching up and everyone should be moving in that direction. But we don’t believe that’s quite what’s happening. I’m joined again today by Ross Katz, principal data science lead at CorrDyn, and we get into where open weights genuinely win today, things like high-volume extraction, regulated data that can’t leave the building, latency that you need to own, and where closed models are better. We also get into a really interesting discussion about whether Chinese models are the security risk that the headlines suggest.
Jason Bradwell: So Ross, AT&T, they’re using 45 billion tokens each day. Open models they say are doing approximately 25% of their AI work. Their chief data and AI officer expects 70 to 80% of this to be the open models in the future. What’s your reaction to these numbers? Are open models now ready for large companies, or is AT&T too large to compare with a normal data team?
Ross Katz: I would say it’s a little bit of both. I would say that if you look at the trajectory of open models over the last one year, two years, this year especially, what you see is that the distance between the open models and the frontier models is getting shorter and shorter over time. So if you look at the artificial intelligence index that they put out, or you look at Epoch AI, basically the open weight models are three to four months behind the frontier capabilities. So if you look at a company like AT&T that’s doing 45 billion tokens per day, obviously they’re at a scale where not all of those tokens are going to require frontier-level capabilities. So for them, the value that can be captured by taking a portion of those requests and routing them to open weight models rather than frontier models, or even just lower-tier models that are closed weight models, is just pure money left on the table for them that they can grab back. If you’re in the Fortune 500 and you’re already spending this amount of money on token generation, primarily I’m assuming for agentic capabilities that you’re building in-house, it’s just a clear win to build a routing layer or to contract with a routing layer. You look at Stripe’s acquisition of OpenRouter, I think one would argue that they see this opportunity as well, that especially among enterprise companies, and over time moving more and more into mid-market companies as you expect AI penetration to deepen throughout the economy, the more tokens you’re using across various tasks within your organization, the more opportunity there is to do some price arbitrage where you don’t need the highest level intelligence that’s out there. As open weight models get closer and closer to the frontier and put pricing pressure on their comparative closed models, what you get is a lot of opportunity to switch to open weight models where the capabilities are there and the cost is right, or just wait for the closed models to lower their prices because they’re experiencing the competition and they’re seeing that companies are routing their workloads to alternative models. Bringing that optionality into the ecosystem is just pure value capture with limited sacrifice in terms of the capabilities that you’re giving up in-house. So does that mean that tomorrow everyone listening to this podcast, and even the people who aren’t, should wake up and say, our company that has just rolled out Claude or ChatGPT, and the only thing we’re using it for is $20 a month subscriptions or $200 a month subscriptions where we’re using Claude Code very deeply, we’re using Codex very deeply, or we’re generating a lot of PowerPoints. The value of open weight models is not at that tier of development, where the tasks are very diverse, it’s very specific to each user how they’re using it, and it’s highly dependent on the harnesses that are deeply coupled with the frontier capability models, and it relies on these frontier models to drive whatever task or workflow you’re doing at your desktop to fruition. Open weight models for most people who are not hackers or privacy enthusiasts or people who just like to try out the frontier of the open weight models, the value’s just not going to be there. So the threshold that I would put on it is if you’re considering open weight models and the value that you plan to drive is not greater than a million dollars a year, then you shouldn’t be thinking about it. Yes, you can drive some value here and there, but, as we might or might not get to, the cost to get you there is dependent on the amount of value that you can capture from the workload that you’re planning to shift over to open weight models.
Jason Bradwell: I want to come back to the router in a second, but maybe you can help me understand the terminology we’re using here, because we’re talking about open source models and we’re talking about open weight models. And if I understood the article from the Wall Street Journal, which we’ll link to in the show notes of this episode and inspired this episode, AT&T are using both types. What is the difference between open source and open weight models? Is there a difference? And if there is a difference, when would you use one over the other? What kind of difference does that give to a data leader? Talk about that in the context of costs as well.
Ross Katz: Honestly, for the purposes of most people who are just trying to get value from AI capabilities, the difference between open source and open weight is not meaningful, except to the degree that the licenses that the open weight models are under might or might not meet the standards that you need them to meet in order to allow you to generate your own IP unencumbered over time as you scale up. Open weight models are basically, the weights are made available, but you don’t have access to the source code that was used to create the model, you don’t have access to the data that was used to train the model, or the recipe that was used to curate the data to train the model. You cannot reproduce the model, you cannot research the model, you cannot tweak things about the training recipe and figure out how to train a better model. All of those things are available to you if the model is considered to be open source. When it’s open weight, what’s happening is the weights are being provided to you, which is, in order to run inference on the model, in order to get outputs from the model, that’s all you need in order to do that. But depending on the license that the weights are under, for example, some of the Llama licenses, if you reach a certain degree of scale, then you start having to pay Meta for the privilege of using their model at that degree of scale. What you want is whatever open weights model you’re using to have a license that allows you to do whatever derivative works you want to do and retain all of your own IP and not have to pay the company that created the model for the ability to use the model. So that’s the difference between open source and open weight, but really both should not cost you anything, as long as you’re paying very close attention to the license that they’re under.
Jason Bradwell: Got it. Okay, that makes sense. So let’s come back to this router, because in the article from the Wall Street Journal, they were talking about how AT&T have built this smart router. Basically tasks come in, it selects which model with the lowest cost and can do the task. So how much of the cost saving are we expecting to see from the open models versus the router and its determination of what needs to go where?
Ross Katz: Just to take a step back for a second, you’ve got your closed model, which is your OpenAI, Anthropic, Google, and then you’ve got your open weight model for the most part. And then you’ve got shared inference, where you’re not hosting the API that you’re hitting, and then you have a dedicated deployment, where you are hosting the model yourself. I think that these things can collapse together if we’re not careful about talking about them. You can have closed models on shared inference, which is the Anthropic API or the OpenAI API. You can have open weight models on shared inference, which is companies like Fireworks or Together or Baseten or Amazon Bedrock. There’s lots of companies offering that. Then there’s a dedicated deployment, which on AWS and on Azure you can get provisioned throughput of a closed model, but then you can also self-host an open weight model so that you own all of the infrastructure on which the model is running. Now that we’ve set that baseline, back to your question: how much of it is the model and how much of it is the router? The cost per million tokens of a lot of the hosted open weight models, the shared inference open weight models, is much lower than the closed models. That’s why the opportunity exists in the first place. But the router is really where a lot of the value comes from, because the truth is that if you’re going to host your own models, you need to have reached a degree of volume that justifies hosting your own models. And in a lot of cases it’s not just volume. You want to specialize the models, you want to fine-tune them, you want to customize them and the way that you’re using them, or you have an entire pipeline that you’re stitching together that includes multiple models in the mix. Maybe not all of them are LLMs, and the latency of the entire end-to-end process is what you really care about, and so owning the infrastructure and removing APIs from the middle gets you to the end result that you really need to get to. Those are some of the things that would lead you to want to host your own open weight model. But in the case of AT&T, it’s just the router. AT&T, obviously being a large company, has a lot of engineers who are very familiar with how to deploy and support machine learning models, has infrastructure, probably large on-premises infrastructure data centers where they can host the models themselves, and has 45 billion tokens per day of volume to optimize against. So whether it’s shared inference externally hosted or a dedicated deployment or both, the routing is really where the value comes from, because you can always choose based on the workload where your cost performance tradeoff is optimized most effectively.
Jason Bradwell: Yeah, in the article talking about data centers, AT&T say they have their own data centers. Most companies obviously are not going to have their own data centers. So let’s say a vendor is telling you, or a vendor is telling one of your clients, that self-hosting is going to cost less. What are the questions that we need to be asking ourselves before we make the determination that yes, that is the path that we want to go down?
Ross Katz: If you need frontier capabilities, I can tell you that even though there’s a lot of noise made about open weight models having frontier capabilities, the open weight models that have frontier capabilities are not hostable for the vast majority of companies. We’re talking huge models across multiple GPUs and a lot of inference optimization that needs to be done in order to get the reliability and performance that you expect from those models. And even if you’re hosting smaller models that are open weights, but your throughput is very high and you need to optimize that throughput in order to really get the value from them, you’re talking about not just purchasing GPUs or renting GPUs from a cloud provider, you’re talking about talent. Really, the people that you need to have on staff in order to do this work are a large proportion of the cost. The people alone, you’re talking probably one to two engineers who are full-time focused on this problem. So right there you’re at 250k to 500k a year. And then you’re talking about the GPUs that you’re either renting or purchasing, and in all likelihood you’re up to somewhere in the range of 600 to 800 thousand dollars a year without trying very hard, which is why I say if you’re not at a use case that is already giving you more than a million dollars in value, then you shouldn’t even be thinking about it as a starting point. So here are some places where I think open weight models are the right thing to be thinking about. You’ve got a high-volume classification or extraction process. You’ve got just tons and tons of documents that are unstructured that you need to get data out of in a really easy to understand format, or you want to analyze at scale tons and tons of logs coming off of your manufacturing equipment or coming off of your software systems. These are things that, if the value is really there for you, then hosting your own open weight model might start to make sense. If you’re in an industry where personally identifiable information or data that is under regulatory control can’t leave your organization, then self-hosting makes sense, because it’s removing the problem of sending any sensitive information to any third party. Oftentimes, even though a vendor like Fireworks has SOC 2 Type 2, has HIPAA compliance, has GDPR compliance, depending on who your compliance and regulatory team is, they may be okay with that, or the risk is still too high even if they’ve been certified capable of doing things in the way that you need them to do them. If you’re doing anything that is really latency sensitive, or where you can’t rely on the reliability of the third party vendor who is hosting the model for you, then taking ownership of the infrastructure and hosting your own open weight model gives you that degree of control where you can apply the attention of your engineers and you can get that latency to a better place, you can get that reliability to a better place. And if you’re fine-tuning a model on proprietary data and you do not want to send that proprietary data to a third party regardless of the assurances that you may receive from them, then doing all the fine-tuning on your own infrastructure and then hosting the model on your own infrastructure allows you to retain that IP. There are other aspects of this, but you can start to get a sense of where open weights will tend to win today. The other thing I would mention, which is relevant to the high-volume classification and extraction tasks and the fine-tuning on proprietary data idea, is that if you have a relatively narrow repeatable task, then what we’ve seen with our clients is you can exceed the performance of a frontier model just by having 3,000 examples of that task that you can then fine-tune on. You can get a relatively small model, like an eight billion parameter model, to exceed the performance, and then you retain the sensitive data boundary, you retain the IP, and you retain the ability to improve that model over time as you get more labeled examples.
Jason Bradwell: Yeah, a lot of those reasons were cited by Andy Markus, the AT&T chief data and AI officer, in the Wall Street Journal article. He was saying that the open models are necessary for reasons of data protection and intellectual property. They’ve built their own models. I think one of them was called OTel, they’ve trained it with their own telecom data, and this is now helping them find network faults. So I think a lot of that tracks. To switch gears a little bit, he was talking in the article about AT&T’s use of Chinese models, and he does this because he says that AT&T doesn’t want to depend on one supplier, which makes sense at face value. But reading online, and what you see in the media, some people say that Chinese models are a security risk. So these two ideas are at loggerheads. How can you accept both of them, if you can at all? What are your thoughts on that?
Ross Katz: Just looking at what the evidence supports right now, I am self-hosting a Qwen 3.8 27B model that just came out and testing its agentic coding capabilities, because I am in that hacker category. And it is a Chinese model. But I’m hosting it on our own internal H100 GPU, and what I am not doing is calling out to the Alibaba API or the DeepSeek API or the Kimi API and sending sensitive data over the wire to Chinese models hosted in China. What I am doing is downloading the weights to our machine, hosting those weights in an environment where the network is protected and where only the things I allow it to do is it allowed to do. So I would say that data exfiltration via an API is actually a thing that has happened. I think South Korea’s regulators discovered that DeepSeek, for example, was transferring prompts to other Chinese companies without consent. But the idea of these hidden backdoors that are built into the weights themselves, I just haven’t seen any evidence of it. It’s basically just like any other software, except it is non-deterministic software that is run within all of these frameworks. It’s not even like you’re downloading a code base that could have malicious code snippets inside it. The ability to put these hidden backdoors in the weights has not really been established. So it’s not something that I’m really focused on or concerned about at this moment, especially if I’m exercising sufficient controls around the model itself. The other thing that people talk about is this idea of censorship or propaganda bias in the models themselves. And it exists. But if you’re fine-tuning, then it’s possible this doesn’t matter to you. It’s just going to go away with the fine-tuning that happens. And if you’re doing agentic coding, it’s hard for me to see the connection between deterministically written code that accomplishes your goal for your business and the potential that there might be political censorship or propaganda that is somewhere in your codebase. So I would say that it is your responsibility, if you’re hosting one of these models, to own the observability, to make sure that the infrastructure meets your compliance obligations and your requirements, so that the model isn’t able to send data over the wire to an unknown destination, for example. If you’re exposing the model to external third parties, you should be trying to protect against prompt injections and things like that. These are all things that you need to own. The model supply chain, for example, is another thing: making sure that the model you think you’re downloading is the model that you’re downloading, so that you’re not downloading a model that has never been used and might have malicious intent somewhere in it. That’s all of the stuff that you need to worry about, but I wouldn’t be spending a lot of time thinking about the idea that because the models themselves are of Chinese origin, that is somehow an important risk from the perspective of using the open weight versions of these models. There are a lot of security concerns that you should be worried about with these models, but that’s not the top of my list.
Jason Bradwell: So I guess there’s the media bubble, right? And there’s the fear mongering, and there’s everything that you see in the news about how these Chinese models could be negative, in whatever way you want to interpret that. But I’m curious from your perspective, as someone who’s working with businesses on the ground, and working with not just big businesses but small and medium-sized businesses who’ve got much bigger problems to be thinking about: is this something they’re even concerned with? Is this something they’re coming to you and asking for solutions around yet?
Ross Katz: If it’s a relatively small business that’s a going concern, you’re not talking about a technology startup or a company who’s building AI capabilities in-house, then it’s not something that we’re really talking about all that much. But there’s a reason why I wanted to have this conversation today, which is that I think that it is a harbinger of things to come. A growing number of organizations are going to be looking to route requests to multiple different model providers, whether they be closed or open. These questions are going to get asked about open weight models, and about whether they can trust open weight models or how concerned they should be about open weight models that are being hosted internally or hosted by a third-party provider. As AI penetration increases, the opportunities to save on cost while getting reasonably similar capability are going to be there for a lot of organizations. So I think that’s going to start with routing, but I also think that as organizations start to discover the capabilities of these models, because I still think even after all these years we’re at the very beginning of discovering what these models can do from the perspective of improving our businesses, as companies discover use cases that can be high volume, there are lots of small to medium-sized businesses that have huge document corpuses inside their organizations with really valuable data locked up in them that could be used to drive value at their businesses. And it’s those kinds of use cases that open weight models are the clear choice for. It’s just not economical to send a million documents for data extraction to Claude Opus. For any small to medium-sized business, your eyes would water once you get the cost estimate for a project like that. But it is economical to either rent or purchase and host a GPU to do that work for you. And I think that as the model capabilities advance and you get more multimodal capabilities, then you start unlocking the value of the unstructured data that’s in image format, and arguably that day has already arrived here. So there’s going to be more and more opportunities for small to medium-sized businesses to leverage the open weight models that are available. What I see is, even though very few are talking about it right now, other than those that are at the tech frontier, it’s something that organizations should be thinking about over the next year or two, especially as the capability gap closes and the potential for the cost of hardware comes down as well.
Jason Bradwell: That’s a great segue into the question I want to use to wrap up this part of the interview, which is: Andy in the article was saying he’s not afraid of a token future. He says that AT&T can control it. So if you think about a data leader that’s got a large AI bill and they’re not using open models yet, what are the questions and decisions that you think they need to be asking themselves and taking, so that if we record this episode again a year from now, they will have set themselves up for what you see as the future?
Ross Katz: It’s the same question that I ask over and over again when it comes to AI. It’s: what is your use case, and have you come up with examples of what that use case looks like in practice that you would use to evaluate an AI system in terms of its capabilities? Once you have the use case and you have that evaluation set, then you can start thinking about: okay, what is the model or models that we should be considering in order to drive that use case? What is the application interface? What is the expected end-user latency that we need to drive? What do we expect the unit economics to be? And then, what do we expect the usage patterns to be? Is this something that is happening in batch and is fairly consistent over time, or is it something that’s very interactive and it’s happening in short bursts over time? These are going to govern decisions that you’re going to make about what model you need, what provider you’re using, whether you’re going to use a provider at all or whether you’re going to host, whether you’re going to purchase a GPU or whether you’re going to rent a GPU. The other thing to consider is, where is your data and your data workloads living today? If you’re already in Databricks, then you’re just going to use Databricks model serving, because your data’s already there and they’re already meeting all of your governance controls and they’re already exposing both open and closed weight models to use for these kinds of workloads. You’re just going to do that. If you’re already focused on AWS or Azure or Google Cloud, you’re probably going to take a hard look at Bedrock and Azure AI Foundry and Vertex respectively before you start hosting your own models. So I think that all of those considerations are the questions you should be asking yourself. But really it comes down to how is AI going to drive value for your business, and is it at a scale where open weight models are even worth consideration? Because maybe you don’t trust OpenAI but you trust Anthropic, maybe you don’t trust OpenAI or Anthropic but you trust Google, maybe you’re willing to trust one of the emerging inference providers for open weight models, Fireworks, Baseten, Together, but all of the choices that you’re going to make are going to be driven by your unique requirements for your unique workload. And that’s where we come in to have the conversation with you about what you’re trying to accomplish and whether open weight models even make sense in the context of the use case that you’re trying to drive.
Jason Bradwell: So Ross, What We’re Watching, my favorite segment of Eventual Consistency. What have you had your eyes on over the last couple of weeks in the world of data and AI?
Ross Katz: The conversation continues about, is this a bubble or is this not a bubble in the AI space? And I feel like this open weight conversation that we just had is pretty relevant to this idea, because obviously there’s a lot of infrastructure being built, there’s a lot of data centers being built, there’s a lot of energy that needs to be created and consumed as part of the integration of AI into the economy. One of the big concerns right now is, is all of this investment founded on nothing? And it just reminds me of the other panicked narratives that we’ve had in the AI space since the beginning of this generative AI renaissance, where AI is going to take all of our jobs, is AI going to be able to do everything that we do? Is AI a bubble? Is all of this investment going nowhere? I think that all of these disruptions are real and tangible, and I don’t want to diminish them, but they’re real and tangible over only a short-ish time horizon. And I don’t think that AI is taking our jobs right now. I think that that is very much overblown. I see the value that generative AI is driving at businesses, and I see the opportunity for more value to be driven over the next three to five years. And will the creative financing regimes that these large tech companies have undertaken in order to build out all of this infrastructure yield returns? As a result, will the stock market drop and all of our pensions and retirement savings go nowhere? I think that while reasonable questions, they’re just overblown, and what people should be thinking about is, what is the value that we hope to drive using this technology? And if the bubble bursts, honestly, for people that are using AI, it’s going to be just great. There’s going to be endless GPU capacity, you’ll be able to buy GPUs very cheaply, open weight models are going to be available to you to use on the cheap. If you already have a use case in mind that’s driving value for people, then the bursting of the AI bubble is actually a great thing for you. So if you’re just sitting on the sideline twiddling your thumbs, invested in the S&P 500, concerned about what’s going on with these tech companies, you can park your money in bonds or in cash. That’s an option that’s available to you. But for the people who are out here using this technology and trying to drive value from it, I think it’s pretty clear that it’s just another tool that organizations can use to automate some proportion of their work so that they can serve more people, more companies, in new ways. Maybe that is an overly optimistic take. I don’t want to underestimate the potential for social contagion effects, like what has happened with social media, to take hold. I actually think that the social critics of AI are the people who I would be listening to most closely, because I think that those criticisms, should we be integrating AI into our schools, for example? Should we be giving AI to children? Is it causing us to outsource our thinking and cognitive decline? These are relevant questions that I think everybody should be asking. But in terms of, is all of economic activity going to collapse as a result of AI? I think that it’s just a way of getting into the news. Fear drives more headlines than just slow, steady, boring growth. So I wouldn’t be paying that much attention to it.
Jason: Awesome. Another great episode of Eventual Consistency. Ross’s answer to a lot of the AI questions I ask him seems to be the same one, and it holds in this episode as well, which is you need to understand what is your use case, and do you have a set of examples that you’d use to evaluate a system against that use case? If you get those things right, the rest will follow: which model, which provider, do you rent or buy, do you host it or don’t? If you get it wrong, you’re going to spend up to six figures optimizing something that you may never have needed. For most organizations, the near-term move isn’t self-hosting, it’s routing. And if your data already lives in Databricks or Bedrock or Foundry, much of that decision has probably already been made for you. We’re going to link the Wall Street Journal piece that prompted this whole episode in the show notes, as always. And if you want to work out whether open weights makes sense for your particular workload, that’s a conversation that CorrDyn can have. So simply head to CorrDyn.com and we’d be happy to have that chat. We’ll see you on the next episode.
