Listen on
Overview
In biotech manufacturing, the intuitive drive to measure more can sometimes obscure critical quality insights. This episode uncovers a counterintuitive problem: re-measuring a carefully combined solution with a single sample can reintroduce significant assay variability, making decisions less precise. For data leaders and executives, this means that misdirected efforts in quality control lead to costly re-work, regulatory delays, and impacts on patient outcomes.
Stewart Fossceco, a seasoned expert with over 30 years applying statistics to biological processes and experience in pharmaceutical manufacturing quality analytics, joins us to clarify this challenge. He explains how statistical rigor, not just more data, is essential for building reliable processes.
Stewart and host Ross Katz discuss how experimental design, statistical simulations, and early data capture contribute to true quality by design. They explore specific strategies for understanding and minimizing assay variability, the strategic importance of stability modeling, and practical advice for even small organizations to implement quality-first approaches efficiently.
Key Takeaways
Intuitive measurement strategies can degrade data precision
A common scientific instinct is to confirm measurements, especially after combining multiple samples. However, re-measuring a solution with a single sample reintroduces significant assay variability, undoing the precision gained from averaging many prior measurements. This highlights that more measurement is not always better; targeted, statistically sound measurement is crucial.
Statistical simulations pinpoint critical variability drivers
Tools like Monte Carlo simulations are underutilized for mapping process variability. By modeling how changes at each step propagate, organizations can identify which early process points disproportionately influence final product quality. This helps focus resources on controlling the most impactful variables rather than diffusing efforts across less critical areas.
Quality by Design (QbD) significantly reduces long-term costs
Investing in strong experimental design and process optimization early in development minimizes costly failures and re-work downstream. While it represents an upfront investment, QbD is more cost-effective than trying to ‘test quality into’ a product later, preventing regulatory hurdles and maintaining consistent product quality.
Stability modeling directly influences market and supply chain strategy
Beyond simply determining shelf life, detailed stability modeling informs critical business decisions. Understanding a product’s degradation rate and environmental sensitivities guides cold chain requirements, supply chain logistics, and even the geographical markets a product can reliably serve, directly impacting revenue and distribution.
Related: CorrDyn helps biotech-life sciences clients solve complex data challenges, including data quality and data reliability. Our team often begins with a data assessment to pinpoint areas for improvement, like those discussed for manufacturing.
Full Transcript
Jason: Hi everyone, this is Jason, producer of Data in Biotech. Before we get started, I wanted to let you know about our latest white paper. It’s a comprehensive guide to implementing machine learning models in biotech manufacturing. It’s a complete overview of all the potential problems of ML adoption and, more importantly, how to solve them. To download it, simply visit connect.corrdyn.com/biotech-ml. We’ve also dropped the link in the show notes of this episode. Okay, let’s get into it. Welcome to Data in Biotech, a podcast from CorrDyn where we explore how companies leverage data to drive innovation in life sciences. Every two weeks we sit down with an expert from the world of biotechnology to understand how they are using data science to solve technical challenges, streamline operations, and further innovation in their business. This week we sat down with Stewart Fossceco, former VP of Quality Analytics at Invitae and expert in the field of pharmaceutical manufacturing. During the interview, he and Ross discuss why experimental design and statistical simulations are valuable tools for minimizing variability in biotech manufacturing processes. They break down why stability modeling is crucial for ensuring the quality and potency of drug products throughout their shelf life and unpack why tradeoffs between stability and market demand must be considered in drug development and manufacturing. Here we go.
Ross Katz: Stewart Fossceco, welcome to the Data in Biotech podcast.
Stewart Fossceco: Oh, thanks Ross. I appreciate it. Thanks for the invite.
Ross Katz: Well just to kick us off, could you just give us a tour through your career and an overview of how you came here today?
Stewart Fossceco: Sure. I started off in grad school concentrating on molecular genetics and quantitative genetics and combining those things. I got into statistics, and through my experience at Virginia Tech where I received my graduate degrees, I found a niche where applying statistics to biological processes and research I really enjoyed and found that I could really help people. It drove me down a whole career path over the past 30, 35 years.
Ross Katz: Over the course of that career, can you share some of the high-level considerations that you’ve seen when working in biotech manufacturing and specifically around quality improvement strategies or QA?
Stewart Fossceco: Over the course of the last 25, 30 years, what we’ve really seen is the pharmaceutical and biotech world embracing things like Quality by Design where quality of a product and robustness of that product and the processes that produce it are developed, and that begins much earlier in development as opposed to later — once it’s been submitted, accepted, and then manufactured. I believe that in large part that’s a response to the recognition that we have levers we can pull during development that allow us to create a product and manufacture it to the highest quality standards and do so as effectively and efficiently and cost-effectively as possible, minimizing the overhead and cost associated with failures as they arise in that manufacturing process, which can be very costly for both the pharmaceutical biotech company as well as the patient. You want to keep those products as price efficient as possible for them.
Ross Katz: What does that mean in terms of being involved in the early development of these drugs and bringing them to market? What are the kinds of things that you bring to the table in the early parts of that process?
Stewart Fossceco: In the early parts of the development process, what you’re really looking for is to identify those factors that are driving the quality of the product — particularly around areas or other aspects of the process that you may not have as much control over. The weather, for example, where you may be manufacturing. If something ends up being very temperature sensitive or light sensitive, you can pull various levers during the manufacturing process to make that entire process much more robust to changes in temperature or light conditions, humidity of course if you’re working with a dried tablet formulation. There’s a plethora of information and research over decades that spans not just the pharmaceutical and biotech industry but also car manufacturing and other industries that have looked at how to design — through experimental design — where we identify those factors and then begin to utilize those to develop either the formulations and/or process itself in combination with the formulations that end up going into the patient.
Ross Katz: As you’re bringing a drug to market, there’s a list-making exercise or a brainstorming exercise where you’re writing down all of the factors that need to be controlled for or need to be varied and then designing an experimental regime around those factors to understand how this therapeutic treatment is going to respond to variations in these different factors. Am I thinking about that right, or can you walk through how that looks?
Stewart Fossceco: Absolutely. It even starts one step earlier than the factors themselves — when we think about the measurement systems we have to measure the qualities we’re interested in. It could be efficacy or potency, maybe a particular drug product degradant that forms over time, which comes into play in our stability programs later on. The actual measurement systems themselves, we have to be sure that we understand the variability in those. Because if the variability of that measurement system exceeds what we can tolerate in the process itself, then we’ll have to work on the measurement system to bring down that variability so that we can measure it precisely. Then we get into the world of experimental design where we’re looking to make the assay process — we’re back into the mindset of having another process that we have to optimize. We want to optimize that assay process and identify those factors that make it robust to various conditions in which we will operate it so that whenever we get a result, we know we can trust it and minimize that variability, because that plays into the kinds of decisions we have to make about the product — not just in manufacturing, but all the way into the clinical trial realm where if there’s too much noise in the product, that could impact our decisions in the clinical trial process where those assays might also get used.
Ross Katz: How do you go about optimizing and or minimizing the variability of those assays so that you can be more confident in the clinical results?
Stewart Fossceco: There may be steps along the way in developing the assay where you have various different components chemically that you’re using — say A, B, and C. Those get used and then you may have various incubation times during that assay process, several steps where you have different incubation times and possibly different temperatures. So you have incubation time, temperature, and the actual components that are part of that assay. You look at designing levels — we want to test a high, low level and maybe some intermediate levels within this process. High temperature, low temperature — if we’re running the assay in the middle of the winter and our lab happens to be cooler or less humid than in the summer and it’s a little less temperature-controlled, how will that impact the final outcome? What we want to do is define a space where we can operate that assay and come out with a reliable outcome. We can often use a combination of experimental design and statistical simulations to demonstrate how those factors are impacting the assay and how some activities during the execution of the assay that may make sense to us intuitively actually hurt us. For example, let’s say we are assaying values that are expected standard concentrations and we’re going to look at 240 of those at a time. They’ve been created under a process that’s in control with an expected concentration and those are going to get all combined into an assay and evaluated during that process. We create the solution with all 240. We measure those beforehand, so we have an estimated concentration of each one. It goes into our solution. On average, we have a very good estimate of the average concentration of each of the 240 component mixture. Intuitively we think as scientists we should measure that one more time once we put everything together. So we take a sample from our solution and measure it one time. Now what’s happened is we’ve taken all the assay variability that we reduced by the square root of 240 — because we took a sample of 240, put them into solution, it’s an average of 240 for which we have an estimate. Assuming it follows a normal distribution, we have a pretty precise estimate. The standard deviation of that mean solution is sigma divided by the square root of N. But intuitively we think, let’s just be sure, so we sample that whole combined solution and measure it. We’ve now taken all the variability of the assay that we reduced by that square root of 240 and reintroduced all of it back into the assay. And that’s where intuit-
Ross Katz: Because your sample size is one. Am I thinking about that right?
Stewart Fossceco: Because your sample size is one. Exactly. It’s a very interesting thing where you can actually see it — you can simulate that process and show that we should just run with what we’ve got, as long as our other upstream process is running and in control and behaving as expected. That’s one example. Another one: let’s say we have an assay for a product that we know well, we produce and it’s very well-behaved. That assay has been worked on for years and the variability is as low as we’re really going to get it unless we want to completely change the process, which would involve additional submissions to regulatory agencies globally. It’s a lot of work and cost — cost prohibitive, in fact. Let’s say we use this assay for drug release or a component release for a diagnostic. We know that assay variability well. We know our process variability very well. We begin to examine those ratios. At some point when the assay variability begins to exceed a certain limit, if you test your material at release and make a decision, you could be making that decision based on just random noise and nothing else. You would be better off not touching it and saying, that is our best estimate given the assay variability. What you want to do then is ask: what are the limits within your specifications? If we’re outside of these limits, that’s going to be outside of what we would have expected at random based on our assay. We think we did have a shift in concentration of the product prior to release. So let’s adjust that concentration either up or down by adding more concentrate or diluting it, then go ahead, test it, make sure it’s good, and release it. It comes down to where you are in those processes and understanding of that assay variability and how it can impact your business decisions later on.
Ross Katz: There’s an intuitive nature where, if you’re of a scientific mind, you always believe that experimenting more is always better. But there’s also the timing aspect of where in the process you should be sampling so that you can get sufficient signal on the variability from the assay and on the drug that you’re bringing to market to make intelligent decisions. But if you’re experimenting at various points in the process with different sample sizes, you can get conflicting information that is challenging for decision making and also has regulatory implications — because the assays you’re using and the design of experiments you’re using are subject to regulatory scrutiny as well. Am I thinking about that right?
Stewart Fossceco: Absolutely. In that last example, you have a manufacturing process set up and in that process you lay out what the decision points are. Because you know the assay variability and the process variability, you say, at this point in the process we’ll take that measurement — and here’s a key point that’s important to understand — the assumption is that the process making that product is in control and is centered where you say it’s centered and where it’s estimated to be. When you get outside that centered part of the process, that’s where you say, we didn’t dilute enough, or maybe the concentrate was a bit more concentrated than we thought, so it’s a little too concentrated and we need to dilute a little bit more. It really becomes part of your manufacturing process. From a regulatory standpoint they will come in and say, you have a process that’s in control and you’re able to measure the variability of both your assay and your process, and you’re simply not going to fiddle with it if it’s within this space — because if you were to do the thought experiment and test again, if you’re inside those bands you’re more than likely to be closer to the mean because of the assay variability.
Ross Katz: That makes a lot of sense. You mentioned statistical simulations playing a role in both constructing the assay and understanding how the assay is performing. What are the sorts of simulations that you use in a situation like this? Is this a Monte Carlo simulation where you’re changing variables and doing random selection, or how does that work?
Stewart Fossceco: It’s an incredible tool. Because it takes a bit more time than fitting a quick regression line, it’s sometimes underutilized. But Monte Carlo simulation and simulation in general are a great way to take a process, map it out, and then at each step along the way you’ll often have data and measurements you can use to inform how much variability is being introduced at that step. You can step through it because the question is how is it transferring itself through the rest of the process — such that a small change early on could get magnified and really impact it later. That may be where you want to spend a lot of time to gain control of your assay or drive down that variability early on, since it has magnified impacts later in the process. It’s very helpful there and in so many places — used a great deal in clinical trials, as well as in stability modeling and how that impacts cold chain and supply chain decisions.
Ross Katz: If I’m understanding you correctly, measuring the variability at different points of your process is important, but also understanding the dependencies between those variabilities and which points in the entire process are most sensitive — that’s one of the insights these models tend to surface, and it helps you focus your efforts on where to concentrate on minimizing that variability.
Stewart Fossceco: Absolutely. What I’ve always found interesting is sometimes the outcome of those is counter-intuitive, because scientifically we’re trained to measure one more time before we take action. But sometimes, because you’re taking a sample of N of one, you’re actually hurting yourself — you’ve already had so many steps where you’ve taken measures and reduced the overall variability of the decision point. So it’s a very helpful tool.
Ross Katz: What I’m hearing is there’s the risk of over-measurement, so there needs to be a lot of up-front thought about where and when you take these measurements. The variability of a particular part of the process may be small and not have a lot of impact on the overall decision at the end, whereas another place counter-intuitively has variability that’s big and impactful later on in the decision-making process. What we want to do is shift resources to where we have the greatest impact in decreasing variability. That often leads us back to the early development process where we’re using experimental design to identify where those are — so we can either control it, or recognize we can’t do much with a random component. Maybe we have a supplier who has a certain amount of variability in their product and we don’t have a lever in our experimental designs to make it robust to their changes. If we don’t, we have to set up rules of engagement with our supplier where as it comes in we’ll do additional testing and they’ll have to meet certain standards before we accept it.
One last thing that came up as I was hearing you talk: we started out in terms of minimizing the variability of the assays and then using those improved assays to figure out your experimental regime that helps you understand the variability of the entire process and the bounds within which you need to be. Does any of this change when you’re monitoring the process at full production scale, or is it a replay of the same thought process?
Stewart Fossceco: The process is typically you execute the things we’ve talked about — experimental design to minimize the assay variability and establish a robust process and product. Prior to going into full-blown production you sit down and say, now we know where our variability comes from and what we can do to manage it. Now we need to validate prior to full production that we can execute on that. You’ll set up a full-blown validation with a protocol and acceptance criteria, with decision points where if our end results all fall within a certain band, we know anything within there is simply due to random noise that we’ve built into our process — and that the quality of our product reaching the patient is good. Once you pass that validation — and that’s usually a fair bit of work, the sample sizes are larger, it’s not a trivial exercise — you go into production and can have a different type of sampling plans and strategies for your in-process measurements and quality measures. Those are going to be your critical quality attributes of the product, with well-established limits, and the sampling necessary to detect when something looks odd so you can engage with a CAPA — a corrective investigation or corrective action if one is required. It could be a special cause: maybe a lever was left on during production and things didn’t work, so you put in an extra check to make sure that lever’s closed from now on. I’m oversimplifying, but I wish they were that easy most of the time.
Ross Katz: It sounds like the assays might be similar but the samples you’re taking and when you’re taking them might be slightly different — but basically the goal of that measurement regime is: is this group of product we’re producing within the expected bounds for efficacy and potency? And then when you encounter a problem, I’m assuming there’s a batch that gets thrown away and then there’s the corrective regime to go in, make sure everything looks good, and pass it all the way through the process.
Stewart Fossceco: Yep. You’ll have those assays that are used in development and characterized there. Those will be the assays used during the clinical trial process to characterize those clinical batches, because they’re used for relating the product properties important for quality back to the clinical efficacy. As those get measured they’ll inevitably be used along with other measures to release the product to the public for consumption. All of that ends up being part of a large program — drug or device stability — to ensure that throughout the shelf life of that product the patient will receive the dose or quality that is expected. You are going to use that assay for many years, and it really becomes a process of continual improvement over time to enhance and pinpoint the very best decisions for the patient in real time.
Ross Katz: That makes sense. This is a good opportunity to transition to stability modeling. We’ve been talking about assay variability and bringing these drugs to market. Could you give people who are not familiar with it an introduction to what stability modeling is and why biotech manufacturers do it?
Stewart Fossceco: I’ll start with a simple example. We have a drug product that will be consumed by the patient and the product will change over time as most do. Let’s say we’re dealing with potency of a particular treatment — when we first release it we want it 100% potent. During testing and clinical trials there are requirements for the drug to meet certain levels of potency; hypothetically let’s say 90 to 110%. The degradation rate or loss of potency over time becomes an important factor in the quality of that product and the experience and outcome for the patient. You need sufficient resistance to whatever environmental factors are going to influence that product.
Companies start very early in the development process typically modeling drug stability over time using both longer term stability programs and what are called accelerated stability programs — where the product, particularly if it’s temperature sensitive, will be held at higher temperatures and humidity levels to model the behavior over the desired lifespan of that product. You can fit various different models to that, everything from simple linear regression models over time where measurements are taken at time zero — the day it’s manufactured — to one month, three months, six, nine, 12, 18, 24 months and maybe out to 36 months for some products. Take things that are very stable like various solid dosage forms that aren’t very temperature sensitive or humidity sensitive — those can last quite a while on the shelf. Things that are refrigerated, if exposed to high temperatures — and we think about certain geographical locations around the globe, very hot environments around the equator — you have to make sure that product stays refrigerated, otherwise you can have a significant loss of potency.
That modeling process becomes a stability program for every product that comes to market. It starts very early, backing all the way up to before clinical trials — as that product is being discovered and worked on in discovery. If it looks promising, formulations scientists, chemists, and process folks will be working hard on ensuring that formulation or construction or engineering of that device is as resistant as possible, and that stability program will begin very early, looking for those optimum formulations.
Ross Katz: As we draw the conversation toward the end, what do you think are some of the biggest challenges that biotech manufacturers have in gathering the right data and implementing the right practices and processes to understand their assay variability, understand the stability of the drugs they’re bringing to market, and optimize the processes and the feedback loops that need to be in place for that information to be in the right hands?
Stewart Fossceco: Some of the biggest challenges are thinking ahead far enough about what data you’re going to need to answer the questions and in what form you need to make that accessible to the people who need to make those decisions. The more we can think ahead about where those decisions need to be made and who needs to make them, we can empower those people — hopefully very early on in the process, at the scientific bench level — to make those decisions quickly with clear statistical and quantitative assessments that give us a probability statement: how sure am I that I’m making the right decision? Because our intuitive selves look at things and think this is the way to go, but when you put your statistical thinking cap on, you sometimes need to rethink. The data capture piece still becomes a barrier. That’s one place where if organizations really think through where they’re going to make those decisions and what they need to put in the hands of people and how they need to structure that data, they can make that happen.
Ross Katz: Getting the data off the assay machines and into a place where it can be analyzed together with the right metadata about the different variables related to stability — so that the sorts of statistical analysis you’re talking about can be conducted at scale and over time. Am I thinking about that right?
Stewart Fossceco: Yes — even things as simple as the genealogy of the components that go into a diagnostic test. All the different components, the manufacturers, the suppliers of those components — because if something does arise, being able to tease that out analytically becomes very difficult if you don’t have the data and the metadata about all of those components. The other place, and I think this continues to gain steam — the philosophy of quality by design. Starting early with those experiments we’ve mentioned multiple times, looking at how all of these things are interacting with one another, because in that data and those experiments there are going to be diamonds that will save us so much pain and hassle later on. We won’t get caught late in either the development process, submission, or manufacturing process, figuring out that we now have so much variability in our process that we have to execute so much testing it becomes very onerous. Many industries have found that trying to test quality into a product is very difficult. It’s much more productive and more cost effective to design quality into the product and the process for robustness in the long term. A little more upfront cost — pay me now or pay me later.
Ross Katz: Especially for small organizations with limited resources where they have a biological hypothesis about how a disease gets cured, it’s difficult to get that type of organization to focus on the quality attributes that will need to be in place of the therapeutic that’s brought to market. Asking them to spend those extra resources during that early phase is relatively difficult. Do you have any advice for organizations like that for how to do quality by design more cost-effectively or how to bring those considerations to bear earlier on?
Stewart Fossceco: Start small. You don’t want to boil the ocean. Start small and run smaller experimental designs geared toward understanding how these things piece together. Even in the short term, what I’ve always found throughout my career is that if I add up all of the small experiments that everybody did along the way — say over the course of six months — there were gaps created simply due to the mindset of we have to move fast, we have to make this first decision today so that we can get to the one we want to make six months from now. What I’ve often found is that if we had paused — not too long, just long enough to understand our end decision, the business decision that’s really going to matter — and then looked at all the pieces in between and designed maybe two or three well-designed experiments to get that information and make that decision with a probability statement, the amount of time it took to understand the processes, design the experiments, execute them, and analyze them took fewer resources, less time, and fewer experimental runs in total than if it had been done in that very small sequential manner.
Ross Katz: Is the business decision you’re talking about something like submitting an IND, deciding to bring something into clinical trial, or deciding to scale up manufacturing? Are those the sorts of business decisions you’re talking about, or what are some examples?
Stewart Fossceco: It could be a business decision like: we need clinical material made, so here’s our decision point — what do we want going into this clinical trial material — and understanding that and working backwards. It could be an IND, but it doesn’t have to be that far along. It could be something much earlier, like looking at the behavior of a sustained release product midway into early pre-clinical testing. The pre-clinical decision point is going to hinge on a particular response. How does what we’re going to build impact the variability of how those folks are going to see that data? Once you make that link you say, here are the things we really need to address, and let’s not have the gaps we might have otherwise.
Ross Katz: Well, Stewart, thank you so much for joining us today. It was a pleasure to have you on and look forward to connecting down the line.
Stewart Fossceco: Well, thank you very much, Ross. Thank you for your time — I’ve really enjoyed the session.
Jason: And that’s it for this episode of Data in Biotech. If you enjoyed the episode, please subscribe, rate or leave a review in your podcast platform of choice. See you next time.
