Listen on
Overview
For pharmaceutical companies, leveraging advanced AI models to accelerate drug discovery hits a critical barrier: the vast majority of high-quality, diverse data needed to train these models is proprietary, sensitive, and scattered across multiple organizations. While public datasets enable foundational research, they are too limited to address the complex, real-world problems faced in industrial drug development, resulting in AI tools that are not yet impactful for critical tasks like accurate co-folding predictions. This means slower pipelines, higher R&D costs, and missed opportunities for competitive advantage.
In this episode, host Ross Katz speaks with Robin Roehm, CEO and Co-Founder of Apheris, who brings a deep background in mathematics and medicine, alongside experience grappling with sensitive genomics data. Robin explains how Apheris directly confronts this challenge by enabling companies to train sophisticated AI models on combined datasets without ever exposing raw, proprietary information. He details how their federated computing platform underpins the AI Structural Biology (AISB) Network, a collaboration of major pharma companies jointly improving co-folding models like OpenFold3.
Our conversation covers the technical architecture supporting privacy-preserving data collaboration, the specific performance limitations of current co-folding models on industrial tasks, and how the AISB Network is overcoming these through shared model training. Robin provides insight into the practicalities of data quality, security, and governance in such networks, illustrating how companies like AbbVie, Johnson & Johnson, and Bristol Myers Squibb are leveraging this approach to push the boundaries of in silico drug discovery.
Key Takeaways
Federated computing provides a practical path to overcome critical data scarcity for AI in drug discovery.
Public datasets like the Protein Data Bank (PDB) contain insufficient diversity for training advanced co-folding models relevant to industrial R&D, leading to significant performance drops when applying models to proprietary targets (e.g., accuracy falling from 90% to 30-40%). Federated learning allows multiple pharmaceutical companies to contribute thousands of unique protein-ligand structures to a shared model training effort without sharing raw data, enabling more reliable models for novel chemistry.
Multi-party data collaboration for sensitive research relies on stringent IP and privacy controls, not just data sharing agreements.
Pharma companies require absolute assurance that proprietary research data remains secure. A federated gateway architecture ensures data stays within each company’s IT environment, with access controls and security vetting on all computations. Only model weights – assessed not to leak underlying training data – are shared, establishing the trust needed for competitors to collectively advance pre-competitive modeling capabilities.
Improved co-folding models significantly reduce the cost and time of drug lead optimization and virtual screening.
Current physics-based methods for predicting binding affinity are computationally intensive, consuming half a day per structure and limiting in silico screening throughput. By training more accurate foundational co-folding models on diverse, private data, the reliability of subsequent binding affinity predictions improves. This enables faster, cheaper virtual screening and lead optimization across a broader chemical space, accelerating drug development pipelines.
Federated networks can guide strategic data generation, optimizing R&D spend by addressing specific model deficiencies.
Even combined proprietary datasets may not fully cover the entire chemical and biological problem space. Intelligent benchmarking, diversity assessments, and confidence evaluations within a federated network identify specific data gaps where a model performs poorly. This insight directs targeted generation of new data by CROs or internal labs, systematically improving model performance for all network participants and making R&D investments more efficient.
Related: CorrDyn helps biotech and life sciences companies work through complex data challenges. Our work includes defining AI strategy and building solid data engineering foundations. We also focus on ensuring data quality for sensitive data environments.
Full Transcript
Jason: Hey everyone, this is Jason, producer of Data in Biotech. I’m excited to announce that this podcast is now an AAPS media partner. Join us on-site at PharmSci 360, November 9th through to November 12th in vibrant San Antonio, Texas. We’ll be interviewing leading scientists about the data trends that span the pharmaceutical development and manufacturing spectrum. Data in Biotech thanks AAPS for this opportunity to share emerging science with the global pharmaceutical community.
Robin Roehm: I myself and many people in the space are not thinking co-folding models will replace the physics-based methods. Ultimately drug discovery is always a pipeline where you start broad and then you de-risk as part of the funnel. These co-folding models, they are breakthrough, they are really impressive, but today they don’t yet have the impact on industrial research because their applicability for these industrial tasks is just not there yet.
Jason: Welcome to Data in Biotech, a podcast from CorrDyn where we explore how companies leverage data to drive innovation in life sciences. Every two weeks we sit down with an expert from the world of biotechnology to understand how they’re using data science to solve technical challenges, streamline operations, and further innovation in their field. Here we go.
Ross Katz: Robin Roehm, welcome to the Data in Biotech podcast.
Robin Roehm: Well, thanks for having me.
Ross Katz: Awesome. Well, just to kick us off, would you mind giving us an introduction to your background and what brought you here today?
Robin Roehm: Sure. I have a background in mathematics and medicine, and then I had a previous startup before Apheris, in the genomics field, where we did essentially machine learning pipelines for secondary analysis. Working with genomics data, that’s very sensitive data, you always have that challenge of centralizing that data or bringing it to a point where you can analyze it. After going through that pain over and over again of not being able to move genomics data, we figured there must be a better way to use that data without moving it and that was the founding story to Apheris.
Ross Katz: Awesome. Will you give us an introduction to Apheris and how you put that insight into practice?
Robin Roehm: Essentially what you see in the industry when you look at how AI is coming to market is that you need very large and diverse data sets to train rich and powerful foundational models. For most of the industry problems, that data is not out in the public domain. For text and maybe simple images, there is a vast amount of data just in the internet, but for sensitive data like health, life sciences data, molecular data, that data is missing. When we figured out how to essentially bring more diverse data that are owned by multiple organizations together, we decided to found Apheris with that mission, essentially help companies to unite their data sets without sharing it, utilizing a key technology that we pioneered which is called federated computing.
Ross Katz: Awesome. We’ve talked a little bit on this podcast about federated computing previously, but for those who didn’t get a chance to listen to it previously, would you just introduce us to the concept of federated computing, maybe a little bit about how it works and then how Apheris has operationalized those ideas?
Robin Roehm: Essentially the old paradigm when you are applying AI to data is move data to a central location and then do all your modeling. That’s the status quo still in many worlds. Now there’s a better paradigm to work with data that is owned by multiple parties, that is instead of moving data to a central location, keep the data within the environments of the data custodians and bring models to that distributed data. The key logic is take a model, train small individualized models on each of the data custodians’ own proprietary data and then somehow aggregate these individual models to a global model. Reality is you need to iterate over that process, so it really requires an infrastructure to make that work but that’s the key concept. When we started working in that field, the first couple of years were really focused on making data owners, data custodians comfortable with that way of working. As you can imagine, the key pillars here are governance, privacy, IP protection and security. You’re dealing with sensitive data, you’re sending models around, so those have been the key pillars that we operationalized and productized over the years to make sure pharma companies can come together and collaborate.
Ross Katz: Awesome. One of the reasons we’re here today is to talk about a specific collaboration that you have ongoing with regard to OpenFold 3, and would you just give us a little bit of a history of that collaboration, how it came to be and then an introduction to OpenFold 3 and what makes it useful and valuable?
Robin Roehm: First of all maybe just introducing this use case, the field of co-folding. What does co-folding mean? Essentially in drug discovery, what you are always interested in is the structure of the molecules that are interacting with each other. The first real breakthrough here was AlphaFold 2 from DeepMind and Isomorphic. Before ChatGPT, that was considered as the biggest AI breakthrough in history. It solved this problem of can I predict the structure of a protein based on sequence? It turns out, yes, you can as well with high confidence. Now if you look at drug discovery, the reality is they don’t just need the structure of a protein, that’s typically the target by which you search a molecule to interact with. You want to develop a molecule, a drug that has a certain interaction with a target you are interested in to have a certain biological effect. This whole field is called co-folding. The AI Structural Biology Network, which is hosted on our product and facilitated by us, that’s a network of large pharma companies where pharma companies come together, jointly utilize their proprietary data to train richer and more powerful co-folding models.
Ross Katz: Obviously if pharmaceutical companies are exploring bringing their data into this federated computing environment that you have at Apheris, there’s some value to making co-folding work. If you could just orient us to what the value is to making co-folding work and then why bringing these data assets together is going to help these companies to capture that value.
Robin Roehm: Maybe let’s start with why co-folding is so hard to model if you think about it. Modeling structure prediction as AlphaFold 2 does is already a very hard problem. The benefit there is you have a lot of open data. The protein data bank PDB is an open data repository with 240,000 structures. A lot of model architecture innovation enabled DeepMind Isomorphic to solve that problem. Now if you take it to co-folding, two things change. Number one, the sheer number of combinations that you can model in co-folding where you not just predict the structure of a protein but as well how it interacts with other molecules, small molecules, large molecules, antibody-antigen interaction. Just that sheer number is orders of magnitude larger. You just have a much larger problem you’re trying to capture in a given model. The second and I think more important problem here is that there is not a rich enough data source in the public domain to meaningfully capture that phenomenon we are trying to solve. If you think about it, the PDB, 240,000 structures, but the number of protein-ligand interactions where the ligand has drug-like properties, where it could be a drug, that is just 2% of that data repository if you take out all the small molecules that are not interested like sugars. Suddenly you just end up with a few thousands of meaningful protein-ligand interactions in the public domain. Now if you go to pharma companies, they have this data in their private data repositories, that’s the key research that they did over the last couple of decades. However, as you can imagine, this data is incredibly sensitive, so opening it up or directly sharing it is not an option. That’s where the AISB Network, the AI Structural Biology Network comes into play. We connect that data in an IP-preserving manner, hence get access to richer and more diverse data to capture that phenomenon we are trying to solve, which is co-folding.
Ross Katz: That’s great. Before we move on to the structure of these models and how federated learning for these models might work, could you just help us understand the relationship between AlphaFold 3 and OpenFold and OpenFold 3 and how they fit together or separately?
Robin Roehm: Sure. Let’s start with AlphaFold 2 again. AlphaFold 2 is the DeepMind Isomorphic model, and then there was OpenFold as the open source reproduction of that, and that again is for structure prediction. Then the next set of models here, the first one was AlphaFold 3, which again is from DeepMind and Isomorphic, which addressed that co-folding problem. The challenge with AlphaFold 3 is that it’s not commercially available and open source available. There is an AlphaFold server, which is essentially AlphaFold 3 and you can query it, but only for academic purposes, for research purposes and as well it’s hosted by DeepMind so you need to share the query with a model that is hosted by them. For industrial purposes, this model can’t be used by pharma companies and internally fine-tuned to their data. That is why the OpenFold consortium, which is a consortium where we are part of and many other pharma companies are part of, they fund Dr. AlQuraishi’s team from Columbia University to essentially build a complement of AlphaFold 3 fully open with all the engineering such that you can fine-tune it, such that you can take it locally hosted. Dr. AlQuraishi is an amazing researcher and so that model is available to the parties of the OpenFold consortium right now and it’s getting fully open source released in a matter of weeks, so maybe by that time this podcast goes live, the model is out and yes stay tuned, impressive performance benchmarks.
Ross Katz: That’s great. One of the things I’m interested in understanding, since we have you on the podcast is what were some of the structural like model architecture or modeling insights that gave us the leap from AlphaFold 2 to AlphaFold 3 or OpenFold 2 to OpenFold 3 and I think that’ll lead nicely into how the different data sources are being used inside of the modeling context.
Robin Roehm: Sure. We could go fully into the weeds here and speak about that, so I’ll keep it at a certain level such that everyone can understand. In a simple form this model has an input derived from the sequence information, the protein sequence and then there’s this big transformer model that essentially uses that. Then in the end, and I think that’s the key aspect here of the co-folding models, there’s this diffusion model right now rather simple. What this whole pipeline essentially does, it uses a lot of additional information that you find in templates or MSAs, so MSAs are multiple sequence alignments and that is essentially capturing evolutionary information that are sitting in other public databases. What you are essentially trying to do is if you have a given protein sequence can I get additional information about that protein family from that evolutionary information that helps me predict the structure of that given protein. That is all coupled in a pretty long data pipeline that this model then uses.
Ross Katz: My understanding is that one of the things that allows this to be possible is the structure is much more flexible in AlphaFold 3 and it scales much better with data. It’s relying on fewer assumptions in the architecture itself. As new data is added, the diffusion model adds several different aspects to the modeling outputs that make them really useful. One is that it’s giving you multiple outputs per sequence so that you can have multiple different options to choose from and it’s also giving you this uncertainty quantification about the output that’s allowing you to prioritize the outputs and maybe utilize them more effectively. Am I understanding that correctly?
Robin Roehm: Yeah, in particular this distillation approach I think is truly interesting because as you say what you’re doing there is during the learning you’re generating more examples and then taking these examples that the model then prioritizes as part of their learning process. If you think about it, that’s as well the area where just the rich pharma data repositories can generate a huge amount of wealth. Of course you can add more structural data but we might come to that as well, how can you add further data modalities to guide that distillation data process. Things like binding data, where you don’t have the full structure, you can use to essentially guide what data during the distillation data process is generated. This binding data is of course as well available in these pharma data repositories, which is very rich, very sensitive, but truly valuable for these type of model architectures.
Ross Katz: This is one of the things that I was curious about as I look at the model architecture. My understanding is that as part of the multiple sequence alignment process at the beginning of the model, there’s this querying of the databases that produces the relevant example sequences that are going to help to condition the model to utilize the most relevant information from evolutionary history to produce the folding output that is most accurate based on the information that we have today. On some level I understand that in federated learning we’re all sharing the weights. There’s this latent space, these embeddings that get created that are shared across these different data sources, but in terms of the OpenFold 3 model it also seems like there’s this initial conditioning process that is more like querying the raw examples before they get into the model. I’m curious, how do you deal with that in a federated learning context that is privacy protecting for the organizations that are sharing the sequences but also producing the best possible outputs for the community that are then consuming the sequences?
Robin Roehm: That’s a great question and by the way this multiple sequence alignment you do not just do for the training but as well for inference, as you query the model you still need an MSA server to be connected. The question is, absolutely right, how do you deal with that in a federated setup both during training but as well during inference as you use that model, what do you do as a company? The reality is pharma companies have these MSA servers in-house. But this is large data and there’s often public data but pharma companies as well have those enriched with their own templates and their private ones. If you think about this federated computing architecture but as well the backbone that you need to use these models downstream, you see how much technological innovation is really needed here because we are not dealing here with a simple smaller model that is being trained, we are dealing with a full pipeline with querying of the MSA server and the rigor we’ve put into setting this up such that during training but as well when you run these models on scale no data, no private data or queries ever leave your local environments has been tremendous. There have been key innovations as well on the federated computing side that we did where we worked with Nvidia. This is to our knowledge one of the most complex and largest models that have ever been used in such a setting.
Ross Katz: On the training side, how does the data enter the pipeline and then what do you get in return, and then I’ll be interested in the inference side as well.
Robin Roehm: Simplified the steps to get such a setup up and running. We have a software component called a Gateway that a pharma company can deploy within their own IT environments. They deploy this, of course you can imagine there’s a lot of security validation that needs to happen before that happens but once it is deployed what a pharma company can do is register data. Register data just means you tell the Gateway that data exists, it points to the storage location, so no movement of data, just like if you are in AWS, on that S3 bucket data that I want to use in a federated setup is sitting. The next thing pharma companies need to do and that’s very important is set up access controls on a computational level. What does that mean? If you have a Gateway you point data that you want to use in a federated network, for pharma companies of course the question is what are you using it for? What computations will you run and how do you ensure these computations protect the privacy and IP of my data? This is something they can set up, they stay in the control, they say this model that has been vetted with these security assessments and these privacy assessments, this model can run in these constraints. That is something the Gateway then enforces. If anyone tries to launch a computation that does not fulfill these settings, the computation is blocked. Those are the settings the custodian does, so each of the pharma companies do, and then once that is set up every pharma company has their data in their own environments, the same settings, then the model provider in this case Dr. AlQuraishi that we collaborate with with the OpenFold 3 model, they are taking their model and sending it to the Gateway. The Gateway assesses is this the right model, is it as well the right configuration, is it security vetted and if yes they deploy it and run it and then all the data pipelines and all of that needs to happen. Everything we discussed before then happens via the Gateway and then these local models are aggregated over that process, it’s iterated until the federated model gets generated.
Ross Katz: If I’m understanding correctly then, to answer my question from earlier, you’re only accessing your own MSA server, your own multiple sequence alignment server when you’re working with the model or when you’re training the model and then what gets shared back centrally is the changes to the weights of the model that can then be used by other organizations. You’re responsible for the data that conditions the training and that later conditions the inference, but what’s shared is really the centralized weights, the embeddings, that sort of thing in the model. Am I thinking about that right?
Robin Roehm: That’s absolutely right. Only the weights and only weights that are essentially assessed to not leak the underlying training data. Only those are shared. Of course there’s a lot of collaboration in such a network from a scientific perspective and alignment of how do we want to use these models, how do they get packaged, how do we do benchmarking, all of these things are of course aligned, but from a technical perspective you are absolutely right, only the weights are ultimately shared between the parties.
Ross Katz: That’s really interesting. Sorry before we move on ’cause I think it’s just really relevant, which pharmaceutical companies do you have working with you to do this federated learning right now?
Robin Roehm: The AISB Network is a large pharma-to-pharma network with eight pharma companies involved. This specific project where we are training co-folding models in the small molecule field is with five pharma companies, that’s AbbVie, J&J, Bristol Myers Squibb, Takeda, and Astellas Pharmaceuticals. All of them are contributing thousands of unique protein-ligand structures, contributing here to be really clear in a privacy-preserving manner, not sharing any data.
Ross Katz: That’s really interesting. Obviously there’s a lot of data moving in and it’s also passed the legal, regulatory and IP protecting hurdles of each of those organizations independently, so that’s a strong vote of confidence in the federated learning framework that you’re bringing to bear. One of the things that’s interesting to me is you mentioned the limitations of the protein data bank that was used to train OpenFold 3, but I’d love to understand more about the limitations of the model outputs that result from the model without this federated training component and what some of the limitations we expect to overcome by bringing in these other pharma companies’ data sets into it.
Robin Roehm: That’s a great question and a lot of the industry ecosystem is already looking at the limitations of these models based on benchmarks that have been built to visualize that in an intuitive manner. Maybe I can just quickly speak about standard benchmarking technique here to assess these models’ qualities. There’s the Runs N’ Poses benchmark and essentially in a simplified manner what it does is it is structures in different bins, so you have different smaller validation sets. These different bins or buckets, they have a different distance to the training data, which is the PDB. In a nutshell, if you see that, you see on the left the data that is much further away from the training data from a similarity perspective and then on the very right, you see the data that is very similar to what is captured in the PDB. When you look at all these co-folding models, AlphaFold 3 but as well the other open source equivalents or some private models that have been built, all co-folding models, you see the same trend and similar performance benchmarks, which is in the data that is similar to the PDB, these models perform quite high, often 90% of the structures they predict in the correct manner, but the further you go away, the less performant they become. The performance drops significantly to something like 30 to 40%. Now if you think about what pharma companies will be using this for, they will be applying these models to their own proprietary research programs with specific chemistry, with maybe targets that are not represented in the public domain or at least with chemistry that is unique, and so you will have a lot of problems that are much more within these bins that are the harder ones. You will have few problems where pharma companies work on aspects that are really well represented in the PDB. That’s a bit what the industry generally sees, these co-folding models they are breakthrough, they are really impressive, but today they don’t yet have the impact on industrial research because their applicability for these industrial tasks is just not there yet.
Ross Katz: That makes sense. As I was researching this, some of the things that came out as major limitations where there was insufficient data in PDB was any protein where there’s large conformational changes, so when they’re moving around a lot, there isn’t enough data to characterize them. Or GPCR, it’s biased toward active states, it doesn’t have inactive or partially active states in there. When there is three component systems, like a ternary complex of tripartite interactions. It’s not working. I don’t know if there’s anything else that comes to mind for you, but those were some of the things that came out.
Robin Roehm: No, I think those are some of the key challenges. There’s aspects like stereochemistry that is still a problem and to be clear here, proprietary data that is rich and diverse is certainly a key pillar to respond to that, but of course there’s as well model architectural innovation that can help with that. I think a key aspect with these foundational models is that if you don’t build the foundation right, all the downstream applications as well don’t become really well. I think the binding affinity is an interesting example of that. Binding affinity’s a huge problem, and for lead optimization or virtual screening you want the ability to predict binding affinity early in the process. The key idea here is if you predict the structure of a protein-ligand interaction really well can you not just use that as an input for a binding affinity head that then predicts you the binding affinity? Turns out, yes, that is something that you can build and it works reasonably well if the structure prediction is correct. That’s why you can post great academic papers that show exactly this, but the reality is, of course, if the structure prediction is not correct, why would this binding affinity head as a downstream application be correct? It wouldn’t make any sense and of course that’s as well what we see in the data. That’s why you need to invest in that foundation really well, you need to get a good high quality structure prediction across the different data modalities you’re looking at and the diverse chemistry such that all these downstream applications for lead optimization, virtual screening become as well useful for industrial applications.
Ross Katz: That makes a lot of sense. One of the things I was reading was that predicting binding affinity is something that you should do in order to evaluate the results that come out of a model like OpenFold 3. You’ll use these very high computation physics-based models to do that kind of prediction, but there’s this hope that if you bring all of these diverse data sets together and create that better foundation that you’re describing and more accurately characterize the structure of these proteins, then the binding affinity can be predicted accurately with much lower computational overhead which allows for much better in silico creation and evaluation of these different proteins. Am I thinking about that right?
Robin Roehm: Absolutely, that’s the hope. To be clear, I myself and many people in this space are not thinking co-folding models will replace the physics-based methods. Ultimately drug discovery is always a pipeline where you start broad and then you de-risk as part of the funnel. Of course, as well, data generation with labs to confirm what you are seeing is a key pillar around that. What you’re trying to do is accelerate and lower the costs and increase the timelines of how fast can you iterate over that space. If you have a method that with much higher likelihood and much shorter timelines as a co-folding model that predicts structure if you can use that earlier, you can use it for things like lead optimization or virtual screening use cases where you can go over much broader space of chemistry compared to a physics-based method where you would use a day to compute such a structure. The number of examples you can just compute is just much slower compared to a co-folding model if it works correctly of course.
Ross Katz: That makes a lot of sense. As the different partners are bringing their data sets to Apheris and to federated learning for OpenFold 3, are you or are they working together to apply automated data quality or curation approaches to the data to make sure that collectively they’re all lifting the quality of the model versus one partner maybe has different data quality practices than another and it leads to a worse model for everyone else?
Robin Roehm: I think fair data contributions and high quality data contributions are the biggest reasons why federated networks succeed or fail. Yes there’s a lot of rigor exactly around this aspect. The good thing with structural data compared to some other data modalities in the molecular field is that it is reasonably well structured. Still the reality of that is there’s a lot of nuances of how you can label that data, so there still needs to be a lot of rigor to that. Yes we have essentially controls, both guidelines and controls, it’s about enablement but as well about qualifying that before data is fed into a given model you check the quality, but at the same time you as well want to provide the scripts and functionality to take data that pharma companies today have to map it to what you need. All of that is packaged essentially in the products that we’ve built, which goes beyond a federated computing infrastructure. This is additionally scripts that take structural data and transform it and then validate it and then feed it into the model, so that’s a lot of extra product functionality that we’ve built to enable such use cases.
Ross Katz: You mentioned the benchmarking approaches and how AlphaFold 3 and OpenFold 3 are evaluated. Do you have benchmarks or metrics that are showing the contribution that this federated approach is making toward making OpenFold 3 a better model?
Robin Roehm: Yes, but I can’t speak about the numbers here of course. There’s a lot of people with stake that want to make sure this gets into the right publication, so of course people will write about this. I can’t speak about the details here. But of course all of that will be published. Maybe what I can speak about is the rigorous benchmarking you want to do here. Benchmarking in a federated manner is a true complexity because you can’t see the data and splitting the data such you can assess aspects like generalizability is really hard. What we’ve done here is essentially making sure the Runs N’ Poses difficulty is retained. You take pharma data and you ensure that the training data you feed in does not leak any information that you have in the Runs N’ Poses data set such that if you evaluate it on Runs N’ Poses, you can have that true comparison. That’s one benchmarking approach. There’s another one to just hold out private data to see how good does it work on programs the data has never seen. But both of those are used in the benchmarking strategy.
Ross Katz: That makes a lot of sense. Is there a thought that in the future the federated fine-tuned version of this model will be made open source or publicly available, or how is the AISB thinking about that?
Robin Roehm: The AISB Network is structured such that there are additional incentives for further parties to join that network and one of the key incentives is the give and get of such a network. The give is you provide access to proprietary data that you own and the get is you get access to these models that have been trained on the richer and more diverse data and can then use them downstream for your own drug research programs. There is this ability to join the AISB Network if you bring data and in order to keep that up, of course you don’t want to have these models openly available. Next to that, I do want to say that there’s of course as well a very rigorous security and IP protection posture and that is one that has a lot of technical safeguards but as well legal and organizational ones. The threat model changes of course as well the more open you go and that’s another reason why to date these models are only accessible to the parties that participated in this network.
Ross Katz: That makes sense. If there’s an organization out there that’s interested in joining the network, can you give some idea of what they need to bring to the table and how they go about joining?
Robin Roehm: Sure. They need to bring structural data and potentially in the future as we launch new initiatives, binding data to the table, proprietary data. Data contributions may differ, there are models that you can think of that as well allow smaller companies to join. Right now this is mostly for the largest pharma companies but indeed you can think of network setups where you can allow smaller pharma companies to participate and have a fair data exchange. As you can imagine with such large names as these eight very large pharma companies, there’s a lot of governance as well around it, and the key process is you reach out to us, we help you get into the network and the network has an overarching contract that allows you to be part of it, and then based on that the data contribution requirements and the give and get is defined.
Ross Katz: Awesome. When we were talking earlier, you talked about the adoption of these models that are trained using federated computing approaches. Maybe it’s not just as simple as computational chemists and drug discovery teams picking up this federated trained model off the shelf and then using it in their day-to-day work. I’m interested in what have you seen in terms of adoption from pharma companies that want to use these models, what are some of the barriers and if there’s anything that you’ve seen that has been best practice would love to know that too.
Robin Roehm: To be clear, I think that’s one of the biggest learnings that we made over the years of supporting this, if you get to a federated model and now you have that federated model and you can even benchmark it and show that it’s superior, that is still a big gap for bringing it to users who are then using it in real world drug discovery programs. These co-folding models, who are they for? Medicinal chemists and computational chemists. These are slightly different user groups and in a large pharma context typically these user groups work together on drug research programs. What we’ve developed in the course of that is essentially a product, we called it Apheris Fold, and it’s accessible as well to parties that are not part of these federated networks, then you don’t get the federated models but just the open ones that are trained on public data. What it essentially allows you to do as a pharma company is take these models, locally deploy them, have an interface for both computational chemists and medicinal chemists jointly in one interface. Then start with benchmarking and assessing the applicability domain on your private data. Benchmarking, that’s a more rigorous approach where you need to build the right benchmarking data set and do that in a proper manner. But sometimes a medicinal chemist as well wants to assess the applicability by just looking at a structure in a visualizer. Both is available in that tool. What happens is typically these parties start with benchmarking and assessing applicability, but then quickly move to the question of how can I now customize that and improve that for my specific drug research program? That can have various forms, that can be a fine-tuning workflow, that can be additional templates where you tell the model predict something in that area, that can be just scaling it out. Instead of running single predictions run thousands and prioritize that and use the output to filter and do that. All of that is available in a tool that we provide and again that is a very easy way to start for smaller companies that want to get in the game of co-folding, so that’s something that we build for the broader community.
Ross Katz: Interesting. Another thing that we talked about previously was this idea that once you start this process of federated learning for improving or fine-tuning a model like OpenFold 3, you can start to assess where the gaps are in the data set that’s feeding that model and there might be a role for data generators like CROs and high throughput labs to play in strengthening the model. I’m interested in hearing from you how do you assess those gaps and what is the role that you see for these data generators in improving the models in the future?
Robin Roehm: First of all quickly explaining the key idea here. Generally if you have a federated network, you have a set of parties and these set of parties of course bring much richer and more diverse data than typically what any individual company or the public domain has. That’s a huge step forward. However, to really solve a problem like co-folding or different problems, ADME predictions, antibody developability, you name it, these hard industry problems, of course very often even the data in a federated network is not comprehensive and captures the whole space in a meaningful manner. What you can essentially do is based on the model’s behaviors, based on intelligent benchmarking, diversity assessments, confidence assessments, you get a signal of where is the model already strong and where do you have gaps in that existing data network that if you could complement that you would meaningfully increase the value for all network participants. That’s something that you can do, it follows the logic of what you typically do in an active learning program, where you use essentially a model to guide the next data that you generate, that’s something similar from a technology perspective what you can do here. You can work with a data generator that then fills these gaps in a systematic manner. We are pioneering that in a different network setup together with Ginkgo, Ginkgo Data Points, Ginkgo Bioworks is the mother, where it’s the field of antibody developability. This is a different model category but that’s for the first time where historic data and then new data that is generated by Ginkgo in a high throughput manner is used to address antibody developability. I think that combination is really powerful because it addresses exactly the problem of bring parties together to use what they historically have, but as well be intelligent and thoughtful of what new data to generate in a cost-effective manner. That’s what I’m really excited about and what I expect to see much more in the networks that we power and generally the industry networks that are out there.
Ross Katz: That makes me think that the opportunities for federated learning in the biotech and life sciences and healthcare space is unlimited. A lot of these problems have similar structures where the most important data is housed inside of these private organizations that would all benefit from modeling together but also are very protective of their IP. I’m interested in whether you see a future where these federated networks can span not just pharma, but also regulators or biotech startups or hospital systems or all the different groups that could contribute data to an effort like that.
Robin Roehm: Totally. I think there are these large important industry problems. Co-folding is one of them, but there’s of course as well others in the early stage drug discovery stages, but of course as you move down towards clinical data, predicting clinical success, in vivo PK prediction, new data modalities, there’s the whole field of Omics and single-cell data that is used. I think all of them follow a similar pattern, the data is generated by different parties, it’s sensitive. If you move to clinical data, it’s as well from a regulatory perspective and a patient privacy perspective truly truly sensitive. For all of those use cases what is true is there is a lot of existing data, but even if you could combine all of that existing data, the problem that you’re modeling in biology is still so complex that you need additional data to generate and that data generation is expensive. If you could lower the costs for a set of partners to generate that data that would be truly valuable. I think the key answer to many of these problems will be federated networks coupled to targeted data generation to lower the cost for each individual participant and have joint ambitious projects. If you look at our portfolio, we have a couple of these networks that we host ourselves, ADME, antibody developability, co-folding, but there are couple of more that we will be hosting. But as well our customers are partly building their own networks. For example pharma-to-site networks is a key thing that our customers build and we just provide the technology, but then they select the set of sites that they build a network to do clinical research, patient demographics and things like that. I think a lot of new innovation will happen in exactly that field.
Ross Katz: Very interesting. Going back to the co-folding problem that we spent most of the episode on, do you have a sense of how close or how far away we are from co-folding models that are as impactful for co-folding as AlphaFold 2 was for single protein structure prediction? How close we are to having co-folding models that are really driving usable binding affinity calculations that can be used downstream in filtering this stuff down?
Robin Roehm: We call that the AlphaFold 2 moment for co-folding. I think that would be truly impactful for the industry. How close are we? I think the key ingredients are there, but you need to bring them together and of course there’s still scientific innovation that needs to happen here. Key pillars: industrial data at scale. That’s a key pillar and I think with the AISB network we are doing exactly this. The second key thing will be architectural innovation. I’m truly excited about the OpenFold ecosystem because they have now built this model, it’s open source, it’s soon open source available and it has this modular engineering pipeline. What you will see is many parties will be building upon that and so that I think will allow for a lot of academic groups to take this further, Dr. AlQuraishi will be a key thought leader here, but as well potentially companies packaging this and applying it to different aspects. Architectural innovation is going to be key. The last point that I think is critical is targeted data generation. There are couple of initiatives out there. I do think that we will see both large initiatives generating more public data, but as well more focused targeted data generation that pharma companies will do. They have a certain area that they are interested in and today they are already generating data for that. How can you generate that data such that you can use it to fine-tune these models as you do your drug research? That’s I think a truly interesting field where we are playing with our applications to make sure the moment new data comes in, parties can fine-tune and customize that to their drug research programs to get the most out of it for their drug research program. All of that combined, but I think it will take some more time to really get this together.
Ross Katz: It sounds like we’re closer than we’ve ever been but still have a lot of work to do. We’ve touched on a lot of aspects of federated computing, of co-folding, of how these networks are formed. If you had one takeaway that you wanted the audience to take from this conversation, what would it be?
Robin Roehm: I would say co-folding is coming. Even if it’s not perfect today for your specific research domain, it is coming and if something like that is coming you don’t want to run away from it, you want to run towards it. That’s the same with AI, AI won’t solve all your operational problems, but you certainly want to embrace it, you want to get used to it, you want to set it up, you want to test it. I think the value here is you can start small. Not every company needs to be part of these large networks from the start, whether it’s a large or small company can deploy these models locally, they are openly available. I think it’s really important that pharma companies, as well smaller pharma companies, don’t forget the security angle here, so deploy them locally, use them, and if you have any challenges around that, reach out to us, we are happy to help and make that as streamlined and easy as possible for you.
Ross Katz: Awesome. Where can people go to learn more about Apheris and the AISB network?
Robin Roehm: apheris.com, there’s a large tab on the AISB networks and some other networks that we host on the website so you can directly go there. As well, of course, LinkedIn, Robin Roehm, you can reach out to me if you’re interested. Those are probably the best entry points.
Ross Katz: Fantastic. Well Robin, it’s been great to have you on the podcast, really appreciate the time and look forward to connecting down the line.
Robin Roehm: Awesome. Thank you for having me.
Jason: And that’s it for this episode of Data in Biotech. If you enjoyed the episode, please subscribe, rate, or leave a review in your podcast platform of choice. See you next time.
