Listen on
Overview
A generative model can propose a molecule that looks compelling on screen and is impossible to make in a lab. That gap — between a structure that scores well and a compound a chemist can actually synthesize, test, and push toward the clinic — is where most of the hard work in drug discovery still lives, and it is the gap most AI drug discovery narratives skip. Paul Finn’s argument is that synthesizability belongs at the front of the process as a first-class constraint, not bolted on at the end, because changing a single atom can take a molecule from trivially easy to make to practically impossible.
Paul Finn is Chief Scientific Officer at Oxford Drug Design, a computational drug discovery company with roots in Oxford’s chemistry department. His career spans more than 40 years of computational drug discovery — from early structure-activity modeling in the 1980s through modern generative methods, with deep experience at what became GSK and Pfizer before the Oxford spinout ecosystem — and he has shepherded a compound all the way to a marketed drug. That track record gives him a particular skepticism toward AI results that look too good to be true, because he has done the work of checking whether they are.
In this episode, host Ross Katz and Paul work through synthesizability as a design constraint, why chemistry has proven so much harder for AI than biology, how a 3D view of molecules gets closer to the physics that governs binding than the 2D strings most models learn from, how Oxford Drug Design runs a Bayesian optimization loop over reagent space across its oncology and antibacterial-resistance programs, and what a pharma buyer should actually ask an AI drug discovery vendor before trusting the pitch.
Key Takeaways
Synthesizability is a first-class constraint, not an afterthought
The most underappreciated problem in AI-generated chemistry is that the molecule has to be made. Paul makes the point that a model can search an almost incomprehensibly large chemical space, surface something with the right predicted properties, and then hand a chemist a structure that cannot be synthesized — all that compute spent on a dead end. Worse, the difficulty is brittle: changing a single atom can move a compound from straightforward to synthesize to practically impossible. Treating synthesizability as something to check at the end, rather than a constraint that shapes the search from the start, wastes the search.
Constrain the search to real reactions and buildable molecules — without losing novelty
Oxford Drug Design’s answer is to confine the generative search to actual reaction schemes and purchasable building blocks, so that everything the system proposes is makeable by construction. The intuitive worry is that constraining the space this way sacrifices novelty. It does not: even restricted to real chemistry, the accessible space is so vast that the overwhelming majority of it has never been made by anyone. The constraint buys synthesizability essentially for free, because the remaining space is still far larger than anything a program could exhaust.
Most generative models learn from 2D strings, two steps from the physics that matter
A large share of generative chemistry models learn from a 2D string representation of a molecule — a notation that is two steps removed from the 3D structure that actually governs how a drug binds to its target. Binding is physics: shape, charge, and how a molecule sits in a pocket. Learning from a flattened string throws away the information that determines whether a compound works. Paul’s view is that getting closer to a 3D, physics-aware representation is one of the levers that separates models that look impressive from models that are actually useful.
Search reagent space, not molecular space
Rather than searching molecular space directly, Oxford Drug Design searches reagent space — which combinations of building blocks and reaction schemes are associated with the highest biological activity. A Gaussian process model sits at the center of a Bayesian optimization loop: as experimental data returns from the assay cascade, the model focuses on the regions of chemistry where activity appears to concentrate, guiding the next round of candidates to make and test. Each cycle feeds more data back, and because the search is defined over real reagents, the synthesizability constraint is built in rather than enforced after the fact.
Benchmark complex models against simple ones, or you will fool yourself
The discipline the field most often skips is comparing a sophisticated model against a simple baseline. High correlations on a validation set can come from biased data preparation rather than from the model learning anything real, and it is the improvement over the simple baseline — not the absolute score — that tells you whether the extra complexity is earning its keep. Paul and his co-authors were able to recover the Halicin antibiotic result using methods decades older than deep learning, a pointed demonstration that an impressive-looking result does not, by itself, prove the sophisticated method was necessary.
Ask what the AI actually contributed — it is often step one of a hundred
When a paper announces that an AI discovered a drug candidate, the AI was frequently step one of a process with ninety-nine more steps, nearly all of them requiring human judgment, and the published literature makes it genuinely hard to untangle what the model contributed. For a pharma buyer, that is the question to press a vendor on: what did the AI actually do, can you explain why it suggested what it did, and does a classical method agree that the suggestion makes physical sense? A generative model can hallucinate a molecule as easily as a chatbot can hallucinate a citation; classical computational chemistry is the sanity check the black box cannot provide.
Related: CorrDyn partners with biotech, healthcare, and life sciences organizations on machine learning, AI strategy, and the evaluation discipline — benchmarking, validation, and honest uncertainty — that determines whether a model is genuinely useful or just looks the part.
Full Transcript
Jason: Welcome to Data in Biotech, a podcast from CorrDyn where we explore how companies leverage data to drive innovation in life sciences. Every two weeks, we sit down with an expert from the world of biotechnology to understand how they’re using data science to solve technical challenges, streamline operations, and further innovation in their business. Here we go.
Ross Katz: Welcome back to Data in Biotech. I’m Ross Katz, data science lead at CorrDyn. There’s a story that the AI drug discovery field likes to tell about itself, where generative models search vast chemical space, identify novel compounds, hand medicinal chemists a short list of candidates ready for synthesis and testing. It’s a compelling story, and in some respects, it’s true. But spend time with a computational chemist who’s been doing this since 1985, who’s worked at what became GSK and Pfizer, who’s shepherded a compound all the way to a marketed drug, and a more complicated picture emerges. Because the challenge of finding a molecule that binds to a target is only one part of the problem. The molecule also has to be metabolically stable, non-toxic, orally bioavailable, and maybe most importantly, it has to be makeable. And that last constraint, synthesizability, turns out to be far harder and far easier to underestimate than most people outside the lab realize. My guest today is Paul Finn, Chief Scientific Officer at Oxford Drug Design, a computational drug discovery company with roots in Oxford’s chemistry department and a decades-long commitment to making sure that what the computer recommends can be made in a lab. Paul brings a rare combination: deep technical fluency across classical and modern computational methods, and a practiced skepticism about AI results that look too good to be true, because he’s done the work of checking whether they are. Today, we’ll get into what it means to search chemical space responsibly, why chemistry has proven so much harder for AI than biology, and what a rigorous synthesizability constraint approach to multi-parameter optimization looks like in practice. Paul, welcome to Data in Biotech.
Paul Finn: Thank you. Thank you, Ross. Very pleased to be here.
Ross Katz: To get us started, you started your career at what’s now GSK in 1985, moved to Pfizer, and then transitioned through a series of Oxford area spinouts before landing at Oxford Drug Design. You’ve got this 40-year arc from classical medicinal chemistry to modern computational methods. I’m interested in what that journey looked like and what drew you toward the computational side of discovery.
Paul Finn: I started off as a biochemist, doing a classical biochemistry degree. But I was always more interested in the theoretical aspects than the practical ones, never so great in the lab. And that ultimately led me to my PhD research, which was in the prediction of protein structure from sequence. This was back in the early 80s when computational approaches were still very much in their infancy. As a side project to that, I got involved in a little structure-activity study predicting the activity of peptide hormones, and that got me interested in drug discovery. So I went straight into industry to apply those methods, and have been applying them to drug discovery ever since, through those companies that you mentioned. Currently at Oxford Drug Design, that really just continues. We have a strong activity in developing novel computational methods, combining the new GenAI ones with traditional ones, and then applying them to our internal pipeline of projects, which are focused in oncology and antibacterials.
Ross Katz: Can you give me some examples of where the methods that have been around historically allow you to see the emergence of generative AI through a different light, or understand the gaps those methods might expose?
Paul Finn: They really all fall into providing checks on what is being produced by these generative AI methods, which still are to a large extent black box methods where it’s difficult to get a firm handle on why these particular molecules are being suggested. If you can then apply some rather straightforward methods — just generating molecular conformations or calculating simple properties — and ask, does this make sense? Could we find plausible reasons for why these molecules have been suggested? That helps give a lot of confidence that they’re not just some hallucination of the particular AI method.
Ross Katz: One of the things you mentioned previously is that it’s great that generative AI can produce a small molecule that in principle does what you need it to do, but there’s also the synthesizability problem that tends to be underestimated. Generative models are searching chemical space, but chemical space drawn on paper and chemical space a medicinal chemist can actually make are very different things. What have you seen happen when synthesizability is treated as an afterthought? And what changes when you treat it as a hard constraint?
Paul Finn: The chemical spaces that can be searched these days with generative AI methods are extremely large. But there are a couple of aspects that make them difficult to apply really effectively. At some point, a medicinal chemist or a synthetic organic chemist is going to have to make that structure so that it can be tested to verify that it really has that activity. It’s often underappreciated how not straightforward it is to be presented with a molecular structure and then told to make that. The link between the chemical structure and how easy it is to make — its synthesizability — is very complex. Just changing a single atom can change a structure from being very simple to make to being very difficult to make. What often happens is that all of the work that’s gone into that generative AI and the extremely large amounts of computation lead to molecules that cannot reasonably be made to validate those methodologies. So if you consider synthesizability from the start, if you somehow constrain that search so that it will only suggest molecules that can be made, this is a much more effective and much more practical method to move forward. I think this is also related to exactly how the AI methods capture that chemical structure in the first place — that also causes the link between chemical structures and their biological activity to be quite difficult to fully model in the generative AI approach.
Ross Katz: Can you unpack why that is? What is it about the way that generative AI models tend to model chemical space that leads to that problem?
Paul Finn: When generative AI was first being applied to chemistry — and still to a very large extent today — the first question is: how do you represent the chemical structure information to the generative AI model? Because so many deep learning methods have been built on things like large language models that deal with text and strings of characters, a lot of the early work, and much of the work still being done, represents chemical structures in that symbolic way — the chemical structure diagrams you see in textbooks and publications. Effectively, it’s a two-dimensional representation of the structure. The problem is that that’s a highly abstract representation relative to what actually happens: the three-dimensional chemical structures that are really interacting with the protein target. So you have quite an abstract chemical representation, and the AI methods then generate further abstractions of that into these spaces to generate their models. The link between the chemical structures and what really matters — a real three-dimensional molecule interacting with a real three-dimensional protein — makes it very hard for the AI to capture what’s really important. As a result, you get predictions that very frequently don’t pan out when you test them in the real world. Constraining the chemical space helps, and representing it in a way that is more closely related to the physical reality is really what’s helpful. There’s some movement in that direction, but it’s still not the predominant way in which this is done.
Ross Katz: Starting with constraining the chemical space — my understanding is that you have to have a sense of what the different building blocks are that you can synthesize and put together, and knowledge of the methods that are out there. It’s a combinatorial problem of figuring out all the different molecules that hypothetically could be made, and then fitting that into the specific problem you’re trying to solve. But one of the challenges is that when you constrain the chemical space, it’s much harder to find some novel approach to solving a particular binding problem or a particular therapeutic problem. Am I understanding the nature of the problem correctly? And if so, how do you navigate the trade-offs between ensuring synthesizability and being able to find something novel?
Paul Finn: The constraining of the chemical space involves synthetic reaction schemes that join building blocks together like pieces of Lego to make more complex structures. It involves having a well-defined set of chemical reaction types that are well-understood and precedented and work. It requires having a list of all the potential reagents or building blocks that can go into those schemes, and then working out what could be made in this potential virtual space. You’re quite right that it’s a valid criticism that taking that approach does shrink the size of the chemical space from the entirely abstract one of all theoretically available molecules. However, I think it’s a theoretical rather than a real limitation, because of the combinatorial nature of the chemistry and the way you can combinatorially combine building blocks in a sequence of reactions. Those chemical spaces are still extremely large — 99.999% of them have never been made by anyone ever in the history of humankind. So you’re still searching extremely large spaces, and there’s plenty of scope to find novel chemotypes that have not been found before. The other practical thing is, if you look at what chemical reactions medicinal chemists use — and there was a paper a few years ago that reviewed all the patent literature and asked exactly that question — there are about 10 reactions that dominate the entire thing, because those reactions work reliably and there are a lot of reagents available to do them. So this is a very viable approach, which still leads to very large chemical spaces and gives you a good opportunity to find novel patentable chemical matter.
Ross Katz: That makes a lot of sense. There’s a balance between over-constraining yourself and under-constraining yourself. When you select the right degree of constraints, you’re able to find the correct combination of space to be creative while also knowing you’re going to get the characteristics of the molecule that you actually want from the process. Am I thinking about that right?
Paul Finn: Exactly right. One other comment: you can expand the chemical space by just increasing the number of reaction schemes you have and the reagents. Building more sophistication into those schemes matters because — another underappreciated point — even when the synthetic schemes are well-known, published in the literature, and people have used them to make molecules, you try to reproduce that or use that scheme to make a slightly different molecule and find the first time you try it, it doesn’t work. You have to tweak the conditions a bit or change the solvent and then you can get it to work. The synthesis is really very challenging, and one of the areas we’re working on is building those synthetic spaces to make them more and more sophisticated — to expand the size of the database, but also improve the reliability that you will actually be able to make them at the end.
Ross Katz: It’s a hard problem to keep track of all of the different new discoveries being made, and also to apply sufficient rigor to the validation of results before incorporating them into the library. There’s this entire spectrum of results that would either more effectively constrain the space or expand the possible chemical space that you have to weigh evidence on — and I’m imagining a very large rules engine for how the combinatorial problem ends up being computed. Am I thinking about that right?
Paul Finn: Yes, absolutely. Everything we do is driven by pragmatism. If a reaction comes up several times and looks interesting — and another point is just how many reagents there are. You could have a lovely reaction, but if there are only three of group A and three of group B, adding nine compounds to your virtual database is probably not going to be worthwhile. It’s a whole bunch of pragmatic and practical things. And then if that reaction seems to be generating interesting molecules, you can go on and further refine the definition to improve the quality, once you know that it might be useful in the future.
Ross Katz: I want to turn back to this idea of modeling molecules better in 3D than generative AI models typically do with the language-based approach. Can you introduce us to some of the ways that 3D modeling more effectively captures how the structure of the molecule relates to its activity and interaction pattern?
Paul Finn: In some senses, that’s very straightforward because it just builds on all those historical computational chemistry techniques we were talking about at the beginning — generating a three-dimensional structure of the molecule where each of the atoms is in a physical three-dimensional space and the atoms are joined together by their bonds. Those techniques have been well-established for a long time. Representing the structure to the AI as a three-dimensional object is in some ways straightforward. However, there are still a lot of complexities to that. Shape is probably the pre-eminent property that drives molecular recognition and binding. You have the shape of the drug, you have the shape of the cavity into which it’s fitting, and they go together in a complementary surface. The more complementary they are, other things being equal, the stronger the binding. But it’s not just the shape. Molecules are electrostatically charged and they’ve got electrostatic potential around them, which is involved in the strength and orientation of the interaction. And then you have the additional complexity that most molecules of pharmaceutical interest are not rigid bodies — they are able to change their shape by rotating about the bonds and adopting other shapes. So for any individual molecule, you have a plurality of shapes that they could adopt. Building all of those and representing them to the generative AI method is, at the first level, simple, but as you try to make a more sophisticated model, it becomes more complex. That’s especially true if you add the protein side to the equation and are trying to represent the shape of the cavity into which the molecule fits. But at the end of the day, that is much closer to the real physics of the interaction. Despite those complexities, in our view, that is a better chemistry representation to be using than these abstract, linear, string-based representations which are very far divorced from that physical reality.
Ross Katz: I came across that some of your colleagues at Oxford are using topological methods to model the shape in different and more nuanced ways than before. Where is our understanding of these different elements limited, and where is research trying to push in terms of exposing more and better information into the models to understand binding behavior better?
Paul Finn: In principle, the physics of what’s going on when a molecule binds to a protein are pretty well known — pretty well worked out. But applying them in sufficient rigor is what’s difficult. In the limit, the quantum mechanical description is going to be the most accurate one, but it’s largely impractical to apply even in the current state of computational power. We use approximate methods, but they’re still pretty accurate. What’s difficult is that there are other factors we haven’t yet discussed. We’ve discussed a little bit the fact that molecules are flexible and proteins are flexible, and that creates lots of additional complications. The other complication is that these interactions occur in an aqueous environment — they occur in water in the cell. But it’s not just water. It’s water with lots of counterions and other proteins and so on. The role of the solvent in this interaction, how it’s involved in stabilizing or destabilizing the interaction between the drug and its protein, is also very complex. We’re still some way from being able to simulate that well at an atomic level. Work is going on in that direction, and in the other direction of trying to make simplified models that are actually tractable. One example from what we’ve been doing — other people are doing similar things — is to generate from those 3D molecular shapes a compact representation. Rather than having to keep the 3D coordinates and all of that electric charge information, you try to generate from that something which is a more compact summary of that information that is more amenable to database searching, but still contains the essence of the interaction, albeit with some noise introduced.
Ross Katz: Should I think of that as a graph-based embedding, or is there some other way of compacting that you’re talking about?
Paul Finn: You can certainly do that with graphs. We do it in a slightly different way, generating vectors of numbers that are chosen to reproduce the shape and electrostatics at a particular level of resolution. There’s not time to go into the details of how that works, but it’s very useful to generate these compact, simpler representations because the molecular comparison then becomes much faster and the data storage becomes much smaller. The topological work is also directed at that, but in some ways it’s looking at topological invariants at different scales — in what ways do the shape or properties you can calculate from the shape stay the same or change at different levels of spatial resolution? That is in principle a way to encapsulate some of the things which the molecule keeps if it changes its conformation. That’s another way of simplifying the structure, but an orthogonal way — so you’re getting two simplified representations that are capturing different aspects of the 3D structure.
Ross Katz: That aligns with what I’m hearing across the model-based conversations we’re having — it’s all the different lenses you can take on a given phenomenon that are orthogonal, and then you have a compact representation of each and concatenate those together, which hopefully feeds more information into the entire modeling system. If it’s okay with you, I’d like to switch gears. You’re searching chemical space for molecules that have certain characteristics: potency, metabolic stability, solubility, selectivity, toxicity, and synthesizability. How many properties are you trying to optimize simultaneously, and how do you approach that problem?
Paul Finn: Drug discovery is always a multi-parameter optimization. The clinical candidate is never the one that’s the most potent or the most soluble or the most stable. You’re always searching for this compromise of properties where it’s good enough against all of the criteria that need to be considered. That list is very long, so it’s actually very hard to try to do this in a comprehensive way — and I think rather foolhardy to try to put all of the things you want into the calculation. There are just too many, and we don’t know enough in detail about some of them for the models to be any good anyway. We focus on the things which, for the particular project, we think are the most important. They can often be pulling in different directions. Making the drug more hydrophobic will probably improve its potency, but it will also probably make it less soluble, and you need both. You have to deal with those tensions. By tailoring the scoring function to your particular problem, you can weight the different parameters to some degree — you might be more keen on maintaining potency than on solubility, or maybe your solubility is already good enough so you don’t need to weight that so much. The other important thing, of course, is to check back frequently with experiment to make sure you’re on the right track, because it’s quite easy to go down blind alleys if you don’t do that.
Ross Katz: In terms of the problem-specific nature of choosing which aspect you focus on, I think it might be useful to ground in the specific therapeutic areas and approaches that Oxford Drug Design is doing right now. My understanding is you’re working in two therapeutic areas: new antibiotics to address antimicrobial resistance and oncology. Can you talk to us about these programs, and then maybe we can return to how you think about optimizing for these different therapeutic areas?
Paul Finn: Those are our two therapeutic areas of interest. In some ways they have some similarities — they’re both about ultimately killing or stopping the growth of particular cells, but in one case they’re aberrantly proliferating human cells and in the other case they’re undesirably proliferating bacterial cells at some infection site. There are some commonalities, but the individual approaches are quite different. The targets, even if they’re similar, are different in those two organisms, so you have to deal with that. A useful example is that if your target, as it often is, is inside the cell, the molecule needs to get inside the cell to reach that target. Getting sufficient concentration of the drug inside the cell for long enough to have the desired effect is quite a challenge — particularly in the bacterial programs, because bacterial cells are very different from human cells. Their structures vary from organism to organism, but essentially they have a complex cell envelope or wall around them that is a very significant barrier for small molecule drugs, getting across that barrier into the site of action. Discovering new antibacterials has proven to be extremely difficult. Not many new antibacterials have been approved in recent years, and almost always this lack of accumulation into the cell is what causes failure. It’s very rare for a compound not to bind strongly to the target — it’s overcoming this formidable cell wall barrier that’s the main issue. So that’s a clear differentiation in terms of the design approach with these two projects. We’ve had to think much more about cellular penetration in the antibacterial projects than in the oncology projects.
Ross Katz: How does that enter the optimization problem? A good way of talking about this would be to orient us to the drug design and discovery process you go through, and how the cellular penetration element enters the process for selecting the next set of candidates to synthesize.
Paul Finn: The way this works is that you have what’s usually called a screening cascade — a series of assays that you run on new compounds. If they have a good result in those assays, you take them on to more complex and usually more expensive assays, to work out how close they are to becoming a clinical candidate. What changes is effectively the assay pipeline in the antibacterial case. You have assays that are giving you a readout with regard to this cell penetration. That can be done in a simple way: for example, you have a compound that’s a very potent inhibitor of your protein target, but when you administer that to bacteria, it has absolutely no effect. Plausibly the drug is not accumulating at the target, and that gives you information so you could try to change the structure in a way that it might accumulate. But you can then develop more sophisticated assays where you take mutants of the bacteria where some aspect of its cell wall is mutated in a way that makes it easier for compounds to get in, or makes it harder for the bacteria to pump them out. Now you’ve got a more sophisticated screening cascade giving you more information about what actually is the problem: can it not get in? Is it metabolized when it’s inside? Is the bacteria pumping it out through efflux mechanisms? You can then build that data into your model. At that point, you’re not mostly optimizing for activity against the target — you’re just maintaining that at some reasonable threshold, and it’s really these other properties that are going into the optimization.
Ross Katz: As I understand it, you’re using Bayesian optimization — this is an active learning loop where you’re choosing the top set of candidates to run your assay cascade against and then feeding the data back into the model.
Paul Finn: Yes, I can explain how that works in outline. We’ve got this synthetic accessibility-based approach. We have reaction schemes, we have reagents, we have a chemical space that we can build. As we generate data, we start to be able to associate that data with particular structures in that virtual space. In contrast to the classical way where you’ve got this enormous space and you’re just picking molecules out of that space that are in some ways already defined, what we’re doing is searching the reagent space. What combination of reagents is associated with the best activity? That’s really what the Bayesian optimization is doing — focusing on those patterns of reagents that lead to molecules associated with high levels of activity. It’s a Gaussian process, and the Bayesian optimization helps, as we build the data, to focus on those regions of chemical space where the activity seems to lie. Those then generate the particular examples of the virtual library which we make, we test those, we get more data into the model, and iterate in that way.
Ross Katz: One of the benefits of the Gaussian process modeling is that it gives you a measure of uncertainty that you can use as part of navigating the trade-offs between exploration and exploitation of the knowledge you’ve gained. Can you give us a sense of how uncertainty is measured inside the process you just described?
Paul Finn: It’s actually quite tricky to do in a sentence or two. We’re looking at the nearby reagents and how likely it is that this would lead to an activity within a certain level of what we already know. It’s actually quite hard to quantify, because of the discrete nature of what we can make. We get a pretty good idea when the signal is pretty flat and not very strong. It’s certainly not at the level where you have a very high degree of confidence in any of these predictions — there are certainly lots of false predictions — but it’s good enough to make progress. At this current stage of development of these methods, that’s all we can hope for at the moment. We have found interesting improvements in the structure through these approaches, and there are many ways they can be improved and which we’re trying to improve them.
Ross Katz: That makes a lot of sense. Gaussian process modeling has been around for a very long time, and deep learning is all the rage right now. I know that you had a 2023 paper in the Journal of Molecular Modeling with Rishi Jagdev and Thomas Madsen where you took the Stokes et al. Halicin paper apart in a specific way, and showed that Halicin can be recovered from the same data set using methods a lot older than deep learning. I’m interested in that result and what conclusion you come to about where data-hungry methods like deep learning belong in a drug discovery process versus where the older, more principles-based methods are the right approach.
Paul Finn: It really comes down to validation of the methods. Many of these generative AI or deep learning methods are very complex. They use a lot of computer power and generate models with very large numbers of tunable parameters. They can produce results that superficially look extremely good — the Halicin publication is one example. But there are many others where, in completely different approaches to deep learning or generative AI, the publication seems to indicate that the method has actually learned the physics of the interaction between the ligand and the protein, because in all their validation they got very high correlations with the test data sets. However, subsequently it’s proven that those results were artifactual, arising from something else — and those something elses are frequently in the preparation of the data. There are many ways in which chemistry data sets could be biased, and that just improves the chances of getting a false correlation. These things are very difficult to do, and generating a good quality data set is difficult. But also what they didn’t do in the Halicin paper — and I think is frequently lacking — is benchmarking against a really simple methodology. There are lots of simple methodologies that could be applied in most cases, and it’s really the improvement you get over those simple methods that tells you to what extent all of that extra complexity and sophistication and computer power are actually doing anything useful for you.
Ross Katz: There are a couple things there that are valuable to press on. Having a valid baseline comparison tells you how much of it is the complexity of the mechanism you have to pick up in the data — that’s what we’re used to happening with deep learning, where it’s able to, by nature of having so many parameters, pick up on patterns that simpler methods are not. And how much of it is overfitting to elements that have leaked between your training and your test set? There’s a degree of care with the data set and with the methods that seems like a lost art form with the emergence of deep learning as the leading approach across so many other parts of the modeling space.
Paul Finn: To be fair, it’s often not just that it hasn’t been done — it’s genuinely very difficult. In some of these protein-ligand prediction methodologies, trying to develop a method that will generally be able to predict binding affinity accurately, the data sets from crystal structures are such that it’s very easy — you never want structures in both your training set and your test set, but it’s very easy given all the molecular biology for there to be a very similar structure in your test set to one in your training set. And so you’re just giving the method an easy win. Testing it on a genuinely novel set of proteins that have genuinely not been seen by the training method takes an awful lot of work, and people have tried to do that well, and then subsequently others have said, “Ah, but you didn’t think about this.” So it’s also genuinely hard to do, as well as people not giving it enough attention.
Ross Katz: It’s hard to know the comprehensive set of questions you have to ask about a model to be sure it’s not memorizing some artifact or hacking your reward function — all the things that models are likely to do when they’re consuming a lot of data and heavily optimized toward an objective. The canonical example within drug discovery is the success of AlphaFold, which was built on decades of accumulated structural data with sequence-to-structure relationships across hundreds of thousands of proteins. But as you gestured at earlier, chemistry doesn’t really have an equivalent — because the relationship between chemical structure and biological activity is much more complex, and the data is more fragmented and much of it is proprietary. I’m interested in your perspective on how data scarcity shapes what’s possible computationally in small molecule drug discovery.
Paul Finn: I think both of those things are true. In biology, large language model type methods dealing with strings of characters work quite well for bioinformatics. There’s a more natural match there — the building block reservoir of proteins is only 20 amino acids, and the structures consist of lots of regular repeating helices and sheets. Whereas chemistry is far more diverse in terms of the chemotypes and the size — it’s just a much harder problem. In terms of the data, we just don’t have it. There’s a lot of chemical data but not so much where there’s associated relevant biological data. I was reminded there was a Nature editorial — maybe three years ago now, but I think it’s still true — the title of which was, “For Chemists, the AI Revolution Has Still to Happen.” It was making this point that there just isn’t that set of data, not even for something relatively straightforward like binding affinity. And then if you go to metabolism, where there are hundreds of metabolizing enzymes, or to other areas, how would you build models that could be effective when the data is so scarce and diverse — different labs produce different results and so on. Quite a lot of it is locked away in Big Pharma vaults. Some of the Big Pharma companies internally may have enough data to do something useful with some of those models, but not an awful lot. There are efforts to put these things into the public domain in a pre-competitive way, but that’s still pretty much lacking. It’s an enormous impediment to generating good models in chemistry.
Ross Katz: If you were in the position of advising a pharma company evaluating an AI drug discovery vendor — someone claiming that their platform can accelerate hit identification or improve synthesizability or reduce attrition — what are the kinds of questions you would tell them to ask?
Paul Finn: In general, the answer is: how well have they supported their claim? Being able to quote very large numbers doesn’t matter at all. But if they’re saying they can find good structures out of these databases, it really comes down to: prove to me that you can actually do this in the real world. Ideally, they would have done this themselves, or with an academic partner — there’s a publication where they have shown that they have identified a molecule that wasn’t known about before, or wasn’t close to a molecule that was known before, that has the activity they were trying to find at the start. Those sorts of validations matter. The other thing is to really try to understand what was the AI’s contribution to that finding? Because many of the AI papers claiming or reporting molecules discovered through AI — when you read the paper, the AI component is step one, and there were 99 other steps afterwards, all of which required human input to ultimately decide that that was the molecule. The publications are not deliberately poor, but it’s very difficult to really understand what was the real contribution of that method versus the contribution of all the things that were done afterwards, and indeed maybe before in terms of data selection. That I think is often a difficult question for the companies because it’s hard for them to demonstrate that the AI really was the key or major contributor to that finding.
Ross Katz: As we head toward the end, are there any methodological unlocks in drug discovery and development — things like representations of molecular flexibility, or the integration of physics-based constraints with learned models, or data set quality and curation — that you think are likely to help advance AI’s contribution to small molecule drug discovery?
Paul Finn: We’ve talked about data preparation and how difficult that is, and that’s an area that still needs work. There are academics working on this — Charlotte Deane’s group in Oxford, who are near to us, are doing some really good work on trying to generate better data sets in their area of interest, which is protein-ligand binding. There’s a lot more that can be done there to make proper tests for these methods. There are also some areas in the chemistry representation that are still not very well treated at the moment. Two examples would be the charge state of the molecule — is it protonated, is it not protonated, is it a salt — and also tautomers, where atoms move around within the structure but what’s in the solution is a mixture of two different things in rapid equilibrium with one another. In chemistry databases, you usually have to pick one or the other as the way you’re going to represent the molecule, and that introduces biases and another chance to make false predictions. There are ways in which chemistry representation could be improved, within the sphere of 3D representations, to give the methods better substrate on which to work.
Ross Katz: Paul, I really appreciate the time. This has been a great conversation, and I look forward to connecting down the line.
Paul Finn: It’s been a real pleasure. Thanks very much for inviting me on the podcast.
Ross Katz: Paul, thank you. This was a real pleasure. What I’m taking away from this is more interesting than the usual story about AI and drug discovery. That story tends to be a fight: old methods versus new ones, classical chemistry versus deep learning. Paul’s whole career cuts against that. The methods chemists developed over the last four decades still work, and the best results come from using them alongside the new ones, not instead of them. Three things stuck with me. First, synthesizability — easy to wave away, hard in practice. A generative model can hand you a great theoretical molecule, and changing a single atom can take it from something a chemist makes before lunch to something nobody can make at all. Paul’s answer is to build the constraint in from the start: start from the reactions that work and the building blocks you can buy, snap them together like Legos, and search that space. The obvious worry is that you give up novelty, but that space is so large that 99.999% of it has never been made by anyone, ever. The constraint barely costs you anything real. A lot of differentiation in drug discovery comes from the order in which synthesizability is brought into the picture, and it’s instructive to see how Paul and his team treat it as a first-class constraint. Second, a lot of these models learn from a two-dimensional drawing of a molecule. The thing that actually does the work in your body is a flexible three-dimensional object moving around in water. The model learns from an abstraction of an abstraction, two steps removed from the physics that matter. A lot of the work is figuring out how to make the model computationally efficient while capturing sufficient information to guide the selection and the assay cascade process that Paul described. Third, a complicated model with enough parameters can look extraordinary but end up measuring nothing but an artifact in your data. The discipline that protects you is the boring part — benchmarking the fancy method against a simple one, and doing the grunt work of searching through your training, validation, and test set to make sure there aren’t artifacts your model is going to learn or data leakage it is going to discover. Paul and his co-authors demonstrated with the Halicin result that for the right problem, simple methods can do just as well as the complex methods, if not better, because they’re not as prone to overfitting — methods decades older than deep learning can recover the same answer or get you to a better one. You can find Paul on LinkedIn. To learn more about Oxford Drug Design’s work in antimicrobial resistance and oncology, visit oxforddrugdesign.com. If you know a computational chemist, a data scientist in drug discovery, or a team being pitched AI tools who’d benefit from a more rigorous lens on what those tools deliver, send them this episode. And if you haven’t already, please subscribe to Data in Biotech wherever you listen. Thanks for tuning in.
Jason: And that’s it for this episode of Data in Biotech. If you enjoyed the episode, please subscribe, rate, or leave a review in your podcast platform of choice. See you next time.
