1189 stories
·
1 follower

Evaluating "Co-Scientist", a New AI Science System

1 Share

There are a couple of very interesting AI-assisted-chemistry paper out this week, and I have been trying to find the time to take them on. Let’s do this one first. It’s a report on “Co-Scientist”, a system to help propose hypotheses, potential mechanisms, et al. for biomedical discovery.

The authors first applied this in what I think is a good test bed for it: drug repurposing. Let’s put aside the not-all-that-high success rates seen in these efforts, because nothing in this field has a high success rate when you get right down to it. I saw that it’s a good area because existing drugs at least have a lot of fairly reliable data around them, and that’s what a system like this needs most of all.

The first test was looking for new single-agent and combination therapies in oncology. The program suite is well-named, actually, because it turns out that human input and judgement is required all along the way (you may be relieved to hear). The beginning step was prediction of the effects of 2300 approved drugs profiled across 34 cancer types, with the predictions then human-evaluated by one or more oncologists. The most actionable predictions were deemed by this review to be the ones for acute myeloid leukemia (AML). The human team selected four different AML cell lines and one control line to evaluate compounds in, but as the paper correctly notes, this is more of a reality check than any kind of big step into clinical developability.

The selection of compounds to assay (like the selection of the particular cancer type and the selection of cell lines to evaluate it) was made with “meticulous expert oversight”, i.e. human oncologists. Thirty drug candidate proposals were reviewed by this committee for their potential AML relevance, and five were selected: binimetinib (an MEK/MAP2K inhibitor), pacritinib (which inhibits JAK2 among other kinases), cerivastatin (an HMG-CoA reductase inhibitor statin drug whose withdrawal from the market ultimately led to the closure of the entire research site I used to work at!), pravastatin (another drug in that same class), and dimethyl fumarate. The first three showed activity in relevant cell assays, with binimetinib being the clear winner in potency.

But hold on: I would have to add at this point that targeting the MEK pathway in AML (as with binimetinib) is not exactly a new idea, with a number of lines of research having been investigated. As for the other two that showed cell activity, pacritinib has already been into the clinic in some AML subtypes, and the unusually strong ability of cerivastatin to induce apoptosis in AML cell lines was noted in the literature 25 years ago (which work could also have predicted pravastatin’s lack of activity here. I do not see citations of any of those references in the manuscript in its current form.

The team then used the Co-Scientist system without oversight to propose single-agent drug repurposing ideas for AML, but as before, they did have an expert human panel review its proposals. The paper doesn’t say how many candidates went into that step, but three came out the other end of the review: nanvuranlat (a LAT1 inhibitor), KIRA6, and leflunomide. When these went into the cell assays, only KIRA6 showed activity. It’s an IRE1-alpha inhibitor, and as the paper does note, that mechanism has actually been proposed before as an AML therapy (although not with this particular compound). I’m not sure if I would call that “repurposing” or not, but that’ll vary on your own definition.

Of course, oncology is all about drug combinations, cancer cells being what they are. The paper goes on to look for possible synergistic combinations, and I would have to say up front that synergy in these situations is a lot less common than you’d hope for. The results were complicated, not that that came as a surprise, I’m sure. In one cell line (MOLM-13) a number of the combinations did show at least additive activity, but in another (KG-1a), there was a wide range of results, with some combinations apparently cancelling each other out. As the authors put it, “Further mechanistic studies will be required to define the molecular determinants of response to combination therapy across AML subtypes, and to identify predictive biomarkers that could enable rational regimen selection”, and they sure are right about that.

The other aspects of the paper are rather less well-documented in this manuscript, but that’s because they are covered in other publications. There’s a paragraph or so about using Co-Scientist with human hepatic organoids and cell imaging to come up with new repurposing ideas for liver fibrosis. The one that they specifically mention is vorinostat (there’s more on that here in a separate paper), but as before, a look through the literature shows that this compound as well has activity against fibroblasts and in other models of fibrosis in general. Indeed, the entire class of HDAC inhibitors (of which vorinostat is the prototype) has been investigated in fibrotic disease models and in liver fibrosis in particular, up to animal models. I think it’s good that the system picked up on this, but this is not exactly a de novo result. I should note that none of the papers just mentioned appear to be referenced in the current manuscript, nor do I find references to them in the separate manuscript on this work linked above.

There’s also a mention of Co-Scientist recapitulating a very recent result on an antibiotic resistance mechanism, the exchange of capsid-forming phage-inducible chromosomal islands (cf-PICIs) to spread resistance genes. The system appears to have proposed the same mechanism that the research team had arrived at, which is that these interact with specific phage “tails” to move into new hosts. You can find more on that in this paper and in this one. And that does seem quite useful, using the system as a focused “results digester”. It would be interesting to see what its proposals would have been along the way as the experimental data developed - I suppose this could be recapitulated, with some effort - and if the experiments themselves would have been redesigned and perhaps converged on the answer more quickly. Or not! It would be quite useful to know.

My overall take is that this looks like a promising system, especially for uses like that last one, where you have a large corpus of experimental data and would like to see what the software makes of it. I’d like to see some other examples of just that sort of test being run (where you know the answer and want to see if Co-Scientist arrives at it and at what stage). The open-literature drug repurposing work is to me more of a mixed bag. I think it’s good that the system identified the mechanisms that it did, but I don’t think that the paper does enough to point out that none of these ideas are without precedent - in some cases, a lot of precedent. I wonder how much of the human-review steps mentioned involved people noticing such papers and prioritizing those compounds and mechanisms (naturally enough!) But as you might imagine, the publicity around this work seems to be pointing in the other direction entirely, which I don’t think is doing anyone much of a service. We definitely need all the help with the literature that we can get, but hype we have enough of already.

Read the whole story
tabithaclem
1 day ago
reply
Share this story
Delete

Reading the Labels on Mutant Mice

1 Share

I’m working on a big post or two, but I did want to get something up today. Have a look at this rather disturbing news from the mouse model world, published here at Science. The authors went to the trouble of genotyping 611 mice tissue samples representing 341 strains at the MMRRC (Mutant Mouse Resource and Research Centers), and as the paper notes, rather dryly and through pursed lips, “Users’ expectations for congenic strains based on nomenclature are not consistently met”.

Yeah, you could say that: nearly half the mice were not really what they were labeled as, with failures at several levels. Worries about this have been building for years as sequencing technology became more and more capable, but I think that this is by far the largest-scale and most detailed look at the problem. To be sure, many of the inconsistencies are believed to be relative minor, but not all of them, for sure. Some of the problem is in the naming conventions used, which are probably promising more than they can deliver:

In addition to the inconsistencies between reported names and genotypes reviewed above, mouse nomenclature as defined by the ICSGNM (10) is not designed to reflect the complexities and nuances in the genetic makeup of many strains. However, names matter, and most users of mouse models have definite expectations for congenic strains based on our frequent conversations with MMRRC and MiniMUGA customers, and with scientists at professional conferences. For example, users expect that congenic names denote strains with the only contribution of the donor strain occurring in the chromosome carrying the relevant mutation. They also expect that such strains are inbred and thus have extremely high levels of genome replicability. These expectations are not consistently met in the MMRRC sample set referenced above, as naming rules allow for a wide range of variation of congenicity and inbreeding levels . . .

A great many might be best described as “incipiently congenic” at best, and the proposal is to capture this in thesummary description of each strain rather than trying to work it into the names themselves. The authors detail an “MMRRC Strain GQC Report” format that tries to get across the key messages using the same layout and the same terms across all the samples - type of strain, the allele and locus of interest, the primary and secondary genetic background, and whether or not the genome is truly replicable (and why not, if that’s the case). It’s designed as well to be able to flag the all-too-common situation of some genetic contributions from yet a third strain of mice along the way., and it also has warnings for constructs that include things that could mess up a variety of experiments, such as having cre recombinase in there or genes for various fluorescent proteins.

It looks like well over a hundred strains have had such reports generated, and teams are working their way through the rest. One hopes that this standard will catch on with other such mutant animal repositories, because there’s no reason whatsoever to think that they have been immune from the problems that got us to this situation. If we’re going to do animal research - and we still have to - we should absolutely be getting the most value out of it and wasting the least number of animals (and the least time and money, needless to say). This is both a scientific and an ethical charge, and I’m glad to see it being addressed.

Read the whole story
tabithaclem
8 days ago
reply
Share this story
Delete

Stemming the AI Preprint Flood

1 Share

The arXiv preprint server has become essential to several scientific fields, and has inspired similar services like BioRxiv and ChemRxiv. But the generative AI age has been hard on these sites, and you can see it by their steady imposition of new rules.

Last November, arXiv announced that it was clamping down on the submission of review articles, particularly in computer science. These were becoming alarmingly easy to put together (often as not padded out with references that didn’t even quite exist), and the moderators were getting overwhelmed. No new reviews would be accepted in computer science, they ruled, unless they had already been through a peer-review process at a journal (or for a conference), which sort of turned the entire idea of the arXiv on its head in that manuscript category. But you can understand why.

Then in late January came the announcement that first-time submitters would need an endorsement from another author who had published there before. The stated reason was a rise in outright fraudulent submissions. That article says that the moderator's rejection rates had tripled over 2025 and were apparently still rising, and something obviously had to be done.

Now the chair of the site’s computer science editorial committee (Thomas Diettrich) has turned the flames up. “If the submission contains incontrovertible evidence that the authors did not check the results of LLM generation, it means we can’t trust anything in the paper. The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue”. He went on to give examples of such evidence, including the above-mentioned hallucinated references (to papers that just don’t exist) or the presence of meta-comments from the LLM software ("Here is your summary" and so on).

Those fake references are indeed becoming a plague, as illustrated by this manuscript at arXiv itself. Those authors set a lower bound of over 146,000 hallucinated citations appearing in 2025, and it's been rising sharply. This obviously threatens the integrity of the scientific literature itself, and that really didn’t need to take any more torpedos, thanks. If we don’t do something about this we run the risk of an eventual painful de-slopping process, and it would be a *lot* better for everyone if we blocked that crap from getting into the literature in the first place. It’ll be a race. . .

Read the whole story
tabithaclem
17 days ago
reply
Share this story
Delete

A Futuristic Synthesis Indeed

1 Share

I definitely need to cover this recent work from Merck, because (1) it’s very interesting scientifically and (2) it has over 130 authors on the paper (!) It details the industrial synthesis of enlicitide, which is a beast of a macrocyclic peptide (see below!) Just looking at the structure tells you that this must be a ferociously active molecule with huge commercial potential, because there is just no way that anyone is going to make this on scale - or find a way to make this on scale! - otherwise.

This drug hits the PCSK9 pathway, which has been quite a story over the years. This target was famously discovered by finding a few people with mutations in the underlying gene who had bizarrely low LDL concentrations and who seemed to have correspondingly robust cardiac health. The first drugs on the market to take action on this idea were antibodies to block the receptor, and those were approved by the FDA some years ago as injectables.

They do indeed lower LDL, but (as that last blog post link notes) the results in humans have to be characterized as “good but not revolutionary”. I don’t know of any studies that show a definite advantage compared to statin (HMG-CoA reductase inhibitor) therapy, for example, although there are statin side effects in some patients that have to be taken into account. (On the other side of the question, statins seem to have some beneficial pleiotropic effects that we don’t quite understand, and whether these are shared by PCSK9 inhibition, I don’t know). And there are other people for whom statin therapy just comes up short, to be sure.

Several companies have tried over the years to come up with a small-molecule (well, smallish-molecule) approach that could lead to an orally dosed therapy as opposed to an injectable, and Merck has apparently made it over the finish line with this one. Here are the results of a trial in over 1900 patients (compared to over 900 in the placebo group) treated with the drug over a year, 20mg once a day. Over that time, the placebo group’s LDL went up about 3%, while the treatment group’s went down about 57%, which is what we call “pretty darn significant”, statistically speaking, with no differences in adverse events.

So now that you have a new cardiovascular drug, how do you make it for the hoped-for large patient population when it looks like, well, that thing to the right? That’s quite the multicyclic peptide, and while a lot of the key bond formations are good ol’ amide couplings, you have several that are not. The team divided up the molecules into “Western”, “Eastern”, and “Northern” pieces (based on the three macrocycles in the final structure) and demonstrated that they could make all of these in crystalline form (thus obviating the need for chromatographic purification). The Northern one was the toughest by far, with three unnatural amino acids and a choice of amine nucleophiles.

The answer to putting all this together was harnessing amino acid ligase enzymes, which lets you couple unprotected amino acid partners when everything is working right. The Merck team looked over a list of AAL enzymes and found one from Bifidobacterium adolescentis that was accomodating enough to deal with their intermediate. All of the candidates turned out to be able to handle peptide chains as the nucleophilic partner while only accepting single amino acids as the electrophile. That’s too bad in a way, because you could imagine assembling larger fragments this way, but if you get the enzymatic process running smoothly enough you can just turn things through it several times in a row. And without having to do protection/deprotection steps!

What looks like the nastiest traffic jam in the synthesis was the final stage of making that fragment, because you’re presented simultaneously with three amines and two carboxyls that need to be brought together in the proper pairings. The Merck group turned to esterase enzymes rather than proteases/amidases, not least because those can’t be tempted to run the amide formation in reverse because they don’t have the nucleophilic horsepower in their active sites. A previously-unreported enzyme from a Roseibacillus species showed promise, although I would certainly like to hear how many others got screened along the way, and even that one needed some engineering to increase its selectivity. They ended up being able to make the Northern piece with four enzymes and three separate building blocks in a single pot, which really is a tour de force.

To elaborate on to the final macrocycle the team called on thioesterase enzymes, which are generally what come into play in biosynthesis pathways for natural products of this type. Another (no doubt wide-ranging) screen identified a likely one from Brevibacillus laterosporus, but this also needed artisinal modification in its sequence to get the yields and selectivity up. I am skipping over a lot of work when I write sentences like that one!

The Northern and Eastern fragments also needed a reductive amination to bring them together, and a ketoreductase enzyme from a Kyrpidia species coupled with an imine reductase (from Pseudogymnoascus species) was used to make the requisite aldehyde, both after still more screening and protein engineering. This reaction mixture needed recycling of NADP/NADPH to run the synthesis enzymes, and still more enzymes were brought in for that cycle. In the end, the team achieved a one-pot five-enzyme process that did the overall transformation in 69% isolated yield.

Coupling this to the Western fragment was done through good ol’ chemical means (diphenylphosphinic chloride) while protected a stray primary amine that needed to be ready for the final macrocyclization. There are many such syntheses that have come to grief at such final steps; these macro-ring closures do not always want to happen easily. Another engineered thioesterase (this time from a Streptomyces species) was found to do the job and without the traditional need for high dilution. You often have to do that dilution to encourage your molecule to bite its own tail and make the large ring as opposed to reacting with another one nearby (which makes useless dimers or oligomers), so this enzymatic route is a huge help. The transformation comes though in 84% yield and >99% purity after liquid-liquid extraction and salt formation!

The overall yield, starting from 5-fluoro-N-aminohexyl tryptophan (you can see it hiding in there) is 39% and this has been run on multikilo scale with no chromatography. This is tremendously impressive, truly state-of-the-art process chemistry. I think we’re going to be seeing the reverberations of this work in macrocycle synthesis (and especially macrocyclic peptides) for a long time to come. Now to see if the drug itself performs up to its commercial and medical potential!

Read the whole story
tabithaclem
20 days ago
reply
Share this story
Delete

COCO AND HER SISTER

1 Share

Coco is on the left.  I get her and Lizzie mixed up!  Two little black Pekes sitting together. 

No matter where their mom and dad are, they are watching.  And mom is being watched the most.  

Like here.  Coco is with her brother, Elwood, and he's joined them.  Their mom just walked out of the room.  Don't worry guys.  She's coming right back.  
My dogs do this, too.  But some don't just watch.  They follow me, especially Piper.  Gwennie isn't far behind.   I bet your dogs do the same.  :-)
 

Read the whole story
tabithaclem
30 days ago
reply
Share this story
Delete

The FDA Cracks Down on Tavneos

1 Share

We have quite a regulatory situation developing around a drug called avacopan (Tavneos), which is given to patients with a particular type of vaculitis. That’s a complex disease area, and comes in several varieties, but a common theme in many of them is an autoimmune attack against various proteins found in neutrophils. The drug is an antagonist of the complement 5a receptor in the innate immune system, and it’s given along with other immunosuppressants. 

It was developed recently by a company called ChemoCentryx, who ran a pivotal trial in 330 patients. Half got the standard of care plus a placebo and half got that standard plus the drug over 52 weeks, and the endpoints were remission at 26 and 52 weeks. Both endpoints were similar, and both showed a real benefit to the patients. Tavneos was approved by both the FDA and the European CHMP in 2022, and Amgen went on to purchase the entire company.

But earlier this year, the CHMP announced that they were starting an investigation based on reports of loss of data integrity in that trial. And the CDER at the FDA is proposing to have it withdrawn from the US market over the same issues. It’s very, very bad - here’s the FDA statement:

. . .new information that only became known to CDER more than three years after approval shows that unblinded study personnel manipulated the results of the pivotal clinical study so the drug looked effective when the original analysis did not support that conclusion. The applicant also did not disclose the original analysis to FDA, in violation of FDA regulations. CDER can no longer conclude that there is, or has ever been, a valid demonstration that TAVNEOS is effective for its approved use.

Ohhh boy. This is about as bad an accusation as you can make about a clinical trial, i.e. “The unblinded data were ugly, so we hocused the numbers until it looked like the drug worked”. I occasionally meet uninformed cynics who assume that this is how we always do things in the drug industry, but oh no, we don’t. We have an 85% failure rate in the clinic! Why would any clinical trial ever fail if we had constant recourse to bullshit like this?

For more details, this FDA document at the Federal Register is the place to go. This all came to light due to a lawsuit against the company for securities fraud, which included a consultant’s report about the avacopan/Tavneos trial process. That all came about because during the initial approval process ChemoCentryx made numerous public statement about how straightforward the trial was and how uneventful their interactions with the FDA had been, but in May of 2021 the FDA review committee hearing instead detailed a whole list of pointed questions the agency had had about the trial design and the interpretability of the results. That sent the stock down about 80%, and that will get you a shareholder lawsuit every time.

But as it turns out, the hapless shareholders don’t seem to have known the half of it. The consultant report introduced as evidence during the lawsuit claims that the initial analysis of the clinical trial showed it missing the primary endpoint, and that the company picked a number of cases for “readjudication”. Wouldntjaknowit, enough of these flipped over to positive during this reanalysis to cause the whole trial to meet its statistics. The report says that ChemoCentryx employees stated as much even before the reworking, calculating how many patient outcomes would need to be flipped. But none of this was disclosed in any way to the FDA, obviously, and yes, that is all flagrantly illegal if it’s what happened. 

The FDA says that it requested a detailed account of the data handling for the trial, and that Amgen’s response a month later “confirmed the key factual allegations” above. But the company went on to claim that the data in the NDA are accurate and that the patient readjudications were appropriate. (As it turns out, the lawsuit was later dismissed without ever addressing these accusations, so it doesn’t have any bearing on this situation).

The case for the data changes being valid rests largely on patient glucocorticoid dosing or missing data, and I won’t get into the merits of that argument. But what seems beyond doubt is that ChemoCentryx made sure that the FDA never heard about it and made sure to submit only the freshly polished data set. Such readjudication-after-unblinding was, as you would imagine, absolutely not permitted under the study protocols. The FDA notes that one of the patients was initially marked down by ChemoCentryx as a non-responder due to missing data at week 26, but that same patient got helpfully moved to the “in remission” category after the unblinding. That’s precisely why you are not supposed to do that sort of thing.

There’s even more bad news: not only are there doubts about the efficacy, the safety profile is looking bad, too. The FDA has received numerous reports of liver toxicity, which was a concern even during approval (Tavneos already has a label warning to that effect). But since getting on the market there have been several fatalities, some of which involve the extremely-alarmingly-named “vanishing bile duct syndrome”, which I had never heard of until now. It’s just what it sounds like: the bile ducts through the liver tissue deteriorate and disappear, and that is clearly just as bad as you think it is. Most of those have been reported in Japan, for reasons unknown, but man, you don’t want that showing up anywhere. I feel pretty sure that a “Whatever happened to the bile ducts” finding during the trials would have shut things down right there. One has the impression that Amgen might have been better off if they'd never heard of this drug at all instead of being shackled to defending it.

The drug was approved with a requirement for a postmarketing study covering both safety and efficacy, but the FDA says that as of the most recent report, only 21 of the planned 300 patients had been enrolled. I’ve written about this problem before - too often, companies have an incentive to draaaaag their feet on these requirements and collect the data at glacial speed, and that may be just what we’re seeing here.

But as it stands, I would agree with the FDA’s contention that the trial results “are uninterpretable and cannot be salvaged with further analysis”. What’s more, it would seem that the company made materially false statements to the agency during the application process. This is all really, really unfortunate - for vasculitis patients most of all, but for the integrity of the whole clinical trial and drug approval process as well. We really don’t need this sort of thing right now - hell, we never have, but especially not now.

Read the whole story
tabithaclem
30 days ago
reply
Share this story
Delete
Next Page of Stories