Faraday's Bold Claim: A Small AI Model Says It Beat Anthropic and OpenAI at Replicating Research, August 24, 2026

Faraday's Bold Claim: A Small AI Model Says It Beat Anthropic and OpenAI at Replicating Research, August 24, 2026 A London AI lab called Inherent Labs says its research agent Faraday beat Anthropic's Claude Opus 4.8 and OpenAI's GPT 5.5 at independently reproducing findings from published scientific papers, posting a 73 percent win rate on its own proprietary benchmark. Built on a comparatively small 27 billion parameter model and trained with a technique the company calls teaching research taste, the claim has not yet been independently verified, and open questions remain about possible data overlap with the model's training set. Hosted by Chris and Laura. The DX Today Podcast brings you daily deep dives into the most consequential stories in the AI ecosystem. #ArtificialIntelligence #AIResearch #MachineLearning #TechStartups #ScienceNews

Transcript

SPEAKER_00 0:00

Welcome to the DX Today Podcast, your daily deep dive into the AI ecosystem. I'm Chris, and joining me as always is Laura.

SPEAKER_01 0:07

Hey Chris, and today we're talking about a story that quietly landed this past weekend but has been generating a lot of chatter in research circles, involving a small London lab called Inherent Labs, and a very bold claim about beating the Giants at their own game.

SPEAKER_00 0:21

Okay, I need the setup here because beating the Giants is a phrase I hear a lot in this industry, and it rarely holds up under scrutiny. So walk me through who Inherent Labs actually is before we get to the claim itself.

SPEAKER_01 0:34

Sure, so Inherent Labs is a London-based outfit working out of a lab in King's Cross, founded by a group of deep mind alumni, including Edward Hughes as chief scientist, along with Lewis Kirsch, Koloyan Alexiev, and Tantum Collins, who previously worked on AI policy inside the Biden White House.

SPEAKER_00 0:52

That's an interesting mix of backgrounds, deep research pedigree, plus someone who's actually sat inside a policy shop watching how these systems get regulated in practice, which tells me they're thinking about more than just benchmark scores.

SPEAKER_01 1:05

Right. And structurally, they set themselves up as a public benefit corporation, which is its own small signal about how they want to be perceived. And they came out of stealth back in May of 2026 with a $50 million seed round.

SPEAKER_00 1:17

$50 million is a serious number for a team that's still only about a dozen people. So who's actually writing that check? And does the investor list tell us anything about how seriously the broader venture world is taking this?

SPEAKER_01 1:29

The round was led by Index Ventures with Radical Ventures also participating. And both of those firms have been fairly active in the AI research space. So it's not exactly a fringe bet. Even if the team itself is still tiny, currently around 12 people with plans to grow to somewhere between 20 and 25 by the end of the year.

SPEAKER_00 1:49

All right, so we've got the company and the money. Now give me the actual claim because that's what got this into our rundown today, and I want to hear it in plain terms before we start picking it apart.

SPEAKER_01 1:58

The headline is that their AI system, which they call Faraday, was tested on a task that's genuinely hard for these models, independently reproducing the findings of published scientific papers. And according to Inherent's own numbers, Faraday beat both Anthropics Claude Opus 4.8 and the GPT 5.5 model from OpenAI on that task.

SPEAKER_00 2:20

Reproducing a scientific paper sounds almost more demanding than writing new code from scratch, because you're not just generating something plausible, you actually have to match a specific verifiable outcome that someone else already published. So how did they measure that?

SPEAKER_01 2:34

They built their own benchmark called Replica, which draws on 310 separate tasks pulled from about 100 machine learning and AI for science papers, and on the subset of tasks that closely resembled what the model was trained on, Faraday posted a 73% win rate specifically against Claude Opus, 4.8.

SPEAKER_00 2:53

73% is a striking number on its face, but I have to ask the obvious question here. What kind of model is actually doing this work? Because if it's some enormous frontier system, then the story is less surprising than if it's something scrappy and small.

SPEAKER_01 3:07

That's actually the most interesting part of this whole story, because Faraday isn't built on some giant frontier-based model at all. It's built on Quen at roughly 27 billion parameters, which is genuinely small by the standards of what Anthropic and OpenAI are running these days.

SPEAKER_00 3:23

So we're talking about a model that's a fraction of the size of the systems it's supposedly outperformed, which either means their training approach is unusually effective, or it means the comparison isn't quite apples to apples, and I suspect the truth involves a bit of both.

SPEAKER_01 3:35

It's genuinely a bit of both. And to be fair to them, Faraday doesn't work entirely alone. It actually orchestrates that small open weight model alongside the GPT 5.5 codex tool from OpenAI specifically for the coding steps. So there is a hybrid architecture happening under the hood rather than one lone small model doing everything.

SPEAKER_00 3:55

That's a meaningful detail that a headline like small model beats the giants tends to leave out. Because if part of the pipeline is literally borrowing compute and capability from one of the very labs it's claiming to have beaten, that complicates the framing quite a bit.

SPEAKER_01 4:09

Exactly. And Hughes, the chief scientist, has actually leaned into a different framing anyway, saying the more interesting part wasn't necessarily the raw score, but the way they went about building the system, which used long horizon reinforcement learning paired with an auto-generated, rubric-based judge instead of standard fine-tuning or RLHF.

SPEAKER_00 4:29

Can you unpack that a little? Because research taste is a phrase I've seen floating around in coverage of this story, and it sounds like marketing language until you actually explain the mechanism behind it. So what are they claiming the model learned to do differently?

SPEAKER_01 4:43

The idea is that instead of just training the model to produce a correct final answer, they're training it to develop an instinct for which experiments are actually worth running in the first place, essentially teaching it judgment about experimental design, which is a skill that's historically been thought of as deeply human and hard to formalize.

SPEAKER_00 5:01

That's a genuinely ambitious goal because if you can actually train a system to have good scientific instincts rather than just good pattern matching, it starts to look less like a chatbot improvement and more like a step towards something that could eventually help discover new things rather than just verify old ones.

SPEAKER_01 5:18

And that's explicitly the long game they're describing. Positioning Faraday's groundwork toward AI that can eventually contribute genuinely new scientific findings, not just replicate results that are already sitting in a published paper somewhere, which is a much bigger and much harder claim than we beat a benchmark this week.

SPEAKER_00 5:37

Okay, now let's get into the skepticism. Because I think our listeners would be rightly annoyed with us if we just repeated a company's own press claims without pushing back on them. So what are the actual weak points in this story that a careful reader should be watching for?

SPEAKER_01 5:51

The biggest one is right there in the setup. Replica is inherent's own proprietary benchmark. They built it, they scored it, and as of right now, there has been no independent rerun of the comparison by any outside party. So the entire outperformance claim currently rests on Inherent's own internal scoring.

SPEAKER_00 6:07

That's not necessarily evidence of dishonesty. Plenty of legitimate research starts with a company's internal benchmark. But it does mean we should treat outperformed anthropic and open AI as a claim awaiting verification rather than an established fact, especially until someone outside the building reruns the comparison themselves.

SPEAKER_01 6:26

There's also a more technical concern floating around, which is the possibility of data overlap between the papers used in the replica benchmark and whatever was in Quen's original pre-training data, because if the base model had already seen material related to those papers, that could meaningfully inflate the apparent score.

SPEAKER_00 6:43

Right. And contamination is one of the oldest problems in benchmarking. And it's exactly the kind of thing that sounds like a minor technical footnote until you realize it can be the entire explanation for a surprising result. So that's a real question mark hanging over this whole announcement.

SPEAKER_01 6:57

To enhance credit, none of this seems to be something they're trying to hide. The caveats about contamination and the lack of independent verification are already circulating in the coverage, and a company that's serious about the science angle presumably understands that its credibility depends on eventually inviting that outside scrutiny.

SPEAKER_00 7:15

Let's zoom out for a second because I think the bigger story here isn't really about one benchmark score. It's about this pattern we keep seeing, where smaller specialized teams try to challenge the idea that only the labs with the biggest compute budgets can produce the most capable systems.

SPEAKER_01 7:30

That's exactly the tension worth sitting with because for the last couple of years, the dominant narrative has been that scale wins, bigger models, bigger training runs, bigger budgets, and stories like this one, even with all their caveats, keep chipping away at the idea that scale is the only lever that matters.

SPEAKER_00 7:48

It also raises an interesting question about what anthropic and open AI actually think about claims like this, because neither company has publicly responded in detail yet. And there's a real difference between staying quiet out of confidence versus staying quiet because a smaller competitor genuinely found a soft spot.

SPEAKER_01 8:06

We honestly don't know which one it is yet, and I think that's the honest answer rather than speculating. But it's worth remembering that both of those companies have far larger research teams whose full-time job is exploring exactly this kind of long horizon reinforcement learning approach. So a response, if there is one, will probably come through their own research rather than a public statement.

SPEAKER_00 8:28

What would actually convince you that this result is real and not just a well-marketed internal benchmark? Because I think that's the practical question our listeners should be asking themselves as they read coverage like this over the next few weeks.

SPEAKER_01 8:40

For me, it comes down to independent replication. If a neutral academic group or a rival lab reruns something like the replica tasks, ideally with contamination checks built in, and Faraday still holds up anywhere close to that 73% figure, that's the moment this stops being a press release and starts being a finding.

SPEAKER_00 8:59

I'd also want to see how it performs outside that and distribution subset of tasks, because the headline number was specifically on tasks that resembled what the model had already been trained around. And performance tends to look a lot less impressive once you push a system into genuinely unfamiliar territory.

SPEAKER_01 9:16

That's a fair distinction, and it's honestly one of the more responsible things about how this story has been reported so far. The coverage hasn't just repeated the headline number, it's actually flagged that in distribution qualifier, which suggests journalists are getting a little sharper about not just laundering a company's own marketing.

SPEAKER_00 9:34

Stepping back even further, and this fits into a broader theme we keep returning to on this show, which is that the frontier of AI progress isn't only about who has the biggest model anymore. It's increasingly about who can train the cleverest, most efficient system for a genuinely hard, narrow task.

SPEAKER_01 9:50

And funding rounds like this one are a signal too. $50 million from serious investors like Index Ventures and Radical Ventures for a 12-person team says that at least some of the smartest money in venture capital think specialized research agents are going to be a real category, not just a research curiosity.

SPEAKER_00 10:08

So where does this leave us practically? For people building products or doing research who are trying to figure out whether tools like Faraday are actually going to change how science gets done, versus just being another interesting but ultimately overstated benchmark story from a small lab trying to get attention.

SPEAKER_01 10:27

My honest read is cautious interest. The technique they're describing, teaching research taste through long horizon reinforcement learning with a rubric-based judge, is a genuinely interesting direction, regardless of whether this specific benchmark holds up. And it's worth watching whether other labs start adopting similar training approaches over the coming months.

SPEAKER_00 10:47

I'll add one more thing. The involvement of Tantum Collins with his policy background makes me curious whether Inherent is thinking ahead about how tools like this get regulated or scrutinized once they're actually being used inside real scientific institutions, rather than just treating this as a pure capabilities race.

SPEAKER_01 11:04

That's a great point to end the analysis on, because if these systems really are heading toward contributing original scientific discoveries someday, the questions about verification, reproducibility, and who gets credit for a finding are going to matter just as much as the raw performance numbers we've been discussing this whole episode.

SPEAKER_00 11:22

This actually reminds me of the Deep Seek moment from last year, when a comparatively cheap model out of China rattled confidence in the idea that only the best funded American labs could push the frontier forward. So I'm curious how similar or different you think this inherent story really is.

SPEAKER_01 11:39

It rhymes, but it's not identical because Deep Seek was mostly a story about training cost and general capability at massive scale. While this is a much narrower claim about one specific skill, reproducing research, so the stakes in the audience who actually cares about it are honestly pretty different.

SPEAKER_00 11:56

Let's talk about who actually gets affected if this keeps improving, because if an AI system genuinely gets good at independently reproducing scientific results, that starts to touch the daily work of graduate students and postdocs, whose job often includes exactly that kind of painstaking replication effort.

SPEAKER_01 12:12

I don't think it replaces those researchers anytime soon, but it could plausibly change what they spend their time on, freeing people up from some of the more mechanical reproduction work so they can focus on designing genuinely new experiments instead of rerunning someone else's already published results.

SPEAKER_00 12:28

So, what's the actual milestone you'd want to see next from Inherent? Something beyond just a bigger score on their own benchmark? That would tell us this technique is really maturing into something the rest of the field has to take seriously.

SPEAKER_01 12:42

I'd want to see Faraday tackle a task where nobody already knows the answer, attempting to replicate or extend a result that's still genuinely uncertain in the literature, because that's the real test of whether this research taste idea produces something more than a well-tuned reproduction machine.

SPEAKER_00 12:58

This is definitely one we'll keep an eye on, especially watching for whether anyone outside inherent actually reruns their benchmark. And if you want to dig into the primary reporting yourself, the original coverage lays out the founders, the funding, and the technical details in more depth than we could fit into one conversation.

SPEAKER_01 13:16

Agreed. And for now, the honest summary is a small, efficient model made a genuinely bold claim against two of the biggest names in the industry. The technique behind it is legitimately interesting, and the verdict on whether it actually holds up is still very much pending independent scrutiny.

SPEAKER_00 13:31

That's all for today's episode of the DX Today Podcast. Thanks for listening, and we'll see you next time.