AI companies like OpenAI and Anthropic are creating ever more specialized toolkits that package the capabilities of their agentic AI models for specific applications. Anthropic’s Claude, for one, now offers a coding assistant called Claude Code and a medical-focused toolkit called Claude for Healthcare. On June 30, Anthropic announced a new hat for its flagship agent to wear: that of a research assistant. Its new toolkit, Claude Science, claims to integrate the tools and packages scientists use into a single agentic offering. It can analyze and produce code, generate images—with a connector to figure tool BioRender—and suggest workflows for wet lab work.
But with scientific accuracy on the line, detail is everything. Scientists have started to deliver their verdicts on Claude Science, and a clearer picture is emerging of whether the tool delivers on its promise.
Faster, More Transparent Science
Anthropic says that Claude Science comes with more than 60 pre-configured “connectors” that can process workflows for genomics, proteomics, and structural biology, among other research fields. Stephen Francis, a neuroscientist at the University of California, San Francisco, was given early access to Claude Science by Anthropic. “It’s really my first experience with a truly agentic AI that was made for my field,” said Francis.
Francis has been using Claude Science as coding support. His neurogenetics studies involve analyzing massive genetics datasets to understand glioma, a type of brain cancer. Claude Science scraped through a milieu of obscure databases to identify supporting evidence for their genetic findings.
The AI then presented a list of possible next analysis steps, with referenced rationales for each idea. “Very often it’ll get my top three that I’m thinking of, and it’ll add a couple more on that I didn’t think of,” said Francis. Not all of the ideas the AI suggests are fruitful, he said. But working through ideas that reach dead ends is a normal part of the scientific process, and the AI helps Francis’s team work through them much faster.
Jerome Lecoq, an associate investigator with the Allen Institute in Seattle, was also given early access to Claude Science. Lecoq said he has been using the tool mainly to create literature reviews. These reviews can be assembled far more quickly than those written by humans, he said. Claude Science also clearly records how it derived each fact in a review and the prompts that led to the review’s generation. “You have much more information about provenance,” said Lecoq.
The reviews weren’t perfect, however. Claude Science occasionally made mistakes, requiring the researchers to carefully check them. And while the AI-led papers covered a much wider body of academic work, Lecoq said the AI was unable to access paywalled papers—a tension between AI and academic publishers that remains unresolved. Human reviewers, he said, are better than AIs at connecting disparate concepts that might not be contained in an AI’s training data.
Anthropic has adapted how Claude Science generates and presents information as compared to the regular version of the AI. Francis said that early on in the toolkit’s beta testing, researchers given early access requested a “replication” button which would review and check the AI’s own output with a skeptical eye, which led to noticeable improvements in accuracy. Kelvin Lau, a researcher at the Swiss Federal Technology Institute of Lausanne, said the tool was more transparent than regular Claude. “It really shows you step-by-step, ‘I am doing this. This is the code I’m using. Why am I doing this?’” said Lau. He added that Claude Science appeared to run more of its analysis locally on a user’s device, which he said was a boon for privacy-focused researchers, even if it meant computations taking longer.
Can AI Truly Think Like a Scientist?
Not all the toolkit’s features were optimized for science, however. Francis said that Claude refused to respond to some of his team’s prompts on herpesviruses and brain tumors, which is in line with the AI’s biosecurity protocols. Francis said he understood why Anthropic wanted to be careful, but he added that the guardrails are currently too stifling. “We’re not doing anything that’s biosecurity-related in the slightest,” he explained.
Researchers also said that the model needed more fundamental changes to its approach to critiquing data. In a human-led literature review, researchers will not only assess how many papers support or reject an idea, but the quality of each paper as well. Lecoq said he was able to instruct the AI to assess the worth of papers in this way to some extent, but he added that more work was needed in this area. “At some point, the reinforcement learning data that was used to train Claude or OpenAI models will have to incorporate the scientific methodology,” he said. Scientists are trained to question data in a way that AIs haven’t yet fully grasped. “It’s not about predicting the next token anymore. It’s about doubting the next token,” said Lecoq.
All three scientists said they would continue to use the tool. At the same time, all three had concerns about how the current scientific model would adapt to the widespread availability of AIs capable of writing papers and analyzing data far more quickly than previous methods. Lau also raised the point that as AIs evolve, grant-stressed lab heads might see them as a shortcut to quicker workflows, potentially at the expense of hiring new scientists. That kind of choice raises big questions about the future of research. “In an academic institute, is your mission more productivity or to teach the next generation?” said Lau.
