Maths Row Fuels Worry That AI Could Erode Trust in Science

A dispute between mathematicians has sharpened concerns that AI could undermine faith in the scientific process. Last month OpenAI said its AI agents had solved a great unsolved problem relating to the Navier-Stokes equations, in 88 hours. The body that runs the prize for solving it said the problem had "apparently been settled", but that its review would take time.

Hours before the announcement, Tristan Buckmaster of New York University went public with concerns. He had formed an independent team with Levent Alpöge, who works for rival firm Anthropic. They were chasing the same goal and kept drafts in OpenAI's coding tool, Codex. Buckmaster asked whether that work could have fed into OpenAI's own, while stressing he was "not accusing anyone of anything".

OpenAI said nobody at the company, human or AI, saw the other teams' work before publication. It first said it could not "rule out" that anonymised product data may have helped improve its models. Days later it said Buckmaster's Codex prompts from the two months before the announcement "could not have influenced the system in any way".

Journals under pressure

Science journalist Dalmeet Singh Chawla, who has reported on research misconduct for a decade, says generative AI has "basically disrupted the whole scientific process". Publishers' integrity staff will need to grow "drastically" to keep up, he says, and right now "it seems like they're not".

Prof Dmitry Kobak of Ghent University searched more than a million biomedical papers for words favoured by language models. He found that by last December nearly 90% showed AI fingerprints. Detection software is imperfect: some tools have labelled the US Constitution "AI-generated". A Stanford team led by Weixin Liang and James Zou found up to 17% of text in some computer science conference reviews had been substantially modified by a language model. A Cornell team found AI-compiled citations included around 147,000 pieces of phantom research last year.

Publishers are responding. Elsevier's research integrity team uses some 100 signals to flag papers for human scrutiny. Nature's journals ban AI-generated images and tell referees never to upload a manuscript to an AI system. The arXiv site bans researchers for a year if they submit invented references.

Successes and doubts

Google DeepMind's AlphaFold predicted 200 million protein folds in a year, and its head and lead scientist won the Nobel Prize for Chemistry. Its later materials work drew challenge. DeepMind said it had found more than two million new crystal structures. Two University of California chemists, Anthony Cheetham and Ram Seshadri, sampled them and found "yet to find any strikingly novel compounds". DeepMind rejects the charge, saying it tested each compound for novelty.

Dr Pushmeet Kohli of DeepMind says turning away from AI would be like computing without calculators, but accepts that "somebody has to take accountability". Prof Andrea Pauli of the Vienna BioCenter used AI responsibly to help find a protein on zebrafish eggs that points to a lock-and-key mechanism for sperm entry.

Source: BBC News