“To be very honest with you, we have hypotheses, but we don’t know exactly why this works.”
Fun Times
DeepMind claims that for the first time, an AI has solved a famously difficult math problem with a solution that eluded human mathematicians — which could be huge if it holds up to scrutiny.
In interviews with MIT Technology Review and The Guardian, Google DeepMind researchers waxed prolific about their new AI tool, which they claim has generated a brand new solution to what’s known as the “cap set problem,” which involves plotting more and more dots without any of them ever forming a straight line.
The novel findings, which the researchers announced in a paper published in the journal Nature, would mark the first time AI has made a unique scientific discovery which, because it was previously unknown, was not part of its training data. That would be a pretty big deal considering that AI is known for conjuring up nonsense and made-up junk even when its training data has the right answers.
DeepMind built the tool in question, called “FunSearch” in reference to mathematical functions (and not the other kind of fun) on the back of its AlphaZero AI, which solves math problems as if it were playing a game. The LLM it uses is called Codey, which is trained and honed on computer code and programmed to reject incorrect answers and feed correct ones back into its model.
No Known Answer
Feeding code into an AI is one thing, but having it spit out a brand-new solution to a famous puzzle — even though it took a few days, as MIT Tech points out — is a different thing entirely.
“It’s not in the training data,” DeepMind research VP Pushmeet Kohli told the website. “It wasn’t even known.”
There is something of a mystical quality to what the DeepMind scientists are claiming: that the LLM managed to — just maybe — think for itself.
“To be very honest with you, we have hypotheses, but we don’t know exactly why this works,” DeepMind researcher scientist Alhussein Fawzi told MIT Tech. “In the beginning of the project, we didn’t know whether this would work at all.”
While there will obviously need to be lots more research to verify the claims and try to figure out exactly how FunSearch generated its novel solution to the cap set problem, its creators are clearly stoked.
“When we started the project there was no indication that it would produce something that’s genuinely new,” Kohli told The Guardian. “As far as we know, this is the first time that a genuine, new scientific discovery has been made by a large language model.”
