This AI Model Never Stops Learning

Great modern language The models (LLM) can write beautiful sonnets and an elegant code, but they even lack rudimentary capacity to learn from experience.

Massachusetts Institute of Technology (MIT) researchers have now designed a way for LLMs to continue to improve by refining their own parameters in response to new useful information.

Work is a step towards construction artificial intelligence Models that learn continuously – a long -standing goal in the field and something that will be crucial if the machines are increasingly immersing human intelligence. In the meantime, this could give us chatbots and other AI tools which are better able to incorporate new information, including the interests and preferences of a user.

The MIT scheme, called Auto Adaptation Language Models (SEAL), implies that an LLM learns to generate its own synthetic training data and update the procedure according to the entry it receives.

“The initial idea was to explore if the tokens [units of text fed to LLMs and generated by them] Could cause a powerful update of a model, ”explains Jyothish Pari, a doctoral student involved in the development of seal. Bet says that the idea was to see if the release of a model could be used to train it.

Adam Zweiger, a first cycle researcher of MIT involved in Building Seal, adds that, although the new models can “reason” their path to better solutions by performing a more complex inference, the model itself does not benefit from this long-term reasoning.

The seal, on the other hand, generates new perspectives and then folds it in its own weights or parameters. Given a statement on the challenges encountered by the Apollo space program, for example, the model has generated new passages that try to describe the implications of the declaration. The researchers compared this to the way a human student writes and examines notes to help their learning.

The system then updated the model using this data and tested to what extent the new model is able to answer a set of questions. And finally, it provides a learning to strengthen Signal which helps guide the model towards updates that improve its global capacities and which help it to continue learning.

The researchers tested their approach on the small and medium versions of two open source models, Meta Lama and Alibaba Qwen. They say that the approach should also work for much larger border models.

The researchers tested the seal approach on the text as well as a reference called the arc which measure the capacity of an AI model to solve problems of abstract reasoning. In both cases, they saw that the seal allowed the models to continue learning far beyond their initial training.

Pulkit Agrawal, professor at MIT who supervised the work, says that the SEAL project addresses the important themes of AI, including how to make AI to understand what it should try to learn. He says it could well be used to help make the AI ​​models more personalized. “The LLMs are powerful, but we don’t want their knowledge to stop,” he said.

The seal is not yet a way for AI to improve indefinitely. On the one hand, as Agrawal notes, the LLMS tested suffer from what is called “catastrophic forgetfulness”, a disturbing effect observed when the ingestion of new information causes the disappearance of older knowledge. This may indicate a fundamental difference between artificial neural networks and biological networks. Bet and Zweigler also note that the seal is with a high intensity of calculation, and it is not yet clear on the best way to plan the most effectively new learning periods. A fun idea, says Zweigler, is that, like humans, perhaps the LLM could experience “sleep” periods when new information is consolidated.

However, for all its limits, Seal is a new exciting path for a more in -depth research of AI – and it is perhaps something that is found in future models of border AIs.

What do you think of the AI ​​who can continue to learn? Send an e-mail to hello@wired.com to let me know.