Tag: AI

The algorithms that underpin artificial intelligence systems like ChatGPT can’t learn as they go along, forcing tech companies to spend billions of dollars to train new models from scratch. While this has been a concern in the industry for some time, a new study suggests there is an inherent problem with the way models are designed – but there may be a way to solve it.

Most AIs today are so-called neural networks inspired by how brains work, with processing units known as artificial neurons. They typically go through distinct phases in their development. First, the AI is trained, which sees its artificial neurons fine-tuned by an algorithm to better reflect a given dataset. Then, the AI can be used to respond to new data, such as text inputs like those put into ChatGPT. However, once the model’s neurons have been set in the training phase, they can’t update and learn from new data.

This means that most large AI models must be retrained if new data becomes available, which can be prohibitively expensive, especially when those new datasets consist of large portions of the entire internet.

Researchers have wondered whether these models can incorporate new knowledge after the initial training, which would reduce costs, but it has been unclear whether they are capable of it.

Now, Shibhansh Dohare at the University of Alberta in Canada and his colleagues have tested whether the most common AI models can be adapted to continually learn. The team found that they quickly lose the ability to learn anything new, with vast numbers of artificial neurons getting stuck on a value of zero after they are exposed to new data.

“If you think of it like your brain, then it’ll be like 90 per cent of the neurons are dead,” says Dohare. “There’s just not enough left for you to learn.”

Dohare and his team first trained AI systems from the ImageNet database, which consists of 14 million labelled images of simple objects like houses or cats. But rather than train the AI once and then test it by trying to distinguish between two images multiple times, as is standard, they retrained the model after each pair of images.

They tested a range of different learning algorithms in this way and found that after a couple of thousand retraining cycles, the networks appeared unable to learn and performed poorly, with many neurons appearing “dead”, or with a value of zero.

The team also trained AIs to simulate an ant learning to walk through reinforcement learning, a common method where an AI is taught what success looks like and figures out the rules using trial and error. When they tried to adapt this technique to enable continual learning by retraining the algorithm after walking on different surfaces, they found that it also leads to a significant inability to learn.

This problem seems inherent to the way these systems learn, says Dohare, but there is a possible way around it. The researchers developed an algorithm that randomly turns some neurons on after each training round, and it appeared to reduce the poor performance. “If a [neuron] has died, then we just revive it,” says Dohare. “Now it’s able to learn again.”

The algorithm looks promising, but it will need to be tested for much larger systems before we can be sure that it will help, says Mark van der Wilk at the University of Oxford.

“A solution to continual learning is literally a billion dollar question,” he says. “A real, comprehensive solution that would allow you to continuously update a model would reduce the cost of training these models significantly.”

Can artificial intelligence be made to tell the truth? Probably not, but the developers of large language model (LLM) chatbots should be legally required to reduce the risk of errors, says a team of ethicists.

“What we’re just trying to do is create an incentive structure to get the companies to put a greater emphasis on truth or accuracy when they are creating the systems,” says Brent Mittelstadt at the University of Oxford.

LLM chatbots, such as ChatGPT, generate human-like responses to users’ questions, based on statistical analysis of vast amounts of text. But although their answers usually appear convincing, they are also prone to errors – a flaw referred to as “hallucination”.

“We have these really, really impressive generative AI systems, but they get things wrong very frequently, and as far as we can understand the basic functioning of the systems, there’s no fundamental way to fix that,” says Mittelstadt.

This is a “very big problem” for LLM systems, given they are being rolled out to be used in a variety of contexts, such as government decisions, where it is important they produce factually correct, truthful answers, and are honest about the limitations of their knowledge, he says.

To address the problem, he and his colleagues propose a range of measures. They say large language models should react in a similar way to how people would when asked factual questions.

That means being honest about what you do and don’t know. “It’s about doing the necessary steps to actually be careful in what you are claiming,” says Mittelstadt. “If you are not sure about something, you’re not just going to make something up in order to be convincing. Rather, you would say, ‘Hey, you know what? I don’t know. Let me look into that. I’ll get back to you.”

This seems like a laudable aim, but Eerke Boiten at De Montfort University, UK, questions whether the ethicists’ demand is technically feasible. Companies are trying to get LLMs to stick to the truth, but so far it is proving to be so labour-intensive that it isn’t practical. “I don’t understand how they expect legal requirements to mandate what I see as fundamentally technologically impossible,” he says.

Mittelstadt and his colleagues do suggest some more straightforward steps that could make LLMs more truthful. The models should link to sources, he says – something that many of them now do to evidence their claims, while the wider use of a technique known as retrieval augmented generation to come up with answers could limit the likelihood of hallucinations.

He also argues that LLMs deployed in high-risk areas, such as government decision-making, should be scaled down, or the sources they can draw on should be restricted. “If we had a language model we wanted to use just in medicine, maybe we limit it so it can only search academic articles published in high quality medical journals,” he says.

Changing perceptions is also important, says Mittelstadt. “If we can get away from the idea that [LLMs] are good at answering factual questions, or at least that they’ll give you a reliable answer to factual questions, and instead see them more as something that can help you with facts you bring to them, that would be good,” he says.

Catalina Goanta at Utrecht University in the Netherlands says the researchers focus too much on technology and not enough on the longer-term issues of falsehood in public discourse. “Vilifying LLMs alone in such a context creates the impression that humans are perfectly diligent and would never make such mistakes,” she says. “Ask any judge you meet, in any jurisdiction, and they will have horror stories about the negligence of lawyers and vice versa – and that is not a machine issue.”