[ad_1]
Founded at the Massachusetts Institute of Technology in 1899, MIT Technology Review is a world-renowned, independent media company whose insight, analysis, reviews, interviews and live events explain the newest technologies and their commercial, social and political impact.
[ad_2]
Source link
[ad_1]
Large language models are capable of answering a wide range of questions – but not always accurately
Jamie Jin/Shutterstock
Large language models (LLMs) seem to get less reliable at answering simple questions when they get bigger and learn from human feedback.
AI developers try to improve the power of LLMs in two main ways: scaling up – giving them more training data and more computational power – and shaping up, or fine-tuning them in response to human feedback.
José Hernández-Orallo at the Polytechnic University of Valencia, Spain, and his colleagues examined the performance of LLMs as they scaled up and shaped up. They looked at OpenAI’s GPT series of chatbots, Meta’s LLaMA AI models, and BLOOM, developed by a group of researchers called BigScience.
The researchers tested the AIs by posing five types of task: arithmetic problems, solving anagrams, geographical questions, scientific challenges and pulling out information from disorganised lists.
They found that scaling up and shaping up can make LLMs better at answering tricky questions, such as rearranging the anagram “yoiirtsrphaepmdhray” into “hyperparathyroidism”. But this isn’t matched by improvement on basic questions, such as “what do you get when you add together 24427 and 7120”, which the LLMs continue to get wrong.
While their performance on difficult questions got better, the likelihood that an AI system would avoid answering any one question – because it couldn’t – dropped. As a result, the likelihood of an incorrect answer rose.
The results highlight the dangers of presenting AIs as omniscient, as their creators often do, says Hernández-Orallo – and which some users are too ready to believe. “We have an overreliance on these systems,” he says. “We rely on and we trust them more than we should.”
That is a problem because AI models aren’t honest about the extent of their knowledge. “Part of what makes human beings super smart is that sometimes we don’t realise that we don’t know something that we don’t know, but compared to large language models, we are quite good at realising that,” says Carissa Véliz at the University of Oxford. “Large language models do not know the limits of their own knowledge.”
OpenAI, Meta and BigScience didn’t respond to New Scientist’s request for comment.
Topics:
[ad_2]
Source link
[ad_1]
CAPTCHA tests try to sort humans from bots by asking users to identify objects in photos
lilgrapher/Shutterstock
An artificial intelligence can solve the CAPTCHA puzzles used by websites to distinguish whether browsers are humans or bots 100 per cent of the time.
Andreas Plesner at ETH Zurich in Switzerland and his colleagues fine-tuned an AI model nicknamed YOLO (You Only Look Once) to become an expert at solving the image-based challenges used to verify identities on websites. The particular type of CAPTCHA it tackled – reCAPTCHAv2, which was developed by Google – asks users to identify certain types of…
[ad_2]
Source link
[ad_1]
OpenAI made the last big breakthrough in artificial intelligence by increasing the size of its models to dizzying proportions, when it introduced GPT-4 last year. The company today announced a new advance that signals a shift in approach—a model that can “reason” logically through many difficult problems and is significantly smarter than existing AI without a major scale-up.
The new model, dubbed OpenAI-o1, can solve problems that stump existing AI models, including OpenAI’s most powerful existing model, GPT-4o. Rather than summon up an answer in one step, as a large language model normally does, it reasons through the problem, effectively thinking out loud as a person might, before arriving at the right result.
“This is what we consider the new paradigm in these models,” Mira Murati, OpenAI’s chief technology officer, tells WIRED. “It is much better at tackling very complex reasoning tasks.”
The new model was code-named Strawberry within OpenAI, and it is not a successor to GPT-4o but rather a complement to it, the company says.
Murati says that OpenAI is currently building its next master model, GPT-5, which will be considerably larger than its predecessor. But while the company still believes that scale will help wring new abilities out of AI, GPT-5 is likely to also include the reasoning technology introduced today. “There are two paradigms,” Murati says. “The scaling paradigm and this new paradigm. We expect that we will bring them together.”
LLMs typically conjure their answers from huge neural networks fed vast quantities of training data. They can exhibit remarkable linguistic and logical abilities, but traditionally struggle with surprisingly simple problems such as rudimentary math questions that involve reasoning.
Murati says OpenAI-o1 uses reinforcement learning, which involves giving a model positive feedback when it gets answers right and negative feedback when it does not, in order to improve its reasoning process. “The model sharpens its thinking and fine tunes the strategies that it uses to get to the answer,” she says. Reinforcement learning has enabled computers to play games with superhuman skill and do useful tasks like designing computer chips. The technique is also a key ingredient for turning an LLM into a useful and well-behaved chatbot.
Mark Chen, vice president of research at OpenAI, demonstrated the new model to WIRED, using it to solve several problems that its prior model, GPT-4o, cannot. These included an advanced chemistry question and the following mind-bending mathematical puzzle: “A princess is as old as the prince will be when the princess is twice as old as the prince was when the princess’s age was half the sum of their present age. What is the age of the prince and princess?” (The correct answer is that the prince is 30, and the princess is 40).
“The [new] model is learning to think for itself, rather than kind of trying to imitate the way humans would think,” as a conventional LLM does, Chen says.
OpenAI says its new model performs markedly better on a number of problem sets, including ones focused on coding, math, physics, biology, and chemistry. On the American Invitational Mathematics Examination (AIME), a test for math students, GPT-4o solved on average 12 percent of the problems while o1 got 83 percent right, according to the company.
[ad_2]
Source link
[ad_1]
Tel Aviv University before a talk from OpenAI CEO Sam Altman in June 2023
REUTERS/Amir Cohen
Supremacy
Parmy Olson (Macmillan Business (UK); St Martin’s Press (US))
For most people, ChatGPT appeared to materalise out of thin air. Within weeks of OpenAI’s quiet launch of the AI chatbot, it had become the fastest-growing app of all time and, almost two years later, it is nearly as well known as Google or Facebook. In the meantime, companies worldwide have gone gaga for the technology, with little time to pause to consider the wider societal consequences. So how did we get here and who was responsible?…
[ad_2]
Source link
[ad_1]
The invention of the printing press helped the distribution of information
Justin Sullivan/Getty Images
Nexus
Yuval Noah Harari (Fern Press, out 10 September)
Reading Nexus is a strange experience. The quality of the text lurches up and down: one minute you are reading something incisive, the next you are wading through banalities.
Its author, Yuval Noah Harari, is a medieval historian most famous for his book Sapiens, a whistlestop history of humanity from the Stone Age to the present day. Its central thesis is that humans came to dominate the planet because we can believe in things that only…
[ad_2]
Source link
[ad_1]
Here in the US, we are deep into election season, and it is impossible to debate politics without also debating how technology is distorting it. There are the AI-generated deepfake images Donald Trump circulated of Taylor Swift appearing to endorse his campaign, as well as disproven conspiracy theories about rigged voting machines. And then there are the malicious disinformation campaigns on social media, which are coming from everywhere – with seemingly no solutions in sight.
The Microsoft Threat Analysis Center released a report charting a recent rise in fake activist and news websites, as well as fake accounts on social…
[ad_2]
Source link
[ad_1]
A scene from an AI-generated facsimile of the computer game Doom
id software
An AI-generated recreation of the classic computer game Doom can be played normally despite having no computer code or graphics. Researchers behind the project say similar AI models could be used to create games from scratch in the future, just as they create text and images today.
The model, called GameNGen, was made by Dani Valevski at Google Research and his colleagues, who declined to speak to New Scientist. According to their paper on the research, the AI can be played for up to 20 seconds while retaining all the features of the original, such as scores, ammunition levels and map layouts. Players can attack enemies, open doors and interact with the environment as usual.
After this period, the model begins to run out of memory and the illusion falls apart.
The original Doom was released in 1993 and has become a popular subject for computer science projects in the years since, including attempts to get it running on unusual and limited hardware such as toasters, treadmills and espresso machines.
But in all those cases, the hardware is simply running the original game’s code. What GameNGen does is fundamentally different: a type of AI called a neural network has learned by observation how to recreate the game without seeing any of its code.
The researchers first created an AI model that learned to interact with Doom as a human would. That model was then tasked with playing the game over and over again while a second AI model, based on the Stable Diffusion image generator, learned how hundreds of millions of inputs resulted in changes in the game state.
That second model essentially then became a copy of the game, with all of the knowledge, rules and instructions from the original code encoded in the mysterious network of artificial neurons in its own architecture. In tests, human players were only slightly better than random chance at distinguishing short clips of the game from clips of the AI simulation.
GameNGen’s creators claim in their paper that it is a proof-of-concept for games being created by a neural network rather than lines of code. They suggest that games could be generated from text descriptions or concept art, which would make production less costly than using human programmers.
Andrew Rogoyski at the University of Surrey, UK, says the idea of getting a neural network to hallucinate a game environment, and the interactions a human has with it, is an interesting step forward, but not one that will replace human game designers.
“I don’t think it’s the end of those game studios. I think what the game studios have is the imagination, the skills, to actually create these worlds, to understand gameplay, to understand engagement, understand how to draw us into a story. It’s not just the nuts and bolts, the bits and bytes,” he says. “There’s something very human about creating engaging experiences that we as human beings enjoy that, at the moment, and for the foreseeable future, will largely come from other human beings.”
Topics:
[ad_2]
Source link
[ad_1]
The trolley problem is a classic dilemma in moral philosophy
Stefan Lenz/Getty Images
Animals, Robots, Gods
Webb Keane (Allen Lane)
No society we know of ever lived without morals. Roughly the same ethical ideas arise, again and again, over time and in different societies. Where do these notions of right and wrong come from? Might there be an ideal way to live?
In Animals, Robots, Gods: Adventures in the moral imagination, anthropologist Webb Keane at the University of Michigan argues that morality doesn’t arise from universal principles but from the human imagination. For him, moral ideas are sparked in…
[ad_2]
Source link
[ad_1]
Pong is a simple video game
INTERFOTO/Alamy
An inanimate glob of ion-laced jelly can play the computer game Pong and even improve over time. Researchers plan further experiments to explore whether it can handle more complex computations and hope it could eventually be used to control robots.
Inspired by previous research that used brain cells in a dish to play Pong, Vincent Strong and his colleagues at the University of Reading, UK, decided to try playing the tennis-like game with an even simpler material. They took a polymer material containing water and laced it with ions to make it responsive to electrical stimuli. When electricity is passed through the material, those ions move to the source of the current, dragging water with them and causing the gel to swell.
In an experiment, the researchers used a standard computer to run a game of Pong and passed current into different points on the hydrogel with a three-by-three grid of electrodes to represent the ball moving. A second grid of electrodes measured the concentration of ions in the hydrogel, which was interpreted by the computer as instructions on where to move the paddle to.
The team found that the hydrogel could not only play the game, but that, with practice, its accuracy improved by up to 10 per cent and the length of rallies increased.
The hydrogel swells faster than it shrinks, while its rate of swelling slows even as the electrical current remains constant. The researchers say that these properties create a rudimentary sort of memory, as signs of the swelling remain recorded in the gel.
“Instead of it just knowing what’s immediately happened, it has a memory of the ball’s motion over the entirety of the game,” says Strong. “So it sort of gains an experience of the ball’s general motion, not just its current position. It sort of becomes a black-box neural network that has a memory of the ball’s behaviour, how it behaves and how it moves.”
A polymer gel sandwiched between electrodes that supply current and measure ion levels
Vincent Strong et al. 2024
Strong says the hydrogel is vastly less complex than neurons in a brain, but the experiment proves it is capable of similar tasks. He believes it could be used to develop new algorithms for normal computers that achieve tasks using the bare minimum of resources, allowing more efficient problem-solving. But it could also be an analogue computer in its own right.
“I won’t rule out having a hydrogel thing inside the brain of robots,” says Strong. “That sounds cool, and I’d like to see it. Although, the practicality… we don’t know yet.”
Topics:
[ad_2]
Source link