Each time a brand new AI mannequin is launched, it’s sometimes touted as acing its efficiency in opposition to a sequence of benchmarks. OpenAI’s GPT-4o, for instance, was launched in Might with a compilation of outcomes that confirmed its efficiency topping each different AI firm’s newest mannequin in a number of exams.
The issue is that these benchmarks are poorly designed, the outcomes onerous to duplicate, and the metrics they use are regularly arbitrary, in response to new analysis. That issues as a result of AI fashions’ scores in opposition to these benchmarks decide the extent of scrutiny they obtain.
AI firms regularly cite benchmarks as testomony to a brand new mannequin’s success, and people benchmarks already kind a part of some governments’ plans for regulating AI. However proper now, they won’t be ok to make use of that means—and researchers have some concepts for a way they need to be improved.
—Scott J Mulligan
We have to begin wrestling with the ethics of AI brokers
Generative AI fashions have grow to be remarkably good at conversing with us, and creating photographs, movies, and music for us, however they’re not all that good at doing issues for us.
AI brokers promise to alter that. Final week researchers revealed a brand new paper explaining how they educated simulation brokers to duplicate 1,000 folks’s personalities with gorgeous accuracy.
AI fashions that mimic you might exit and act in your behalf within the close to future. If such instruments grow to be low cost and simple to construct, it would elevate plenty of new moral considerations, however two specifically stand out. Learn the complete story.
—James O’Donnell
