Meanwhile on the AI scene. . .

I read recently where many researchers are now convinced that to achieve a more humanlike AI, or artificial general intelligence (AGI), will require something more than merely mastering language and images.  It will, in their estimation, require AIs which can reason about space, causality, and the consequences of actions.  While this is more broadly necessarily for the evolution of AI, it is especially required if AI is to control humanoid robots, operate factories, and explore other planets—fulfilling at least some of the promise of those visionaries who laud its future.

Among those who argue for this need, AI pioneer Yann LeCun put it clearly. “I joke that the smartest systems we have today are not as smart as a house cat,” he says. A cat can’t code like a large language model (LLM), but it can survive by its wits. The notion that simply scaling ever larger an LLM will get to AGI is “complete nonsense,” he says. “It’s like saying you’re going to get into orbit by scaling airplanes. There’s a very powerful delusion circulating in Silicon Valley that this is the case.”  LeCun left a top job at Meta to co-found one of a growing number of labs and startups developing “world models”—systems that build representations of how the world works—and agents that operate within them to learn or plan. Ultimately, these researchers hope that more closely mimicking how the human mind learns will give AI stunning new powers. 

There is another problem.  Companies  are spending hundreds of billions of dollars a year to build AI models with hundreds of billions or even trillions of adjustable parameters, trained on trillions of words and images scraped from the internet.  The latest GPT model from OpenAI is several times the size of its largest 2020 model and trained on an order of magnitude more data.   So LLMs can pass lawyers’ bar exams and doctors’ medical licensing exams, match the achievements of top high school students in math, write poetry that many readers find more beautiful than human works, and do such an efficient job that top programmers now use them to write most of their code.  However, each advance in AI requires disproportionately more resources.  They are running out of input.  Literally.  One 2024 study estimated they’ll exhaust high-quality public text data in the next few years.  The equipment (computer chips and algorithms are gaining efficiency), but not quickly enough and data centers under development will each draw so many gigawatts of power and water that they are straining the grid and turning the minds of the public against them.

In other words, building a better human may be even more difficult that improving the ones we have.