There’s been a persistent belief in artificial intelligence (AI) practitioners that kind of explains the ongoing development. The mantra is “if we can understand it, it’s no longer AI”. This is relevant to what we’re seeing today, in important ways. So let’s unpack that.
The point of AI is that we’re trying to model intelligent behavior, usually human intelligence. It doesn’t have to be human, of course, we can be trying to model intelligent behavior that’s not human! However, at least for much of AI, we’re trying to understand our own thinking. After all, the human brain is arguably the most complex thing in the known universe! For instance, we really still don’t understand consciousness; why does it even exist? And, what is it really? We’re getting closer, but we’re not really there.
I’ve mentioned before that machine learning came about as a reaction to the failure of symbolic approaches to capture some of the nuances of human behavior. For instance, recognizing images in noisy environments is something we were better at than systems. Symbolic systems were ‘brittle’, in that they did what they did, but failed as soon as you tried to generalize. The neural net approach, where we trained systems, more closely mimicked how we ourselves perform. And, really, the general trend has been that as soon as we understand it, it’s no longer AI, it’s just good programming. Which is fine. We’ve advanced our understanding, and now we can build it.
What we encode in the weights of networks is essentially what’s known by the system. We don’t really know what we know, and we create models to explain how things work. We can get more complex with non-linear models (which our brains struggle with), but non-linearity itself is an understanding we’ve created to explain how the world works. Yet, what we actually capture in our networks isn’t available for scrutiny. Same with neural nets. Most networks were small, relatively speaking (massive, but small compared to our brain). They were also essentially limited. Trained on X, or Y, so check recognition or loan approval, but our brains can handle much more.
We’re seeing now that while the algorithms have made some modest improvements, we’ve vastly scaled the net size to achieve large language models (LLMs). However, we’re still running limited foci. So, for instance, language (training on the text of the internet), or images, or video, or music. Yes, there are systems that are integrating, but as far as I know (which might be wrong), they’re really grafted and not integrated completely. Most folks I know go to different tools for different things.
The result is that, particularly in text, we now have the general experience that we can ask questions of LLMs, and get meaningful replies. And, make no mistake, this is useful. For general language tasks, such systems are great. They’re doing pretty great jobs of summarizing prose. This passes the so-called “Turing Test”. (Alan Turing posited that if you had a system and interacted with it so that you couldn’t discriminate between system and human, it passed the test.) An article now suggests that a reasonable assessment would say we’ve achieved artificial general intelligence (AGI).
Apparently, I’m not a reasonable person. They’ve addressed my complaint, they posit. The article (in the parts I could read before the paywall) say that such systems do, indeed, have a world model I argue that such systems don”t really know what a ‘dog’ is, having never seen one, petted one, raised one, walked one. Read prose describing all, yes. Experienced it? No. To me, there’s a difference (and I’m channeling Stevan Harnad’s Symbol Grounding Problem).
I also believe that there’s an AI architecture that’s more likely to achieve AI. (Particularly if we give it sensors to experience the world; and imagine what progress if they got the same hype as LLMs!) LLMs are trained on the vast corpora of text (illegitimately, I’ll suggest), and so understand an average that’s greater than any one person. (They’re also trained on other data, like video or images, for other media.) But they’re not as good as an expert in the field. They can do impressive things in areas you aren’t an expert in, but an expert in those areas can point out flaws. Yes, they can solve unsolved problems in, for instance, math, and pass tests, but those are knowledge tests, not ‘do’.
Ultimately, I don’t have to agree with the consensus, and maybe I’m just a grumpy old man (and get off my lawn!). I would like the progress we’ve made and what we can do with LLMs, if it weren’t for the hype that’s distorting business and mindspace. AI is more than LLMs, despite the way folks are speaking, and the valuations aren’t real. Also, we now know what they’re doing, and how, so I can say “it’s no longer AI” and mean it ;).