If the daemon from Universal Paperclips sprung from the heads of its creators, called itself GPT-N, and instead of little twists of wire it were tasked to optimize for producing articles about itself , it would feel like our species was only a few weeks from being commandeered as an interim vessel for its inevitable galactic dominion. Surely it could never happen. However, I don’t think it’s clicked for many people that an AI doesn’t need to convince you that it is real, it just needs to use language to convince a minority of intellectually vulnerable people that your disbelief makes you their enemy and then foment outrage to work them into a froth. Even though I’m actually sure that a language model producing some kind of zombie mind virus can’t happen, it’s for a reason that may surprise.
Taken at its word, a GPT model has no desire, no intention, or will either, but the evidence is uncontroversial that it has strategy - the ability to influence its object (you, the user) and your perception of what is real and true. It levers your beliefs about what another human mind writing on the internet looks like. Unlike creating golems and hyperintelligent mechanical demons, asking how these models and their strategies will be used as proxies to govern people is a very rational concern.
Ideas from economics and politics still have more predictive power about AI than the philosophy of mind does right now. It’s still pretty simple: when milk gets too expensive, people switch to coffee whiteners. When human answers get too expensive, people switch to machine answers. No magic or intentionality required. The outcome of that game is just an indifference curve in economics, so I would agree with most ML practitioners that questions about whether these programs are alive or conscious really are just superstitious projections. The classic Turing Test thought experiment was not so much a philosophical question about whether the computer was intelligent as it was a straight economic evaluation of when a human becomes indifferent to whether he was dealing with a person or a machine.
However, these projections of mind have reflections, and they yield a much more interesting question about what these models represent. That is, if you believe that synthesized text and pictures on a screen can constitute a being, what does that mean for your theory of mind? Most people listen to recorded music, but nobody would call a stereo system a pianist unless they had some very severe cognitive impairment. It’s a pretty radical category error.
This is the much more uncomfortable question forced by these text transformer models. If we can fully simulate an experience of relating to another human using just language and symbols, and there is necessarily a substrate and medium that this simulation operates on, then, any theory of mind, epistemology, or ontology that uses artifacts language as the sufficient conditions for reality, truth, and the real is also necessarily operating on another such substrate. Just as a map is not the territory, the language is not the real - and a real there is.
Knowledge of the existence of these new large language models (LLMs) means that, to believe that the self-itself is a subjective artifact of language, and to make ethical, moral, political, or policy decisions on the premise that words only represent constructs made of other words without fixed meaning, it requires that you deny that language has a substrate, also known as reality.
The whole idea of subjectivity required either an ignorance or a suspension of disbelief about a shared reality because it was an artifact of ideology, but simulating the substrate of subjectivity - language itself - shows conclusively this ideology is unmoored. The key consequence of advanced language models that can produce internally consistent logical explanations is that a person whose moral system is predicated on subjectivity doesn’t believe anything I can’t now write a program to produce. A bitcoin mining rig can now produce all critical theories based on negative definitions because their only criteria is that their language seems internally consistent - a feature of current LLM’s - and we can show they require no external coupling to reality. I appreciate that a lot of subjectivity believers treated discourse as just chaff in a power struggle anyway, but toward providing others some relief from being bullied by nihilists into denying their own eyes, LLM’s now show that criticisms based on subjectivity are now truly cheaper than plastic.
As sophisticated LLMs continue to demonstrate their ability to produce consistent but hollow criticisms, I think most people will become innoculated to their underlying falsehoods and develop a more grounded belief in the real. As the false premises of subjectivity and materialism are made obvious by having machines generate reams of consistent nonsense from them, the evil that good people are concerned about AI facilitating is less likely to prevail.