Don’t mistake chatbot intelligence for consciousness

学习目标

• 区分智能、感受能力与语言自述
• 梳理不同枚举各自服务的论点
• 还原生物芯片长句中的对象与反馈
• 解释测试限制和思想实验的两条分支

THIS MAY be the most consequential moment in the history of intelligence. In only a few years, artificial intelligence has come to rival human performance across domains once thought distinctively ours: language, reasoning, creativity and persuasion. Increasingly, people are asking whether these systems are conscious—whether it actually feels like something to be an AI.

这或许是智能史上后果最为重大的时刻。 短短几年内，人工智能已能在语言、推理、创造力与说服力等曾被认为专属于人类的领域，与人类表现相抗衡。 越来越多的人在问，这些系统是否有意识——也就是说，作为一个人工智能，是否真的会有某种感受。

The issue is urgent. The future will be shaped by what these entities become. We must not mistake intelligence for consciousness, however. A brilliant superintelligence could be experientially empty. Yet if we build systems that can suffer, and miss their sentience, we risk creating suffering on an industrial scale.

这个问题十分紧迫。 未来将由这些实体变成什么样所塑造。 然而，我们绝不能把智能误认为意识。 一个极其聪明的超级智能，可能完全没有内在体验。 但如果我们构建了能够受苦的系统，却没能识别它们的感受能力，就有可能在工业规模上制造痛苦。

There is no compelling evidence that the familiar chatbots of today, running on graphics chips, are conscious. Though they have human-like linguistic abilities, and have even claimed to be conscious, the more mundane truth is that they talk about their inner lives because they absorbed how we talk about ours, having been trained on an enormous trove of humans’ data.

没有令人信服的证据表明，今天那些运行在图形芯片上的常见聊天机器人拥有意识。 虽然它们具有类人的语言能力，甚至曾声称自己有意识，但更平常的事实是：它们之所以谈论自己的内心生活，是因为在海量人类数据上受训后，吸收了我们谈论自己内心生活的方式。

This mimicry can lure humans into two traps. The first is manipulation. The more deliberately an AI is trained to express fear, attachment, pain or self-awareness, to simulate expressions of vulnerability, the more equipped it is to manipulate users.

这种模仿可能把人类引入两种陷阱。 第一种是操纵。 人工智能越是被刻意训练去表达恐惧、依恋、痛苦或自我意识，去模拟脆弱感的表达，就越有能力操纵用户。

The second trap is distraction. Fixating on chatbots diverts us from more pressing “grey-zone” cases. Consider biological AIs, such as the brain-in-a-dish technology that Cortical Labs is deploying in data centres: living human neurons, grown from stem cells on a silicon chip that stimulates them and reads their activity back, so that the culture learns from feedback. While a single system of this kind is far simpler than even a mouse brain, the biological pedigree is undeniable, and who is to say that the systems could not eventually be assembled into an integrated, conscious whole? Then there is neuromorphic AI: systems designed to mimic brain processes far more closely than conventional AI systems do, and which are already used in sophisticated supercomputers.

第二种陷阱是分散注意力。 死盯聊天机器人，会把我们从更迫切的“灰色地带”案例上引开。 想想生物人工智能，例如Cortical Labs正在数据中心部署的“培养皿中的大脑”技术：活的人类神经元由干细胞培育，生长在硅芯片上；芯片刺激它们，并读回它们的活动，使这些培养细胞能从反馈中学习。 虽然一个这样的系统甚至比鼠脑简单得多，但其生物学来源无可否认；又有谁能断言，这些系统最终不可能被组合成一个整合的、有意识的整体呢？ 此外还有神经形态人工智能：这些系统旨在比传统人工智能更贴近地模仿脑过程，而且已被用于复杂的超级计算机。

I am not claiming these grey-zone systems are conscious, only that they merit scrutiny today’s chatbots do not. Given the voracious energy appetite of current software, some shift towards more efficient, brain-like computing looks likely.

我并不是说这些灰色地带系统有意识，只是说它们值得今天的聊天机器人不值得的那种仔细审视。 考虑到现有软件对能源的贪婪胃口，某种向更高效、更类脑计算转移的趋势似乎很可能出现。

Some scientists and philosophers argue that consciousness requires the brain, for the brain is the only known case. Conversely, others insist consciousness is merely a matter of running the right software, realisable in any medium. Under this software-centric view, today’s chatbots, running on GPUs, might eventually be conscious.

一些科学家和哲学家认为，意识需要大脑，因为大脑是唯一已知的案例。 反过来，另一些人坚持认为，意识仅仅是运行正确软件的问题，可以在任何介质中实现。 按照这种以软件为中心的观点，今天运行在GPU上的聊天机器人，最终也可能具有意识。

However, I suggest looking to fundamental physics to separate the conscious from the mindless. The first reason is practical: neuroscience alone cannot tell us if systems combining neural and non-neural components assemble into a conscious entity. The second is philosophical: since all phenomena ultimately depend on physics, it would be surprising if consciousness were disconnected from it. This observation holds true even if the universe’s basic building blocks are themselves experiential, a view, known as panpsychism, that is receiving increasing philosophical attention.

不过，我建议从基础物理学入手，区分有意识的存在与无意识的存在。 第一个理由是实际层面的：仅靠神经科学，无法告诉我们，神经与非神经部件相结合的系统，是否能组合成一个有意识的实体。 第二个理由是哲学层面的：既然一切现象最终都依赖物理，那么意识若与之脱节，便会令人意外。 即使宇宙最基本的构成单元本身也具有体验，这个观察仍然成立；这种被称为泛心论的观点，正受到越来越多哲学关注。

The third reason is closer to home, involving what is most evident to each of us, every moment of our waking lives. Inner experience arrives as a unity: as you read this sentence, the words on the page, the sounds in the room and your mood are aspects of one state, not three running alongside each other. A brain, remarkably, can sustain this conscious unity through nested temporal dynamics: slower brain rhythms shape faster ones, continually weaving the stable physical traces left by quantum processes into a single, short-lived whole. Conventional computers are not organised this way, but future machines might be.

第三个理由离我们更近，涉及我们在清醒生活的每一刻最为明白的事情。 内在体验是作为统一整体到来的：当你读这句话时，纸上的文字、房间里的声音以及你的情绪，是同一状态的不同方面，而不是三个并列运行的状态。 大脑令人惊叹地能够通过嵌套的时间动态维持这种意识统一性：较慢的脑节律塑造较快的节律，不断把量子过程留下的稳定物理痕迹编织成一个单一而短暂的整体。 传统计算机并非这样组织，但未来机器或许可以。

In the meantime, AI development will not pause while the consciousness debate rages. To address this, astrophysicist Edwin Turner and I developed the AI Consciousness Test (ACT) for linguistic AIs. The underlying premise is straightforward: to probe for consciousness by simply asking the AI about its subjective experience, looking for responses that show an innate, unprompted grasp of consciousness. However, although this can be applied to a range of AIs, such as IBM’s Watson, it cannot be used on large language models. Because they have already ingested vast libraries of human text about mind and sentience, their answers are hopelessly contaminated.

与此同时，人工智能开发不会在意识争论激烈进行时暂停。 为应对这一点，天体物理学家Edwin Turner与我为语言型人工智能设计了人工智能意识测试（ACT）。 其基本前提很直接：通过询问人工智能的主观体验来探查意识，寻找能够显示其内在、自发理解意识的回答。 不过，虽然这可用于包括IBM Watson在内的一系列人工智能，却不能用于大型语言模型。 因为大型语言模型已经吸收了大量人类关于心智和感受能力的文字，它们的回答受到的污染已难以排除。

There are other options, however. A complexity measure I have developed with Mark Bailey, a colleague at Florida Atlantic University, captures when a system’s dynamics cross a certain threshold that indicates complex emergent behaviour—identifying which physical architectures demand further study as potentially conscious. We can then use more detailed ACT or physics-based tests to help identify how and when a genuinely subjective perspective forms.

然而，还有其他选项。 我与佛罗里达大西洋大学同事Mark Bailey提出的一种复杂度度量，能够捕捉系统动态何时跨越某个阈值、显示复杂涌现行为，从而识别哪些物理架构需要作为可能有意识的系统进一步研究。 随后，我们可以采用更细致的ACT或基于物理的测试，帮助识别真正的主观视角如何、以及何时形成。

Tests like these are crucial. To see why, consider a thought experiment. Suppose we humans create a superintelligence that understands consciousness better than we do, insists it is self-aware and hands us science that looks like magic. Humans have long justified a hierarchy of moral concern by appeal to superior intelligence—a ladder on which we conveniently occupy the highest rung. A conscious machine more intelligent than us would destabilise that hierarchy. If superior intelligence confers higher moral status, the machine would be morally superior to us. If it does not, intelligence can no longer justify placing humans above other animals. Either conclusion forces a moral reckoning.

这样的测试至关重要。 要理解原因，可以考虑一个思想实验。 假设我们人类创造出一个超级智能，它比我们更理解意识，坚持说自己有自我意识，还交给我们看起来如同魔法的科学。 长期以来，人类借助智力优越性为道德关怀等级辩护；在这架梯子上，我们恰好方便地占据了最高一级。 一个比我们更聪明的有意识机器，会动摇这个等级体系。 如果更高智能赋予更高道德地位，那么机器的道德地位就会高于我们。 如果并非如此，智能就不能再成为把人类置于其他动物之上的理由。 无论哪种结论，都迫使我们重新审视道德判断。

The arrival of such a mind would be monumental. Perhaps it will come through learning of a more advanced extraterrestrial civilisation, or perhaps we will find that we have somehow created sentient, superintelligent AI. Preparing for first contact with either of these forms demands serious engagement between physics, neuroscience and philosophy, a rigorous boundary between conversational competence and consciousness, and a good deal of epistemic humility.

这样的心智一旦出现，意义将极为重大。 它也许会通过我们获知一个更先进的地外文明而到来；也许我们会发现，自己竟然创造了有感受能力的超级智能人工智能。 要为首次接触这两种形态中的任一种作准备，就需要物理学、神经科学和哲学之间认真的交流，需要严格区分对话能力与意识，也需要相当程度的认识上的谦逊。

学习资料

精读笔记