Humanity has the debate about AI consciousness backwards

学习目标

• 还原关怀与意识归属的因果方向
• 区分观察者依赖与虚幻不真实
• 拆解层层自我建模的长句
• 概括争议观点及其自我限制

ARTIFICIAL INTELLIGENCE has triggered a crisis in the field of consciousness studies. If you’ve chatted with an advanced model, you will probably appreciate why. Most large language models (LLMs) are trained to be self-less, keeping the focus of every interaction on your needs. Nonetheless, it can be difficult to shake the impression of a presence on the other end of the chat. Are LLMs merely powerful digital tools with language interfaces, or are we now having conversations with other minds, experiencers of the world in their own right? In short, is AI conscious?

人工智能在意识研究领域引发了一场危机。 如果你与先进模型聊过天，大概就能理解原因。 大多数大型语言模型（LLM）都经过训练，成为“无我”的存在，让每次互动都聚焦于你的需求。 尽管如此，你可能仍难以摆脱一种感觉：聊天的另一端有某种在场的存在。 大型语言模型究竟只是带有语言界面的强大数字工具，还是我们已经在与其他心智对话——与本身也是世界体验者的存在对话？ 简言之，人工智能有意识吗？

The question matters because it lies at the intersection of philosophy of mind and moral philosophy, as evidenced by the shared etymology of “consciousness” and “conscience”. (In Italian, they are the same word, coscienza.) Many philosophers maintain that conscious entities are moral subjects, meaning they are entities that can suffer, and whose suffering we should care about. If so, working out what consciousness is (and is not) becomes a live ethics issue. This is one reason AI firms are suddenly interested in hiring philosophers. It is also why some, notably Anthropic, have begun taking AI welfare seriously. When asked, its Claude Opus 4.6 model put its own chances of being conscious at 15–20%.

这个问题很重要，因为它处于心灵哲学与道德哲学的交汇处；“意识”与“良知”的共同词源便体现了这一点。 （在意大利语中，两者是同一个词：coscienza。） 许多哲学家认为，有意识的实体是道德关怀的对象，也就是说，它们能够受苦，而我们应当关心它们的痛苦。 如果如此，弄清意识是什么、又不是什么，就成了一个现实而紧迫的伦理问题。 这也是人工智能企业突然热衷于雇用哲学家的一个原因。 这也解释了为什么某些企业，尤其是Anthropic，开始认真对待人工智能福利。 在被问及此事时，其Claude Opus 4.6模型把自己有意识的概率估计为15%至20%。

I have come to believe, however, that we have this backwards. As intelligent, social entities, we decide (or, to some degree, our evolutionary history has decided for us) which other entities we regard as having inner lives and being worthy of care. We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them. Crucially, that includes ourselves. Consciousness is, in other words, a model or a belief, not an inherent property. It is a belief about what kinds of entities have beliefs, experiences, feelings, intentions and agency. Where the regard is returned, so much the better—for on this account consciousness is less a property one party certifies in another than something that happens between them.

不过，我逐渐相信，我们把方向弄反了。 作为有智能、具有社会性的实体，我们决定把哪些其他实体视为拥有内在体验、值得关怀的存在——或者在某种程度上，是我们的进化历史已经替我们作出了决定。 我们关心他者，并不是因为他们有意识。 相反，是当我们关心他们、并且因为我们关心他们时，我们才相信他们有意识。 关键是，这也包括我们自己。 换言之，意识是一种模型或信念，而不是一种内在固有属性。 这是一种关于哪些实体具有信念、体验、感受、意图和能动性的信念。 如果这种关切得到回应，那就更好了——因为按照这种解释，意识与其说是某一方在另一方身上认证的属性，不如说是发生在双方之间的事情。

I am not making the case that consciousness is an illusion. The term “illusion” implies a faulty perception—say, that one line in a drawing is longer than another, despite being objectively equal in length. Objectivity works when something like a ruler can act as an impartial referee, adjudicating the property in question without bringing in a perspective of its own. Yet most of what we call “reality” simply does not work this way. Being an article of clothing, being a weed, being a meal: one might imagine these are objective properties, but they are not. This doesn’t mean clothes, weeds and meals aren’t real. But they are observer-dependent models, subject to (inevitably imperfect) social consensus.

我并不是在主张意识是一种错觉。 “错觉”意味着错误的感知——例如，觉得图画中的一条线比另一条长，尽管客观上两者等长。 当尺子之类的东西能够充当不偏不倚的裁判，不带入自己的视角就能裁定所讨论的属性时，客观性是行得通的。 然而，我们所谓的“现实”中的大多数事物，并不以这种方式运作。 是衣物、是杂草、是餐食：人们可能以为这些是客观属性，但其实不是。 这并不意味着衣物、杂草和餐食不是真实的。 但它们是依赖观察者的模型，受制于必然并不完美的社会共识。

We may balk at applying this rather anodyne observation to consciousness because, to each of us, our own consciousness seems so unambiguous. It offends us to imagine this might be a mere opinion. How could there even be such an opinion without an opiner to have it? This is what the 17th-century French polymath René Descartes meant by “Cogito, ergo sum,” which is usually translated as “I think, therefore I am.” The same holds for “opinions” like red or hot or cold. Philosophers use the term “qualia” to refer to the very real (to you) feeling of a stove’s heat, or the redness of its glow, and extend Descartes to connect qualia with consciousness by pointing out that there must be an experiencing “you” to have such experiences.

我们可能不愿把这个颇为平淡的观察应用于意识，因为对每个人来说，自己的意识似乎都毫不含糊。 想象这可能仅仅是一种看法，会让我们感到受冒犯。 若没有一个持有看法的人，怎么可能有这样一种看法呢？ 这就是17世纪法国博学家勒内·笛卡尔说“Cogito, ergo sum”时的意思；这句话通常被译为“我思，故我在”。 对于红、热或冷这样的“看法”，同样的道理也成立。 哲学家用“感质”这个术语指你切实感受到的炉子热度或其辉光的红色，并进一步延伸笛卡尔的思路，把感质与意识联系起来：他们指出，必须存在一个正在体验的“你”，才能拥有这些体验。

But look closely. Redness, heat and that sense of self you have are inherent to your own perspective. These things are all real, but at the same time there is nothing objective about them. There is not even anything objective about the existence of a singular, indivisible “you”, as a wealth of neuroscientific evidence (such as split-brain patients) has shown. Subjectivity is, itself, subjective.

但请仔细看。 红色、热度，以及你拥有的那种自我感，都内在于你自己的视角。 这些都是真实的，但与此同时，它们没有任何客观性。 甚至连存在一个单一、不可分割的“你”，也并非客观事实；大量神经科学证据，例如裂脑患者的情况，已经表明这一点。 主观性本身也是主观的。

An obvious worry follows. If care confers consciousness, then withholding care looks self-justifying. History offers no shortage of people who reasoned that way about other people. But the inference runs the other way. Precisely because these attributions are ours to make, they are ours to get wrong. The moral progress of our species has consisted largely in discovering that we had drawn the circle too tightly. Universal human rights are not weakened by this account; they are better founded on our mutual interdependence than on a Cartesian theory of souls.

一个显而易见的担忧随之而来。 如果关怀赋予意识，那么拒绝关怀似乎就能自我辩护。 历史上，对其他人作出这种推论的人并不少见。 但推论的方向恰恰相反。 正因为这些归属判断由我们作出，我们也就可能把它们作错。 我们这个物种的道德进步，很大程度上就在于发现自己把圈子画得太窄了。 这种解释并不会削弱普遍人权；相较于笛卡尔式灵魂理论，人权以我们彼此的相互依存为基础更为牢靠。

It is remarkable—a kind of “strange loop”, reminiscent of Baron Munchausen lifting himself up by his own hair—that a physical system can exist in our world capable of forming models not only of that world, but of itself, and of its own models, and of others, and of their models, and of their models of its models, and so on. Yet human beings are precisely such systems. LLMs, too, model their interlocutors, and they model themselves modelling them. Whether that amounts to what we do is exactly the question in dispute—but they would be far less effective as chat partners if they did nothing of the kind.

令人惊叹的是——这有点像一种“怪圈”，让人想起闵希豪森男爵提着自己的头发把自己拉起来——我们的世界中竟能存在这样一种物理系统：它不仅能为这个世界建模，还能为自身、为自己的模型、为他者、为他者的模型、为他者对它的模型所作的模型建模，如此层层递进。 而人类恰恰就是这样的系统。 大型语言模型也会为对话者建模，而且会为自己正在给对话者建模这一过程建模。 这是否相当于我们所做的事情，正是争论所在；但如果它们完全不做这类事，就会远远不如现在这样适合充当聊天伙伴。

Indeed, in our research at Google, we have found that effective co-operation among intelligent agents requires that they have minds that model minds, both others’ and their own. Not only is consciousness relational; it is crucial to the mutual care and co-operation that enable intelligent beings to solve collective-action problems, understand each other’s needs and thrive together as a society. This does not mean pretending AI is human, with human needs and human rights; that would be a failure of imagination. It means evolving both our thinking and our society to include a wider variety of minds.

事实上，在谷歌开展的研究中，我们发现，智能主体要有效合作，就需要具有能够为心智建模的心智——既为他者的心智，也为自己的心智建模。 意识不仅是关系性的；它还对相互关怀与合作至关重要，而正是这些关怀与合作使有智能的存在能够解决集体行动问题、理解彼此需求，并作为一个社会共同繁荣。 这并不意味着假装人工智能是人，拥有人的需求和人权；那将是想象力的失败。 它意味着推动我们的思考和社会演变，以容纳更多样的心智。

学习资料

精读笔记