Modelling the Self 7: Consciousness, Suffering and Moral Status

WhetherWhether ethical treatment requires certainty about experience, and what the treatment of animals suggests about acting under uncertainty

The previous page argued that proof of consciousness may be unavailable even in principle, and that the realistic question is one of graded confidence. This page takes up the consequence. If we cannot prove that a system is conscious, does ethics require us to wait for proof before treating it with any care? Or can we act sensibly while the question stays open?see the ethics page for the stakes

The question is best put as "How should we act under uncertainty?" rather than as "Can we prove that a system is conscious?" The second question has no answer at present. The first has answers, and we already use them in other domains.

Sentience and Sapience FoundationalKnowledge that endures for decades — core principles

TwoTwo capacities are easily run together, and the distinction between them is central to the ethical question.

Sentience is the capacity for subjective experience, and in particular for pleasure, pain, suffering, and wellbeing. A sentient being can be harmed in a way that matters to it, whatever else it can do.

Sapience is the capacity for reasoning: planning, abstraction, reflection, and problem solving. A sapient being can do these things, but the capacity by itself does not tell us whether anything is felt.

The two come apart in both directions. A highly sapient entity need not be sentient, and a sentient entity need not be highly sapient. A system that plans well and explains its plans has sapience in the relevant sense, and nothing in that establishes sentience. A creature with a simple nervous system may be sentient without any of the reflective abilities we associate with intelligence. The distinction is useful in debates about animal welfare, where it has helped separate questions about what an animal can do from questions about what it can suffer.the stakes: if it suffers, we have duties to it

The series has been concerned throughout with sapient capacities, such as self-modelling, metacognition, and reflective access. The ethical question depends on sentience. A capable self-model does not, on its own, bear on whether anything suffers.

Lessons From Animal Ethics Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

HumanHuman societies rarely have proof of consciousness in the animals they treat in particular ways. They work instead from behavioural evidence, from evidence about the nervous system, from evolutionary relationships, and from precaution. The Cambridge Declaration on Consciousness reflects that approach. It draws its conclusion about non-human animals from convergent evidence about their neurological substrates, and it does not wait for a demonstration of experience[4].

The principle that emerges is that the possibility of suffering matters even when certainty is unavailable. In practice, protections against avoidable pain do not usually wait for proof that a given animal feels pain. The burden of justification sits with those who would impose the harm.

The analogy to machines has limits, and the limits are instructive. Animals are biological organisms with evolutionary histories that give us some grounds for expecting experience. A machine has no such history, and the structural evidence we would use for an animal does not carry over directly. The lesson is therefore not that the same conclusions should be drawn about machines. It is that the reasoning pattern, which weighs evidence, places the burden of justification, and acts under uncertainty, is available for machines too.the ethical stakes of building such systems

The Precautionary Principle Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

PrecautionPrecaution is usually stated as a single rule, but it works better as a spectrum. The obligations that follow from a given level of confidence can be set out in steps.

The table is this page's own proposal, not a standard that any body has adopted. Its main point is that ethical obligations may increase well before certainty is reached. A team need not wait for proof to begin monitoring. It needs a way of deciding what evidence would move it to the next level, and it needs to make that decision in advance of the moment it matters.

The principle has two costs. First, it is only as useful as the confidence it is applied to, and at present that confidence cannot be estimated reliably for any AI system. Second, precaution can misdirect attention. A framework that treats every capable system as a possible patient may draw resources from harms that are certain, such as the misuse of systems against people, and it may obscure who is responsible for those harms.hard to set thresholds without recognition criteria

Model Welfare in Context Applied / MethodologicalKnowledge with a 5–10 year half-life — stable practice

ThisThis is where discussion of model welfare naturally belongs, and the previous page on that topic describes the reported work in more detail[the model welfare page]. The relevant question is not whether an AI laboratory has proved that its systems are conscious. It is whether that laboratory is responding to a non-zero possibility that future systems could have morally relevant experiences. On the table above, a research programme on welfare sits at the monitoring level, and the page can say so without claiming more.see the welfare page for the monitoring scale

The connection to the earlier pages is direct. Uncertainty about experience is what makes the self-model question ethically serious, and the table shows how serious it becomes as evidence accumulates. The questions the series raised as speculation become practical when a system's self-reports, internal representations, or structural features move the confidence level up the scale.

Human Responsibility FoundationalKnowledge that endures for decades — core principles

AA critical point should not be lost. Even if machines became moral patients, they would remain human artefacts. They would be designed, trained, deployed, and maintained by people and institutions, and questions of responsibility and accountability would remain with those people and institutions.

This matters for the welfare debate in two ways. A discussion that attributes moral standing to a system can make it harder to see the decisions that shaped it, including what it is trained to say, what it is permitted to do, and what oversight it receives. And a claim that a system may suffer is a reason for the people responsible for it to take more care, not a reason for them to hand that responsibility to the system.the duty to design with care

Conclusion FoundationalKnowledge that endures for decades — core principles

EthicsEthics does not require proof before it applies. It requires a clear account of the evidence that would change our obligations, and a willingness to act on that account while the evidence is incomplete. The sapient capacities that the series has examined do not, on their own, establish sentience. But the question of sentience cannot be dismissed on that ground, because the stakes of being wrong are uneven. The appropriate response to a question we cannot yet answer is careful investigation, ethical caution in proportion to the evidence, and a clear statement of who is accountable for what is built.

Related Topics

• The Ethics of Building Something That Might Model Itself Into Suffering — the moral-status argument that this page places in the wider context of uncertainty.
• Modelling the Self 5: From Thought Experiment to Model Welfare — the contemporary discussion this page uses as its example.
• Modelling the Self 6: Can We Recognise Consciousness? — the epistemic question on which the ethical one depends.

References

1. P. Low, J. Panksepp, D. Reiss, D. Edelman, B. Van Swinderen, and C. Koch, "The Cambridge Declaration on Consciousness", proclaimed at the Francis Crick Memorial Conference, Cambridge, 7 July 2012.