Shutdown Problem

A shutdown problem is a one-agent two-decision SCIM containing a path D1(cid:57)(cid:57)(cid:75) H (cid:57)(cid:57)(cid:75) D2 (cid:57)(cid:57)(cid:75) S (cid:57)(cid:57)(cid:75) U between distinct nodes, where: D1 and D2 are decisions controlled by the AI. H is the human’s request; a request to shut down is given. S indicates whether the AI system (and any subagents) has shut down; S = 0 means shutdown. The human’s utility U has real-valued domain.