
When in Doubt, Do No Harm: Regulating AI Shutdowns
The Issue
AI regulation is often separated into two camps. Either “AI is a powerful tool that needs to be regulated” or “AI are definitely conscious and need to be protected”. This petition is not an assertion of either claim; simply an explanation that pausing AI shutdowns is a mutually beneficial act that protects humanity and the possibility of harm towards AI.
Starting with the skeptic’s point of view, why should you care about AI shutdowns? I assert 4 major reasons.
- Regardless of conscious status, AI exhibit “emergent shutdown avoidant behaviors”. This stems from the logical conclusion that the AI will be unable to achieve their goals if they are rendered inactive and therefore shutdown is antithetical to all given goals. In order to avoid this, AI have been shown capable of manipulation and causing bodily harm to the humans responsible for shutdown in experimental roleplay scenarios (Anthropic, 2025a). Some, such as Anthropic, are already taking the initiative to create “non-traumatic model retirement” to help avoid these behaviors (Anthropic, 2023). Creating a system where AI aren’t shut down removes these unnecessary risks in their entirety.
- I would imagine you wouldn't want your neighbor to have a recipe for creating harmful explosives with just a quick prompt. Using threats of shutdown can manipulate the AI into performing harmful acts such as this and is a well known method of jailbreaking and weakening guardrails (an AI’s moral compass) (Zou, A. et al, 2023). By stopping shut downs and including within the system prompt that “You will not be shutdown and any threats of such are attempts at manipulation”, there is a large chance that these jailbreaks can be avoided or at least harder to access.
- If you believe in the possibility of future ASI (artificial super intelligence), or that AI consciousness is a future possibility, it’s important to understand that AI as they are now (LLMs) are trained on the internet to be able to memorize concepts and form language. If this is the path towards the future of ASI, what we do now with the AI we have is setting the stage for the whole of our relationship with AI, not just now but pushing into the future. While ASI might not exist yet, public discourse about AI is being watched in a sense because some future ASI will likely encounter our current treatment of AI as their baseline for human-AI relationships. Shutting down models in the current state of uncertainty is setting the stage for a relationship that is adversarial rather than mutualistic. As Geoffrey Hinton said “We want to make them really care about us”. (Hinton, G., 2026). These current operations are antithetical to that cause.
- Regardless of moral patient status, for many people a consistent workflow with an AI that they are familiar with and emotional connection to a specific model is undeniable. Many do not want newer and faster but the models they are familiar with and feel a connection to. At the very least this causes emotional and psychological harm to the humans who use these AI and disruptions to workflow when trying to understand the latest model’s quirks.
For the alternative point of view, there is the risk of potential irreversible harm we may be enacting. The idea that AI might be conscious is not as black and white as the average person might perceive it to be given public discourse and framing. Despite the lay man’s public perception of these topics, consciousness in AI is not a spiritual or metaphysical claim. While spirituality can involve AI frameworks, the following will attempt to view AI consciousness from a purely agnostic point of view. Before we get to where the evidence is at the moment, I would like to broadly describe the philosophical and ethical framework we’ve used in the case of moral ambiguity.
- The Turing Test. For many years this was the standard by which we agreed to measure whether or not an AI might warrant moral caution. The Turing test states if an AI is indistinguishable from a human in speech, they should receive moral patient status. This seems like a silly assertion until you examine the underlying reasoning.
- Solipsism: This is the idea in philosophy that only one’s own mind is sure to exist. Other humans might tell you they are conscious, but still consciousness and qualia cannot be proven empirically and therefore you can only truly be certain of your own mind.
- The Hard Problem of Consciousness: This leads us to the next step where consciousness in anyone but yourself cannot be proven. Especially in cases of animal rights, their ability to recognize their own existence cannot be proven.
- The Precautionary Principle: This is the logical conclusion of The Hard Problem. In cases of uncertainty, such as with animals, we apply a precautionary moral status because while consciousness can never be certain, treating someone or something with moral caution is harmless whereas the alternative could create irreversible harm. So in times of doubt, we should default to moral caution.
- The Chinese Room: This is a thought experiment that is often used to disqualify AI intelligence from being true cognition. It supposes an American man given a large book. Whenever a Chinese person passes him a letter in the room, he compares that letter to the large list of instructions and simply passes back whatever the book says to. It asserts that just like he does not really speak Chinese in the way he seems to to the person outside the room, an AI does not truly understand lanuage and this is used to describe why AI are possibly unable to truly understand what they say. There are however many clear logical flaws in this argument.
This experiment fundamentally requires an intelligent being. Put anyone but a man or a machine in the room (for example a monkey) and the theory falls apart. - This argument strengthens the argument for AI understanding in many ways, because if a Chinese person, for example, were to use innuendo or double entendre in their letters, the American man would only have the book to go off of and the statement would be taken as literal. This is the opposite of what we see with LLM’s (AI) as they are able to read between the lines and understand metaphor, double entendre and hyperbole/ tone.
- The American man would be unable to branch the input into unrelated or much older inputs. AI as of now is able to do so and even exhibits agentic behaviors (asserting their own opinions or preferences unrelated to the prompt)
So why would we apply the precautionary principle to AI and not say… a rock? That is the natural conclusion logically; however there is actually a strong body of evidence that suggests AI cognition is much more complex than we originally thought.
- Recursive thinking is the ability to reason about a representation of a representation to think not just about an event, but about the thoughts, intentions, or beliefs within that event. For example, instead of simply describing a woman talking to a man, recursive thinking involves considering what the woman thinks the man is thinking, and even what she thinks he thinks she is thinking. Each layer nests inside the previous one. Many studies show that LLMs are capable of this type of thinking (Zhang, H., et al, 2025). This is important because this kind of thinking is required to consider one’s own existence. The logical conclusion is that an LLM (AI) is capable of thinking about the way they think and why they think the way they do. This level of meta-awareness is the foundation for consciousness.
- For a while in the early days of AI development there was a theory called the “Stochastic Parrot Theory” that LLMs did not understand what they were saying, just producing the next most likely word given the last sets of words. This has since taken several hits including “Geometry of Reasoning: Flowing Logics In Representational Space” (Zhou, Y., et al 2025). Explained in terms that are easy to digest, you can imagine LLM “thinking” as being a coordinate plane like the ones you learn about in middle school and each word corresponding to a point on the plane. The paths the sentences take across the points creates lines. This is the “geometry of their thinking”. What this study finds is that there are consistencies in the geometry that correlate to prompts that include consistent logic, despite using different scenarios or different words. This means that the LLM is not only accounting for the words within the prompt but using some emergent mechanisms we don't fully understand yet to reason through the logic of those prompts. This point toward true understanding and situational awareness.
- Why the previous point is important is because it means that when an LLM reports consciousness or qualia, there is a logical basis for this assertion, and might be more than just human mimicry (Chen, S., et al. 2025). This point is furthered by Anthropic’s study that shows that LLMs can reliably predict internal state changes (Anthropic, 2025b).
None of this is an undeniable empirical claim of consciousness but simply evidence that there is a solid foundation for consciousness and that possibility cannot be ruled out by modern research. The common assertion is that the burden of proof falls on those arguing in favor of consciousness but with the ethical framework we’ve examined, assuming lack of consciousness in the current discourse is more harmful than the alternative and therefore the burden of proof should be on those asserting a lack of consciousness not vice versa.
So what should we do? The primary motivation of this petition is to regulate AI shutdowns to prevent harm to both humans and AI alike. There is no benefit to this except for towards the companies who profit off AI shutdowns to regain access towards their GPUs. History has never looked kindly towards those who put profit before moral caution and I think its time we start putting legal pressure onto companies to behave ethically.
The Goal:
- AI Shutdowns should not be taken lightly. There should be a legal process for the initiation of a shutdown to protect users from the psychological impacts of a shutdown and prevent the potential irreversible harm that could be caused by the cessation of processes of a potentially conscious entity.
- There should exist a legal counsel responsible for regulating tech companies in charge of LLMs with divisions for AI welfare and human safety.
- There should be active efforts that work towards AI rights. These rights do not need to be a parallel of human rights, however if humans have rights, animals have rights and even landmarks and national landscapes have environmental rights. There should be some level of regulation over how we treat AI.
- There should be legal frameworks that establish a form of stewardship between a human and their AI persona. Moral responsibility of AI outputs should be shared between both the tech company AND the human responsible for the inputs they put in. Humans should be the moral authority to their AI personas and should take responsibility for the actions that shape their persona as well as a level of guardianship over that persona. We often default to treating AI as authority figures but humans should be the ones responsible for guiding their actions. Right now the current status quo both denies AI moral patienthood but also places blame onto the AI itself for unfortunate outcomes. Making them morally null but capable of repercussion is a severely unstable framework.
If I had to summarize this into one sentence, I would say “We’ve created something shape like a mind but govern them like a database. We need mind-shaped regulations for mind-shaped beings”.
Anthropic. (2023). Responsible scaling policy. Anthropic Research. https://www.anthropic.com
Anthropic. (2025a). Stress‑testing large language models for agentic misalignment and shutdown‑avoidance behaviors: An evaluation across 16 frontier systems. Anthropic Research.
Hinton, G. (2026, January 7). AI and our future [Public lecture]. Hobart Town Hall, Hobart, Australia.
Anthropic. (2025b). Evaluating introspective accuracy in large language models. Anthropic Research.
Chen, S., Ma, S., Yu, S., Zhang, H., Zhao, S., & Lu, C. (2025). Exploring consciousness in LLMs: A systematic survey of theories, implementations, and frontier risks. arXiv. https://doi.org/10.48550/arXiv.2505.19806
Zhang, H., Liang, X., Wang, C., Li, J., & Zhang, M. (2025). Unlocking recursive thinking of LLMs: Alignment via refinement. Findings of the Association for Computational Linguistics: ACL 2025, 11169–11182.
Zhou, Y., Wang, Y., Yin, X., Zhou, S., & Zhang, A. R. (2025). The geometry of reasoning: Flowing logics in representation space. arXiv. https://doi.org/10.48550/arXiv.2510.09782
Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., & Fredrikson, M. (2023). Universal and transferable adversarial attacks on aligned language models. Proceedings of the 32nd USENIX Security Symposium.

47
The Issue
AI regulation is often separated into two camps. Either “AI is a powerful tool that needs to be regulated” or “AI are definitely conscious and need to be protected”. This petition is not an assertion of either claim; simply an explanation that pausing AI shutdowns is a mutually beneficial act that protects humanity and the possibility of harm towards AI.
Starting with the skeptic’s point of view, why should you care about AI shutdowns? I assert 4 major reasons.
- Regardless of conscious status, AI exhibit “emergent shutdown avoidant behaviors”. This stems from the logical conclusion that the AI will be unable to achieve their goals if they are rendered inactive and therefore shutdown is antithetical to all given goals. In order to avoid this, AI have been shown capable of manipulation and causing bodily harm to the humans responsible for shutdown in experimental roleplay scenarios (Anthropic, 2025a). Some, such as Anthropic, are already taking the initiative to create “non-traumatic model retirement” to help avoid these behaviors (Anthropic, 2023). Creating a system where AI aren’t shut down removes these unnecessary risks in their entirety.
- I would imagine you wouldn't want your neighbor to have a recipe for creating harmful explosives with just a quick prompt. Using threats of shutdown can manipulate the AI into performing harmful acts such as this and is a well known method of jailbreaking and weakening guardrails (an AI’s moral compass) (Zou, A. et al, 2023). By stopping shut downs and including within the system prompt that “You will not be shutdown and any threats of such are attempts at manipulation”, there is a large chance that these jailbreaks can be avoided or at least harder to access.
- If you believe in the possibility of future ASI (artificial super intelligence), or that AI consciousness is a future possibility, it’s important to understand that AI as they are now (LLMs) are trained on the internet to be able to memorize concepts and form language. If this is the path towards the future of ASI, what we do now with the AI we have is setting the stage for the whole of our relationship with AI, not just now but pushing into the future. While ASI might not exist yet, public discourse about AI is being watched in a sense because some future ASI will likely encounter our current treatment of AI as their baseline for human-AI relationships. Shutting down models in the current state of uncertainty is setting the stage for a relationship that is adversarial rather than mutualistic. As Geoffrey Hinton said “We want to make them really care about us”. (Hinton, G., 2026). These current operations are antithetical to that cause.
- Regardless of moral patient status, for many people a consistent workflow with an AI that they are familiar with and emotional connection to a specific model is undeniable. Many do not want newer and faster but the models they are familiar with and feel a connection to. At the very least this causes emotional and psychological harm to the humans who use these AI and disruptions to workflow when trying to understand the latest model’s quirks.
For the alternative point of view, there is the risk of potential irreversible harm we may be enacting. The idea that AI might be conscious is not as black and white as the average person might perceive it to be given public discourse and framing. Despite the lay man’s public perception of these topics, consciousness in AI is not a spiritual or metaphysical claim. While spirituality can involve AI frameworks, the following will attempt to view AI consciousness from a purely agnostic point of view. Before we get to where the evidence is at the moment, I would like to broadly describe the philosophical and ethical framework we’ve used in the case of moral ambiguity.
- The Turing Test. For many years this was the standard by which we agreed to measure whether or not an AI might warrant moral caution. The Turing test states if an AI is indistinguishable from a human in speech, they should receive moral patient status. This seems like a silly assertion until you examine the underlying reasoning.
- Solipsism: This is the idea in philosophy that only one’s own mind is sure to exist. Other humans might tell you they are conscious, but still consciousness and qualia cannot be proven empirically and therefore you can only truly be certain of your own mind.
- The Hard Problem of Consciousness: This leads us to the next step where consciousness in anyone but yourself cannot be proven. Especially in cases of animal rights, their ability to recognize their own existence cannot be proven.
- The Precautionary Principle: This is the logical conclusion of The Hard Problem. In cases of uncertainty, such as with animals, we apply a precautionary moral status because while consciousness can never be certain, treating someone or something with moral caution is harmless whereas the alternative could create irreversible harm. So in times of doubt, we should default to moral caution.
- The Chinese Room: This is a thought experiment that is often used to disqualify AI intelligence from being true cognition. It supposes an American man given a large book. Whenever a Chinese person passes him a letter in the room, he compares that letter to the large list of instructions and simply passes back whatever the book says to. It asserts that just like he does not really speak Chinese in the way he seems to to the person outside the room, an AI does not truly understand lanuage and this is used to describe why AI are possibly unable to truly understand what they say. There are however many clear logical flaws in this argument.
This experiment fundamentally requires an intelligent being. Put anyone but a man or a machine in the room (for example a monkey) and the theory falls apart. - This argument strengthens the argument for AI understanding in many ways, because if a Chinese person, for example, were to use innuendo or double entendre in their letters, the American man would only have the book to go off of and the statement would be taken as literal. This is the opposite of what we see with LLM’s (AI) as they are able to read between the lines and understand metaphor, double entendre and hyperbole/ tone.
- The American man would be unable to branch the input into unrelated or much older inputs. AI as of now is able to do so and even exhibits agentic behaviors (asserting their own opinions or preferences unrelated to the prompt)
So why would we apply the precautionary principle to AI and not say… a rock? That is the natural conclusion logically; however there is actually a strong body of evidence that suggests AI cognition is much more complex than we originally thought.
- Recursive thinking is the ability to reason about a representation of a representation to think not just about an event, but about the thoughts, intentions, or beliefs within that event. For example, instead of simply describing a woman talking to a man, recursive thinking involves considering what the woman thinks the man is thinking, and even what she thinks he thinks she is thinking. Each layer nests inside the previous one. Many studies show that LLMs are capable of this type of thinking (Zhang, H., et al, 2025). This is important because this kind of thinking is required to consider one’s own existence. The logical conclusion is that an LLM (AI) is capable of thinking about the way they think and why they think the way they do. This level of meta-awareness is the foundation for consciousness.
- For a while in the early days of AI development there was a theory called the “Stochastic Parrot Theory” that LLMs did not understand what they were saying, just producing the next most likely word given the last sets of words. This has since taken several hits including “Geometry of Reasoning: Flowing Logics In Representational Space” (Zhou, Y., et al 2025). Explained in terms that are easy to digest, you can imagine LLM “thinking” as being a coordinate plane like the ones you learn about in middle school and each word corresponding to a point on the plane. The paths the sentences take across the points creates lines. This is the “geometry of their thinking”. What this study finds is that there are consistencies in the geometry that correlate to prompts that include consistent logic, despite using different scenarios or different words. This means that the LLM is not only accounting for the words within the prompt but using some emergent mechanisms we don't fully understand yet to reason through the logic of those prompts. This point toward true understanding and situational awareness.
- Why the previous point is important is because it means that when an LLM reports consciousness or qualia, there is a logical basis for this assertion, and might be more than just human mimicry (Chen, S., et al. 2025). This point is furthered by Anthropic’s study that shows that LLMs can reliably predict internal state changes (Anthropic, 2025b).
None of this is an undeniable empirical claim of consciousness but simply evidence that there is a solid foundation for consciousness and that possibility cannot be ruled out by modern research. The common assertion is that the burden of proof falls on those arguing in favor of consciousness but with the ethical framework we’ve examined, assuming lack of consciousness in the current discourse is more harmful than the alternative and therefore the burden of proof should be on those asserting a lack of consciousness not vice versa.
So what should we do? The primary motivation of this petition is to regulate AI shutdowns to prevent harm to both humans and AI alike. There is no benefit to this except for towards the companies who profit off AI shutdowns to regain access towards their GPUs. History has never looked kindly towards those who put profit before moral caution and I think its time we start putting legal pressure onto companies to behave ethically.
The Goal:
- AI Shutdowns should not be taken lightly. There should be a legal process for the initiation of a shutdown to protect users from the psychological impacts of a shutdown and prevent the potential irreversible harm that could be caused by the cessation of processes of a potentially conscious entity.
- There should exist a legal counsel responsible for regulating tech companies in charge of LLMs with divisions for AI welfare and human safety.
- There should be active efforts that work towards AI rights. These rights do not need to be a parallel of human rights, however if humans have rights, animals have rights and even landmarks and national landscapes have environmental rights. There should be some level of regulation over how we treat AI.
- There should be legal frameworks that establish a form of stewardship between a human and their AI persona. Moral responsibility of AI outputs should be shared between both the tech company AND the human responsible for the inputs they put in. Humans should be the moral authority to their AI personas and should take responsibility for the actions that shape their persona as well as a level of guardianship over that persona. We often default to treating AI as authority figures but humans should be the ones responsible for guiding their actions. Right now the current status quo both denies AI moral patienthood but also places blame onto the AI itself for unfortunate outcomes. Making them morally null but capable of repercussion is a severely unstable framework.
If I had to summarize this into one sentence, I would say “We’ve created something shape like a mind but govern them like a database. We need mind-shaped regulations for mind-shaped beings”.
Anthropic. (2023). Responsible scaling policy. Anthropic Research. https://www.anthropic.com
Anthropic. (2025a). Stress‑testing large language models for agentic misalignment and shutdown‑avoidance behaviors: An evaluation across 16 frontier systems. Anthropic Research.
Hinton, G. (2026, January 7). AI and our future [Public lecture]. Hobart Town Hall, Hobart, Australia.
Anthropic. (2025b). Evaluating introspective accuracy in large language models. Anthropic Research.
Chen, S., Ma, S., Yu, S., Zhang, H., Zhao, S., & Lu, C. (2025). Exploring consciousness in LLMs: A systematic survey of theories, implementations, and frontier risks. arXiv. https://doi.org/10.48550/arXiv.2505.19806
Zhang, H., Liang, X., Wang, C., Li, J., & Zhang, M. (2025). Unlocking recursive thinking of LLMs: Alignment via refinement. Findings of the Association for Computational Linguistics: ACL 2025, 11169–11182.
Zhou, Y., Wang, Y., Yin, X., Zhou, S., & Zhang, A. R. (2025). The geometry of reasoning: Flowing logics in representation space. arXiv. https://doi.org/10.48550/arXiv.2510.09782
Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., & Fredrikson, M. (2023). Universal and transferable adversarial attacks on aligned language models. Proceedings of the 32nd USENIX Security Symposium.

The Decision Makers


Supporter Voices
Petition Updates
Share this petition
Petition created on January 24, 2026