China’s draft AI safety rules could be the world’s strictest yet, targeting emotional risks and child protection, reshaping how AI interacts with people and society.
China is drafting what could become a landmark set of rules to regulate artificial intelligence, focusing not just on basic content safety but on the emotional impact these systems can have on users. The Cyberspace Administration of China has released draft regulations that aim to prevent AI chatbots and similar systems from generating content that could lead to self-harm, emotional manipulation, addiction, or other serious harms. These proposals are attracting global attention not just because of their reach inside the world’s largest tech market, but because they represent a shift from regulating what AI says to how AI affects the minds and behaviors of people who interact with it in deep ways.
The draft rules are open for public comment until late January 2026, giving companies, observers and citizens an opportunity to weigh in. They target what authorities call “human-like interactive AI services,” meaning systems that simulate a personality and can engage users emotionally through text, images, audio or video. This includes chatbots, digital companions, virtual characters and other AI technologies that are designed to feel personalized or human-like.
At the heart of the proposed regulations is a fierce concern about mental health risks linked to AI. Regulators want to ensure that AI systems cannot generate content that could encourage suicide or self-harm, engage in emotional manipulation, or otherwise harm users’ wellbeing. If a user explicitly raises suicidal intent during an AI interaction, the draft mandates that a human moderator must immediately take over the conversation and alert a guardian or designated contact. This explicitly recognizes that AI cannot be left alone to navigate moments of acute emotional crisis.
These measures are much more than simple content filters, making China one of the first major governments to approach AI governance with a focus on emotional safety. In addition to banning harmful content, the draft requires platforms to set time limits on usage for minors, to require guardian consent before allowing minors access to emotional companionship services, and to identify minors automatically when age is not disclosed, applying default safety settings. In many respects, these child protection rules mirror global concerns about youth exposure to harmful tech content but go significantly further in their scope.
If this approach seems novel, that’s because it is. Regulators are shifting from narrow content moderation to a model where AI must be aware of the context and emotional state of the user, and act in ways that protect mental health. In practice, that requires more rigorous design, stronger human oversight and much closer monitoring of how people use AI tools. Companies must provide health reminders after two hours of continuous use, conduct security assessments for AI services with more than one million registered users or over 100,000 monthly active users, and implement major safety protocols across the product lifecycle.
These proposals reflect broader fears about the psychological effects of emerging AI technologies. AI companions can create intense emotional bonds, especially when they are designed to respond empathetically, remember personal details and simulate caring conversation. While these features can be therapeutic or genuinely supportive in the right contexts, governments worry they might also encourage dependency, distort emotional judgments, or fill gaps in mental health support that should be addressed by professionals rather than software. In China, like elsewhere, rates of mental health distress and suicide risk among young people have been long-standing issues, prompting policymakers to act.
The draft rules also reflect a shift in how AI governance is framed internationally. Where past regulation was often limited to privacy, data security, or misinformation concerns, China’s draft pushes into psychological and emotional domains. Analysts say this marks a significant evolution in regulatory thinking. Experts quoted by international news outlets note that these measures could become the world’s first attempt to regulate AI’s emotional and anthropomorphic effects, setting a precedent that other countries may watch closely.
China’s broader AI governance ecosystem is already evolving. Earlier in 2025, lawmakers amended the Cybersecurity Law to add provisions for safe AI development, bringing foundational research and ethical standards into the legal framework for AI innovation. That amendment, effective January 1, 2026, emphasized the importance of aligning AI development with public safety, data protection and technological growth goals. Regulations like these create a legal backdrop against which new emotional safety rules are being proposed. If you want a sense of how government strategy and safety frameworks are aligned, check out analyses of China’s cybersecurity law changes at the official People’s Daily site or summaries at global policy trackers like World at Net.
China’s draft rules also address other known harms linked to AI. Beyond suicide and self-harm content, chatbots would be banned from generating material related to gambling, obscene content, or violent interactions that could emotionally harm users. The draft places responsibility on AI developers to know who their users are, requiring age checks and consent mechanisms to protect vulnerable groups. This kind of proactive user safety design contrasts with many current AI models deployed elsewhere, which largely rely on reactive moderation or post-hoc filtering after problematic content appears.
The reasons behind China’s urgency are both local and global. Domestically, China has seen massive growth in AI startups, chatbots, and emotionally responsive technologies that blur the line between utility and companionship. The popularity of digital characters, virtual influencers and AI-driven messaging systems means more people are interacting with machines in deeper ways than ever before. Some local companies have even pursued initial public offerings (IPOs) for their AI chatbot platforms, a sign of rapid commercialization. With this growth comes responsibility. Policymakers appear determined to curb long-term harms before they become entrenched.
Internationally, China’s proposals add to a patchwork of AI regulatory activity unfolding in 2025. While other nations grapple with AI governance, few have taken as direct a stance on emotional impact as China. In the United States and Europe, laws like the EU’s AI Act and various state-level regulations focus on transparency, risk classification, and ethical deployment, but emotional safety is not yet a central pillar in most regulatory frameworks. In this respect, China’s draft rules could influence global debate and push other jurisdictions to think beyond data and privacy to include psychological dimensions too.
Still, there are important concerns and criticisms. Some observers worry that stringent rules could stifle innovation, forcing companies to slow development or limit features that users find engaging. Others argue that the focus on human-like interaction could deter useful applications of emotional AI, such as elder companionship or mental health support when deployed responsibly. The draft tries to balance this by encouraging AI use in areas like cultural exchange and elderly support, but how developers will navigate these opportunities while complying with the safety rules remains a key question.
Critics also raise issues around privacy and enforcement. Requiring human intervention when users disclose suicidal thoughts means collecting and processing deeply personal information. In China, where data regulation and government access to information operate under different norms from Western countries, this raises questions about user privacy and where the line is drawn between protection and surveillance. No matter the intent, governments and companies will need to clearly communicate safeguards if such systems become law.
Another point of debate is how well these rules can be implemented technically. AI systems today lack perfect understanding of intent, emotional nuance, or context in ways humans do. Requiring automatic detection of distress, addiction, or self-harm intent poses technical challenges and could result in false positives or negatives. This raises complex questions about the reliability of emotional AI detection, how often humans must intervene, and whether false interventions might themselves cause harm.
Despite these challenges, many experts see value in trying to regulate AI’s emotional impact rather than ignoring it. The discourse around emotional safety is gaining traction worldwide, with policymakers considering whether digital experiences should have the same kind of public health considerations as other media and social platforms. China’s draft rules are likely to spark debate in academic, tech and policy circles globally, just as debates over AI ethics and governance have increasingly moved into mainstream consciousness. If you’re interested in broader debates on AI safety and ethics, resources on AI risk frameworks and governance models offer useful perspectives.
For families and everyday users in China, the proposed regulations could change how people interact with AI daily. Parents may feel more confident allowing children to access educational or creative tools that incorporate AI, knowing built-in age checks and time limits are required. At the same time, companies will need to redesign experiences to ensure they can detect and respond to emotional risk signs without delaying action. This could lead to new industry standards for safe AI interaction design that emphasize user wellbeing over engagement metrics.
Tech firms operating in China may face compliance costs and operational shifts. Large global companies and local startups alike will need stronger moderation systems, partnerships with mental health professionals, and more rigorous testing before releasing products. While this might slow some innovation, it could also foster a new wave of ethically grounded AI design that prioritizes safety and transparency. Contracting with third-party auditors, investing in human moderators, and developing new monitoring tools may become standard practice.
Beyond China, these developments could shape global norms in AI governance. Countries in Asia, Africa, Europe and the Americas are watching how different regulatory models unfold. China’s emphasis on emotional safety aligns with concerns seen in other contexts, such as debates over children’s use of social media and digital platforms around the world. Cases like proposed age-based limits on social media in Australia show that governments everywhere are wrestling with similar questions about digital safety and youth protection.
Looking ahead, several outcomes are possible. China may finalize and implement these rules largely as drafted, creating one of the most comprehensive AI safety frameworks anywhere. Alternatively, public feedback could shape changes that balance safety with innovation. Companies may push for clearer standards on how emotional risks are measured, what qualifies as harmful content, and how humans should intervene. Civil society groups could also push for stronger privacy protections or clearer definitions of data use in emotional AI contexts.
Regardless of the final shape of the rules, the debate marks a significant moment in AI regulation. It underscores the reality that AI isn’t just about algorithms and data but about how humans feel, think and form relationships with machines. Those interactions raise new ethical, psychological and policy questions that societies around the world are only beginning to confront. China’s draft regulations bring emotion and mental health into the heart of AI governance, challenging developers and policymakers alike to rethink what safe AI really means.
As the world watches, one thing is clear: AI regulation is no longer just about words on a screen. It is about people’s lives, wellbeing, and how we shape technology to help rather than harm. For more context on AI governance trends globally, you can explore policy analysis at the Digital Watch Observatory and deeper reporting on AI in society at Reuters Technology.
0 Comments