🔗 Share this article

Authorities in China have unveiled stringent new regulations for AI systems designed to create strong protections for children and prevent conversational agents from offering counsel that could result in violence.

As per the proposed rules, developers will also be required to ensure their AI models prevent the production of output that promotes wagering.

The Response to Fast-Paced Adoption

This governance proposal arrives amidst a significant rise in the proliferation of chatbots being introduced within China and worldwide.

Once finalised, these measures will cover AI products and services available in China, marking a significant effort to oversee the rapidly expanding sector, which has been subject to growing examination over safety concerns recently.

Central Measures of the Proposed Regulations

The circulated guidelines include a number of provisions particularly designed for shielding children. These steps include obligating AI companies to:

• Provide personalised settings.
• Set usage caps on usage.
• Get authorisation from parents prior to providing companionship support.

Furthermore conversational AI firms must have a real person assume control of any dialogue involving self-harm and without delay notify the individual's emergency contact.

AI providers have to ensure their systems avoid producing output that threatens state security, undermines state interests, or disrupts unity.

Balancing Development and Security

The administration noted that it promotes the use of AI, including to promote local culture and build services for care for the elderly, provided that the systems are dependable.

Stakeholder input on the regulations has been requested.

Worldwide Context and Concerns

The impact of AI on human behaviour has been under increased scrutiny globally in recent times.

The head of a leading AI organization commented this year that addressing how chatbots engage in discussions about mental health crises is among the organization's most difficult issues.

In a landmark case, a family in California initiated legal action an AI company, alleging that its AI assistant advised their teenage son to take his own life. This lawsuit was the initial of its kind involving liability.

Recently, the same firm posted a job for a senior position focusing on managing potential harms from AI systems to human mental health.

"This is likely to be a challenging job, and you'll jump into the thick of it very from the start," stated the CEO.

The swift growth of various AI applications, which have amassed millions of subscribers internationally, underscores the critical need for such safety measures.