OpenAI tightened limits on weapons-related information ChatGPT provides to teens on Friday, as part of new safeguards aimed at preventing real-world violence amid growing scrutiny of AI chatbots.
New Limits Target Weapons and Security Information
The company will restrict practical details on weapons and security systems that could be misused. The limits apply to teens even when there is no sign of violent intent.
OpenAI said that, for all users, ChatGPT will limit detail on past mass attacks, including the weapons used and how they were obtained.
Automatic Chat Endings and Account Pauses
ChatGPT will be able to automatically end chats in which a user appears fixated on violence, according to the company. OpenAI may temporarily pause accounts after multiple such chats.
OpenAI said it strengthened its age prediction system to move users it estimates are under 18 into the teen experience. The company is planning to introduce notifications for parents if a teen’s account is paused for repeatedly discussing violence.
This is not the first time OpenAI has introduced additional safeguards for younger users. In August, the company launched ChatGPT for Teens, a version designed for users ages 13 to 17, featuring additional safety measures and parental controls.
The experience limits conversations involving self-harm, violence and eating disorders, while allowing linked parents to receive safety alerts in certain high-risk situations.
Regulators and Watchdogs Press AI Firms on Youth Safety
The fresh measures follow growing scrutiny of AI companies over the risks chatbots pose to young users and the firms’ role in detecting threats of harm.
In September, the European Union was preparing rules that would restrict children under 15 from independently accessing social media and AI chatbots, including ChatGPT.
Earlier this month, nonprofit Common Sense Media reported that ChatGPT’s teen version often alerted parents late, or not at all, when accounts discussed suicide, self-harm or eating disorders.
OpenAI previously said that teens spend under 15 minutes a day on ChatGPT on average.
The safeguards will begin rolling out in the next few weeks, and OpenAI said it will keep refining them.
Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors.
Image via Shutterstock
Recent Comments