French Experts Want Consumer Chatbots Tested Before Launch

A panel of nine experts set up by the French presidency wants consumer chatbots to pass a safety test before they can be sold in Europe. Its report was handed to Emmanuel Macron on 10 November and makes 18 recommendations. None of them is law yet.

The presidency has published a summary of the report on its website. The group was led by the AI researcher Serena Villata and two psychiatrists, Raphaël Gaillard and Amine Benyamina. It held 55 hearings with doctors, researchers, companies and child protection groups.

What the experts want

The main idea is a check before launch. Chatbots for the general public, including those built into social networks, would have to show that they do not exploit or worsen the weaknesses of the people who use them. A second check would follow every major change to the model, or a warning from users.

The panel also wants strict limits on design tricks that are meant to keep people using a chatbot for longer. It names two of them. One is a chatbot that acts too much like a human. The other is sycophancy, which means a chatbot that flatters users and agrees with whatever they say.

Toys with a chatbot inside would need approval before they could be sold, as well as age limits. Hidden advertising in chatbot answers would be banned, and companies would have to explain how they make money from their chatbots.

The numbers behind the report

The summary says 33 million people in France used generative AI at the start of this year. A quarter of French people say they worry about its risks.

Among French teenagers who use these tools, 92% say they spend more time than they would like on at least one AI app. The figure comes from a survey by the think tank Interface with YouGov. For AI companion apps, which are designed to act like a friend, the share is 59%. These figures are set out in the full report. The panel adds that heavy use of an app is not, on its own, a medical sign of addiction.

Next steps for the report

France cannot bring in these checks on its own. The report says several of its ideas could shape the French position in talks on EU rules, including an update to the EU AI Act and a proposed EU law to protect children online.

On Thursday, four AI companies agreed to independent child safety tests under a charter presented in Paris. We covered that agreement in a separate article. At the time, Macron said no AI should reach children before it has been tested. The new report explains how a rule like that could work. Australia is also taking steps against chatbots that act as children’s best friends.

Why it matters: Most rules for AI apply after harm has been done. A test before launch would make companies prove that a chatbot is safe for the public before anyone uses it.