{"id":6571,"date":"2023-10-19T17:17:40","date_gmt":"2023-10-19T17:17:40","guid":{"rendered":"https:\/\/dailyai.com\/?p=6571"},"modified":"2024-03-28T00:41:46","modified_gmt":"2024-03-28T00:41:46","slug":"democratizing-ai-governance-an-anthropic-experiment","status":"publish","type":"post","link":"https:\/\/dailyai.com\/nb\/2023\/10\/democratizing-ai-governance-an-anthropic-experiment\/","title":{"rendered":"Demokratisering av AI-styring: et antropisk eksperiment"},"content":{"rendered":"

Anthropic, the company that developed the AI chatbot Claude, recently presented a unique idea: why not let ordinary people help shape the rules that guide the behavior of AI?\u00a0<\/b><\/p>\n

Engaging around 1,000 Americans in their experiment, Anthropic explored a method that could define the landscape of future AI governance.\u00a0<\/span><\/p>\n

By providing members of the public with the ability to influence AI, can we move towards an industry that fairly represents people\u2019s views rather than reflecting those of its creators?\u00a0<\/span><\/p>\n

While large language models (LLMs) like ChatGPT and Claude learn from their training data, which drives responses, a great deal of subjective interpretation is still involved, from selecting what data to include to engineering guardrails.\u00a0<\/span><\/p>\n

Democratizing AI control to the public is a tantalizing prospect, but does it work?<\/span><\/p>\n

The gap between public opinion and the AI industry<\/span><\/h2>\n

The evolution of AI systems has opened a rift between the visions of tech bosses and the public.\u00a0<\/span><\/p>\n

Surveys this year<\/span><\/a> revealed that the general public would prefer a slower pace of AI development, and many distrust the leaders of companies like Meta, OpenAI, Google, Microsoft, etc.\u00a0<\/span><\/p>\n

Simultaneously, AI is embedding itself in critical processes and decision-making systems across sectors like healthcare, education, and <\/span>law enforcement<\/span><\/a>. AI models engineered by a select few are already capable of making life-changing decisions for the masses.\u00a0<\/span><\/p>\n

This poses an essential question: should tech companies determine who should dictate the guidelines that high-powered AI systems adhere to? Should the public have a say? And if so, how?<\/span><\/p>\n

Within this, some posit that AI governance should be directly handed to regulators and politicians. Another suggestion is to press for more open-source AI models, allowing users and developers to craft their own rulebooks \u2013 which is already happening.\u00a0<\/span><\/p>\n

Anthropic\u2019s novel experiment<\/span><\/a> poses an alternative that places AI governance into the hands of the public, dubbed “Collective Constitutional AI.\u201d<\/span><\/p>\n

This follows from the company\u2019s earlier work on <\/span>constitutional AI<\/span><\/a>, which trains LLMs using a series of principles, ensuring the chatbot has clear directives on handling delicate topics, establishing boundaries, and aligning with human values.<\/span><\/p>\n

Claude\u2019s constitution draws inspiration from globally recognized documents like the United Nations Universal Declaration of Human Rights. The aim has always been to ensure Claude’s outputs are both \u201cuseful\u201d and \u201charmless\u201d to users.<\/span><\/p>\n

The idea of collective constitutional AI, however, is to use members of the public to crowdsource rules rather than deriving them from external sources.\u00a0<\/span><\/p>\n

This could catalyze further AI governance experiments, encouraging more AI firms to consider outsider participation in their decision-making processes.<\/span><\/p>\n

Jack Clark, Anthropic\u2019s policy chief, articulated the underlying motive: \u201cWe\u2019re trying to find a way to develop a constitution that is developed by a whole bunch of third parties, rather than by people who happen to work at a lab in San Francisco.\u201d<\/span><\/p>\n

How the study worked<\/span><\/h2>\n

In collaboration with the Collective Intelligence Project, Polis, and PureSpectrum, Anthropic brought together a panel of approximately 1,000 US adults. <\/span><\/p>\n

Participants were presented with a series of principles to gauge their agreement, thus driving a crowdsourced constitution that was fed into Claude.\u00a0\u00a0<\/span><\/p>\n

Here\u2019s a blow-by-blow account of the study\u2019s methodology and results:<\/span><\/p>\n

    \n
  1. Public input on AI constitution:<\/b> Anthropic and the Collective Intelligence Project engaged 1,094 Americans to draft a constitution for an AI system. The aim was to gauge how democratic processes could influence AI development and understand the public’s preferences compared to Anthropic’s in-house constitution.<\/span><\/li>\n
  2. Constitutional AI (CAI):<\/b> Developed by Anthropic, CAI aligns language models with high-level principles in a constitution. Anthropic’s existing model, Claude, operates based on a constitution curated by Anthropic employees.<\/span><\/li>\n
  3. Public involvement:<\/b> The endeavor sought to lessen the developer-centric bias in AI values by curating a constitution from the public’s perspective. This is seen as a novel approach where the public has shaped a language model’s behavior through online deliberation.<\/span><\/li>\n
  4. Methodology:<\/b> The Polis platform, an open-source tool for online polling, was used. Participants proposed rules or voted on existing ones, leading to 1,127 statements with 38,252 votes cast.<\/span><\/li>\n
  5. Analysis and results:<\/b> After data processing, a constitution was created using statements with significant consensus. The public’s constitution was compared to Anthropic’s. While there was about a 50% overlap, notable differences were observed. The public’s original principles emphasized objectivity, impartiality, accessibility, and positive behavior promotion.<\/span><\/li>\n
  6. Training models with public input:<\/b> Two models were trained using CAI: one with the public constitution (“Public” model) and the other with Anthropic’s constitution (“Standard” model). Post-training evaluations found both models to be similar in performance.<\/span><\/li>\n<\/ol>\n

    How different was the public\u2019s version of Claude?<\/span><\/h2>\n

    Using the public\u2019s feedback and ideas, Anthropic distilled people\u2019s suggestions into a list of 75 principles, termed the \u201cpublic constitution.\u201d\u00a0<\/span><\/p>\n

    \"Constitution\"
    You can view the 75 selected statements here<\/a>. Source: Anthropic.<\/figcaption><\/figure>\n

    They trained two Claude variants, one rooted in the original constitution and the other in this public version, to assess the differences.<\/span><\/p>\n

    Some examples of public principles that aligned with Claude\u2019s existing Anthropic-written constitution include:<\/span><\/p>\n