The mysterious ‘gpt2-chatbot’: a surprise breakthrough in AI?

April 29, 2024

  • An unidentified chatbot, "gpt2-chatbot," appeared on the LMSys Chatbot Arena
  • Testers say it remarkably rivals or beats GPT-4 and Claude 3 Opus
  • No one knows where it came from, though it seems to be an OpenAI creation

A mysterious chatbot named ‘gpt2-chatbot’ has sparked immense excitement and speculation across the AI community.

This chatbot, which you can find and experiment with yourself on the LMSys Chatbot Arena, is touted by some users to be more advanced than current state-of-the-art models like GPT-4 and Claude 3 Opus.

Emerging seemingly out of thin air, its origins are totally unknown. Could it be an experimental project from a leading AI lab or a strange leak? 

Reddit users mainly think it’s from OpenAI since the model admits that the answer style and tone seem different from GPT-4 and GPT-4 Turbo, but the knowledge cut-off differs from currently available OpenAI models. 

I asked gpt2-chatbot myself, “Who are you,” to which it answered:

“I was created by OpenAI, a research organization that focuses on developing and promoting friendly AI for the benefit of humanity. My development is based on the GPT (Generative Pre-trained Transformer) series of language models, specifically the GPT-4 version, which is an evolution of previous models like GPT-3. These models are trained on a diverse range of internet text, but they are designed to generate human-like text based on the prompts they receive.

OpenAI started with the goal of ensuring that artificial general intelligence (AGI) benefits all of humanity. They conduct research in the field of AI with the aim to develop and direct artificial intelligence technologies in ways that benefit humanity as a whole, unencumbered by financial obligations to investors.

If you have any more questions about AI, OpenAI, or related technologies, feel free to ask!”

When I quizzed it on why it’s named gpt2-chatbot, it replied:

“To summarize, I am not called “GPT-2”; I am a variant of GPT-4, designed specifically for interactive applications, such as this conversational format. My design includes improvements in handling dialogues, safety features, ethical considerations, and overall response quality. If you see references to “GPT-2,” they are likely discussing the model that came before GPT-3 and GPT-4.”

gpt2-chatbot’s performance provokes speculation

Regarding performance, one user on the Y Combinator forum discussed the bot’s domain knowledge in material science, discussing turbine blade manufacturing with depth they’d not witnessed otherwise. 

Another user, the well-known data scientist Simon Wilson, was impressed by its handling of highly personalized queries, suggesting it has a detailed understanding of people’s personal backgrounds.

Others identified standard LLM shortcomings, such as hallucinations and factual inaccuracies.

Having used it myself, I’d agree that it feels like a variant of GPT-4. 

As to whether it’s better than GPT-4, as some speculate, it’s worth highlighting that many felt that GPT-4’s performance dropped over time (possibly coinciding with OpenAI dedicating compute resources to training new models).

So, this potentially unadulterated variant might feel like GPT -4 did when it first came out. Throw in some slight differentiation in behavior and performance on specific tasks, and you can see why speculation is high. 

As discussions of gpt2-chatbot hit X, some have inevitably speculated it’s evidence of GPT-4.5, GPT-5, or even AGI. However, most are unsure whether it represents concrete progress over current models. 

Overall, there’s no way an AI company would train an entirely new chatbot like this and release it in such a way. It’s almost certainly a GPT-4 variant.

Ultimately, it’s hearsay until the creator steps out from behind the curtains and claims ownership.

Join The Future


Clear, concise, comprehensive. Get a grip on AI developments with DailyAI

Sam Jeans

Sam is a science and technology writer who has worked in various AI startups. When he’s not writing, he can be found reading medical journals or digging through boxes of vinyl records.


Stay Ahead with DailyAI

Sign up for our weekly newsletter and receive exclusive access to DailyAI's Latest eBook: 'Mastering AI Tools: Your 2024 Guide to Enhanced Productivity'.

*By subscribing to our newsletter you accept our Privacy Policy and our Terms and Conditions