LMSYS-Chat-1M
LMSYS (UC Berkeley and collaborators)
One million real conversations with 25 LLMs, collected from Vicuna demo and Chatbot Arena visitors.
Real user prompts and replies gathered April to August 2023 from about 210,000 unique IP addresses. One row is one conversation with its model name, text in OpenAI JSON format, language tag, and moderation tags. Access is gated and requires agreeing to terms.
Access
- Cost
- FreeFree but gated; accept the terms on the Hub.
- License
- License not statedNot in the record; the Hub card points to dataset-specific terms
- Sign-up
- Free account
- How you get it
- API and Bulk downloadParquet
What's in it
- One row is
- One row per conversation
- Covers
- Global users
- Years
- April to August 2023
- Updated
- One-time release
- Size
- 1 million conversations
Quality
Rare view of what people actually ask LLMs. Users self-selected and the models are from 2023.
- Anonymized but may still contain personal information
- Includes toxic and unsafe prompts
- Users of these demo sites skew technical
Connects to
- WildChat-1M: Comparable corpus of real ChatGPT conversations with demographic fields
- Chatbot Arena Conversations: Smaller sibling with pairwise human votes
Questions it could answer
- What do people ask LLMs, by language and model?
- How often do conversations trip a moderation flag?