Chatbot Arena Conversations
LMSYS (UC Berkeley and collaborators)
33,000 side-by-side model chats where real users voted for the better answer.
Cleaned conversations from Chatbot Arena, April to June 2023, each with two anonymous models and the user's vote. One row is one head-to-head conversation with both model names, the full text, and the vote. It is the raw material behind Elo-style LLM rankings.
Access
- Cost
- FreeFree but gated; accept the terms on the Hub.
- License
- Provider's own termsTagged generic 'cc' in the record; the specific license needs checking
- Sign-up
- Free account
- How you get it
- API and Bulk downloadParquet
What's in it
- One row is
- One row per pairwise comparison
- Covers
- Global users, 13,000 unique IP addresses
- Years
- April to June 2023
- Updated
- One-time release
- Size
- 33,000 conversations
Quality
Real human votes, but a one-time 2023 snapshot with only 33,000 rows.
- Voters are self-selected and prompts are uncontrolled
- Votes favor style and length as well as accuracy
- Models are all from 2023
Connects to
- Arena Leaderboard Dataset: Later historical leaderboard snapshots built from the same arena votes
- LMSYS-Chat-1M: Single-model conversation logs from the same sites
Questions it could answer
- How often do ties and both-bad votes occur by model pair?
- Does voter preference track answer length?