What’s Out There
← Back to swiping
Language & textFreeAcademic

UltraFeedback

OpenBMB (Tsinghua University)

64,000 prompts, four model answers each, scored by GPT-4 on four aspects.

Prompts were drawn from sources such as UltraChat, ShareGPT, Evol-Instruct, TruthfulQA, FalseQA, and FLAN, then answered by multiple LLMs. One row is a prompt with its four completions and fine-grained AI ratings with critiques. It is widely used to build preference pairs for reward models and DPO.

Access

Cost
FreeNo charge.
License
Open licenseMIT
Sign-up
No sign-up
How you get it
API and Bulk downloadParquet, JSON Lines

What's in it

One row is
One row per prompt with four rated completions
Covers
Not geographic
Years
2023
Updated
One-time release
Size
About 64,000 prompts with 4 responses each
instructioncompletionsmodel namesfine-grained scorescritique textoverall score

Quality

3 of 5

Influential for open preference tuning, but all ratings are by GPT-4 and known score-bias problems exist.

  • Annotator is an LLM with length and style bias
  • Some overall scores were later found inconsistent
  • Terms of the generating models may apply

Connects to

  • UltraChat: One source of prompts
  • HelpSteer2: Human-annotated alternative for reward modeling

Questions it could answer

  • How often does GPT-4 prefer the longest answer?
  • How much do scores disagree with a human-rated set like HelpSteer2?

Similar data sets

  • HelpSteer2

    Prompt-response pairs rated on five attributes, small enough to inspect.

    Language & textNVIDIA (with Scale AI)Free

  • UltraChat 200k

    A 200,000-dialogue filtered slice of ChatGPT-generated conversations that trained Zephyr-7B.

    Language & textHugging Face H4 team (filtered from Tsinghua's UltraChat)Free

  • GAIA

    450 questions that are simple for people but need browsing, tools, and multi-step reasoning for AI assistants.

    Internet & techGAIA benchmark team (Meta, Hugging Face, and AutoGPT collaborators)Free

  • GPQA

    Graduate-level science questions where outside experts score only 34 percent with Google.

    EducationDavid Rein and collaborators (NYU and others)Free

  • MMLU-Pro

    A harder MMLU with ten answer options and 12,000 reasoning-heavy questions.

    EducationTIGER-Lab (University of Waterloo and collaborators)Free

  • Chatbot Arena Conversations

    33,000 side-by-side model chats where real users voted for the better answer.

    Language & textLMSYS (UC Berkeley and collaborators)Free