What’s Out There
← Back to swiping
Language & textFreeCompany

HH-RLHF (Helpful and Harmless)

Anthropic

Human preference comparisons that helped define how chat assistants are trained to be helpful and harmless.

Two kinds of data from Anthropic's 2022 RLHF work: pairs of assistant replies where a human picked the better one on helpfulness or harmlessness, and red-team transcripts where crowdworkers tried to provoke harmful replies. One row is a chosen and a rejected conversation. The card says it is for preference models, not for supervised dialogue training.

Access

Cost
FreeNo charge.
License
Open licenseMIT
Sign-up
No sign-up
How you get it
API and Bulk downloadJSON Lines

What's in it

One row is
One row per pair of chosen and rejected conversations (preference data) or per red-team transcript
Covers
Not geographic
Years
2022
Updated
One-time release
Size
100,000 to 1 million rows across the preference and red-team subsets
chosen conversationrejected conversationred-team transcriptharmlessness ratingtask description

Quality

4 of 5

Historically important and well described in a paper, but crowdworker judgments on an early model are noisy by modern standards.

  • Contains offensive and harmful content by design
  • Labeler agreement is modest and reflects a small crowd
  • Responses came from early 2022 models

Connects to

Questions it could answer

  • Which topics draw the most harmful completions in the red-team transcripts?
  • How often do raters prefer the longer reply?

Similar data sets

  • HelpSteer2

    Prompt-response pairs rated on five attributes, small enough to inspect.

    Language & textNVIDIA (with Scale AI)Free

  • PersonaHub

    Short persona descriptions that steer an LLM into generating varied synthetic data.

    Language & textTencent AI Lab (proj-persona)Free

  • UltraFeedback

    64,000 prompts, four model answers each, scored by GPT-4 on four aspects.

    Language & textOpenBMB (Tsinghua University)Free

  • OpenAssistant Conversations (OASST1)

    161,443 chat messages written and ranked by volunteers in 35 languages, with no model in the loop.

    Language & textOpenAssistant (LAION-affiliated volunteers)Free

  • Chatbot Arena Conversations

    33,000 side-by-side model chats where real users voted for the better answer.

    Language & textLMSYS (UC Berkeley and collaborators)Free

  • NaturalReasoning

    Millions of hard questions back-translated from pretraining text and decontaminated.

    Language & textMeta (Facebook AI Research)Free