What’s Out There
← Back to swiping
Language & textFreeCompany

MultiCoNER Datasets

Amazon

Multilingual named-entity recognition data with hard cases like movie titles and short text.

MultiCoNER 1 is a large multilingual named-entity recognition dataset in 11 languages, built to represent contemporary challenges: short, uncased text, syntactically complex entities such as movie titles, and long-tail entity distributions. MultiCoNER 2 extends the challenge to fine-grained entities.

Access

Cost
FreeNo charge to download; hosted in an AWS S3 bucket.
License
Open licenseCC BY 4.0
Sign-up
No sign-up
How you get it
API and Bulk downloadsee documentation

What's in it

One row is
One sentence with entity spans and labels
Covers
Not geographic (11 languages in version 1)
Years
Not stated
Updated
One-time release
Size
Not stated
tokenentity spanentity typelanguagesentence

Quality

4 of 5

Purpose-built to be hard, with shared-task evaluation. Mostly short, partly synthetic-style text.

  • Short context
  • Annotation noise in long-tail entities

Connects to

  • Common Crawl: Web text source for building multilingual NER data
  • Sudachi Language Resources: Language resources for one of the languages

Questions it could answer

  • Which entity types are hardest across languages?
  • How much does limited context hurt NER accuracy?

Similar data sets

  • COCO (Common Objects in Context)

    330,000 everyday photos with every person, dog, and fork outlined by hand.

    Images & videoCOCO Consortium (Microsoft and academic partners)Free

  • Dolma

    The three-trillion-token corpus behind OLMo, with the recipe and tooling published alongside it.

    Language & textAllen Institute for AIFree

  • OPUS Parallel Corpora

    Over 100 billion aligned sentence pairs across more than 1,000 languages.

    Language & textUniversity of Helsinki (Language Technology Research Group)Free

  • Corpus of Contemporary American English (COCA)

    A billion words of American English, balanced across genres from TV scripts to academic journals.

    Language & textEnglish-Corpora.org (Mark Davies)Free tier, paid upgrades

  • SmolLM-Corpus

    The mix of synthetic textbooks, educational web text, and Python code that trained small SmolLM models.

    Language & textHugging Face (SmolLM team)Free

  • Common Corpus

    2.27 trillion tokens of text that is either uncopyrighted or openly licensed, with per-document provenance.

    Language & textPleiasFree