OPUS Parallel Corpora
University of Helsinki (Language Technology Research Group)
Over 100 billion aligned sentence pairs across more than 1,000 languages.
A collection of 1,200+ sentence-aligned translation corpora gathered from subtitles, EU and UN documents, software localization files, religious texts, and web crawls. Each pair links a sentence to its translation, which makes OPUS the first stop for training or evaluating machine translation, especially for smaller languages.
Access
- Cost
- FreeFree downloads and APIs.
- License
- Provider's own termsVaries by corpus; each corpus page states its source's terms
- Sign-up
- No sign-up
- How you get it
- API and Bulk downloadMoses plain text, TMX, XML
What's in it
- One row is
- One row per aligned sentence pair within a corpus and language pair
- Covers
- 1,038 languages
- Years
- Varies by corpus
- Updated
- Irregular
- Size
- About 102.9 billion sentence pairs across 1,214 corpora
Quality
The most complete hub for parallel text, though quality ranges from professional translation to noisy web-mined pairs.
- OpenSubtitles and NLLB make up nearly half the pairs and are noisy
- Licenses differ corpus by corpus
- Low-resource pairs are often mostly religious or software text
Connects to
- Universal Dependencies Treebanks: Grammatical annotation for many of the same languages
- Mozilla Common Voice: Speech data for many of the same lower-resource languages
Questions it could answer
- For which languages does nearly all available parallel text come from Bible translations?
- How much aligned text links each African language to English versus French?