MTEB-BR - Brazilian Portuguese (PT-BR)

A public benchmark for evaluating text embedding models on Brazilian Portuguese, built on top of the mteb library.

What you'll find here

• 🏆 Leaderboard — interactive ranking, 93 models × 22 tasks, with Pareto chart
• 💻 GitHub repo — task definitions, evaluation scripts, paper sources, issue templates
• 📚 Task list & sources — every task linked to its original dataset / paper
• 📄 Paper on arXiv (MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguese)

Submit a model

Two channels — pick whichever fits:

Required for a submission:

1. model_id (HF repo path or vendor product name)
2. Per-task result JSONs for the 16 headline tasks
3. Reproducible evaluation command

We re-run a sample of each submission to verify before merging.

Propose a new task

Open a GitHub Issue with the task template describing the dataset, license, size, and discrimination evidence. A task is accepted if it's native PT-BR (not machine-translated), has clear licensing, and discriminates between models.

Maintainer

Tardelli Stekel — IFSP, São Paulo, Brazil
✉️ stekel@ifsp.edu.br

Contributions, corrections, and discussion all welcome.

Citation

@misc{mteb-br-2026,
  title  = {MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguese},
  author = {Stekel, Tardelli R. C.},
  year   = {2026},
  eprint = {2607.04581},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL},
  doi    = {10.48550/arXiv.2607.04581},
  url    = {https://arxiv.org/abs/2607.04581}
}

Acknowledgments

Built on top of the mteb library (Muennighoff et al., 2023). The multilingual sub-benchmark methodology follows MMTEB (Enevoldsen et al., 2025). Task datasets contributed by their original authors — see the task suite for sources.