You're currently viewing an old version of this dataset. To see the current version, click here.

Pt-Corpus

The dataset used for training the TeenyTinyLlama pair consists of a concatenation of open-source Brazilian Portuguese datasets, including Wikipedia, CulturaX, OSCAR, Common Crawl, and ROOTS. The dataset is filtered to exclude samples classified above a pre-defined toxicity threshold.

Data and Resources

Cite this as

Nicholas Kluge Corrêa, Sophia Falk, Shiza Fatimah, Aniket Sen, Nythamar de Oliveira (2025). Dataset: Pt-Corpus. https://doi.org/10.57702/t8bhdyjk

DOI retrieved: January 3, 2025

Additional Info

Field Value
Created January 3, 2025
Last update January 3, 2025
Defined In https://doi.org/10.1016/j.mlwa.2024.100558
Author Nicholas Kluge Corrêa
More Authors
Sophia Falk
Shiza Fatimah
Aniket Sen
Nythamar de Oliveira
Homepage https://huggingface.co/nicholasKluge/TeenyTinyLlama