Proof-Pile-2

The dataset used for continual pre-training of large language models, with a focus on balancing the text distribution and mitigating overfitting.

Data and Resources

Cite this as

Yunfan Shao, Linyang Li, Zhaoye Fei, Hang Yan, Dahua Lin, Xipeng Qiu (2024). Dataset: Proof-Pile-2. https://doi.org/10.57702/7iiqekx3

DOI retrieved: December 16, 2024

Additional Info

Field Value
Created December 16, 2024
Last update December 16, 2024
Defined In https://doi.org/10.48550/arXiv.2402.14526
Author Yunfan Shao
More Authors
Linyang Li
Zhaoye Fei
Hang Yan
Dahua Lin
Xipeng Qiu
Homepage https://arxiv.org/abs/2305.02309