In authors or contributors

2 resources

  • Ramon Pires, Hugo Abonizio, Thales Sales...
    |
    Dec 28th, 2023
    |
    preprint
    Ramon Pires, Hugo Abonizio, Thales Sales...
    Dec 28th, 2023

    As the capabilities of language models continue to advance, it is conceivable that "one-size-fits-all" model will remain as the main paradigm. For instance, given the vast number of languages worldwide, many of which are low-resource, the prevalent practice is to pretrain a single model on multiple languages. In this paper, we add to the growing body of evidence that challenges this practice, demonstrating that monolingual pretraining on the target language significantly improves models...

  • Ramon Pires, Hugo Abonizio, Thales Sales...
    |
    Dec 28th, 2023
    |
    preprint
    Ramon Pires, Hugo Abonizio, Thales Sales...
    Dec 28th, 2023

    As the capabilities of language models continue to advance, it is conceivable that "one-size-fits-all" model will remain as the main paradigm. For instance, given the vast number of languages worldwide, many of which are low-resource, the prevalent practice is to pretrain a single model on multiple languages. In this paper, we add to the growing body of evidence that challenges this practice, demonstrating that monolingual pretraining on the target language significantly improves models...

Last update from database: 28/12/2024, 08:15 (UTC)
Powered by Zotero and Kerko.