Adapting BLOOM to a new language: A case study for the Italian
The BLOOM Large Language Model is a cuttingI think that the authors’ way of doing self-training is quite interesting. I suggest writing about self-training in the abstract and generally expose self-training as one of the paper’s contributions.-edge open linguistic model developed to provide computer...
Guardat en:
| Publicat a: | URI:https://journals.openedition.org/ijcol, |
|---|---|
| Autors principals: | , , , , |
| Format: | Article ou chapitre numérique |
| Idioma: | Anglais |
| Publicat: |
IJCoL
2024
|
| Accés en línia: | Accès Université d'Orléans et IFPM Accès Université d'Orléans et IFPM |
| Sumari: | The BLOOM Large Language Model is a cuttingI think that the authors’ way of doing self-training is quite interesting. I suggest writing about self-training in the abstract and generally expose self-training as one of the paper’s contributions.-edge open linguistic model developed to provide computers with natural language understanding skills. Despite its remarkable capabilities in understanding natural language by capturing intricate contextual relationships, the BLOOM model exhibits a notable limitation concerning the number of included languages. In fact, Italian is not included among the languages supported by the model, making its use challenging in this context. Within this study, we explore the language adaptation strategy based on continuing training on language-specific data. Moreover, we fine-tune both the BLOOM and the adapted models on several instruction datasets and different downstream classification tasks over EVALITA datasets. It has been observed that language adaptation followed by instruction-based fine-tuning is shown to be effective in correctly addressing a task never seen by the model in a new language learned on language-specific data. |
|---|