Linguistic Profiling of a Neural Language Model
Using 68 probing tasks, the study finds BERT loses linguistic information after fine-tuning, while more readable representations improve NLI accuracy.
Alessio Miaschi, Dominique Brunato, Felice Dell'Orletta et al.