Pre-training via Paraphrasing
MARGE introduces a retrieval-based pretraining method for multilingual multi-document paraphrasing, achieving strong zero-shot performance on translation and summarization.
Mike Lewis, Marjan Ghazvininejad, Gargi Ghosh et al.