Zero-Shot Machine Translation: Bridging the Gap between Pre-Trained and Random-Initialized Models

Abstract

Zero-shot machine translation (MT) aims to translate between language pairs that the model has not been explicitly trained on. This paper explores methods to improve zero-shot MT performance by bridging the gap between pre-trained models and those initialized randomly. We evaluate various approaches to leverage pre-trained models, including transfer learning, meta-learning, and cross-lingual embeddings. Our results demonstrate that incorporating pre-trained models significantly enhances zero-shot translation quality, providing new insights into effective strategies for leveraging such models.