A Quantitative Characterization of Forgetting in Post-Training

Generative models play a crucial role in modern machine learning, especially in fields like image and text generation. However, as tasks increase, generative models face the problem of catastrophic forgetting during continual training, where learning new tasks leads to forgetting old ones. Although various algorithms have attempted to address this issue, such as replay mechanisms and regularization methods, a systematic understanding of the forgetting mechanisms remains lacking. This paper provides a theoretical foundation for post-training processes of generative models by quantifying forgetting during continual training.

The core problem is the phenomenon of forgetting in generative models during continual training. Specifically, when models are trained on new tasks, the performance on old tasks often degrades significantly. The challenge lies in maintaining memory of old tasks while learning new ones. Existing solutions mainly focus on algorithmic responses but lack a systematic understanding of the forgetting mechanisms.

The core innovations of this paper include the first quantification of forgetting in generative models during continual training using a two-mode mixture abstraction model. • Analyzing the different impacts of forward and reverse KL objectives under different data distributions to reveal the mechanisms of forgetting. • Presenting behavioral analysis of forward and reverse KL objectives, revealing the role of replay mechanisms in preventing forgetting. • Analyzing recently proposed near-on-policy post-training methods, providing new theoretical guarantees and engineering possibilities.

• �� Employing a two-mode mixture abstraction model to abstract the continual training process of generative models into a mixture of 'old task' and 'new task' distributions. • Analyzing the different impacts of forward and reverse KL objectives under different data distributions to reveal the mechanisms of forgetting. • Quantifying the effectiveness of reverse-KL objectives in retaining the quality of old tasks while learning new ones through overlap-gated misassignment probabilities controlled by the Bhattacharyya coefficient. • Further quantifying replay interactions with these objectives, revealing the critical role of replay in preventing forgetting.

The experimental design includes training on equal-covariance Gaussian modes using forward and reverse KL objectives. By comparing the performance of different objectives on new data, the impact on the quality of old tasks is analyzed. The experiments also analyze the effectiveness of replay mechanisms under different objectives, revealing the critical role of replay in preventing forgetting.

Experimental results show that reverse-KL objectives can effectively learn new tasks while retaining the quality of old tasks. Forward-KL objectives trained on new data drive the old weight to zero, while reverse-KL objectives avoid mass forgetting through overlap-gated misassignment probabilities controlled by the Bhattacharyya coefficient. Replay interactions with these objectives are further quantified, revealing the critical role of replay in preventing forgetting.

The research findings can be directly applied to the continual training of generative models, especially in scenarios where multiple tasks need to be learned simultaneously. By quantifying forgetting in generative models during continual training, it provides a theoretical foundation for post-training processes of generative models. In industry applications, generative models can be applied based on the analysis in this paper, choosing appropriate objectives and replay mechanisms to avoid forgetting.

The analysis is primarily based on equal-covariance Gaussian modes, which may not directly generalize to other distributions. The effectiveness of replay mechanisms depends on the selection and distribution of training data, which may be limited in practical applications. The computational cost and time consumption for complex models are not detailedly analyzed. Future research directions include extending the methods to more diverse data distributions and model structures, exploring the effectiveness of replay mechanisms in different application scenarios, and developing more efficient algorithms to reduce computational costs.