cs.LG 2606.24998

Internal Data Repetition Destroys Language Models

Using a fitted no-repetition scaling law, quantifies systematic damage caused by repeated data, revealing an intermediate repeat count causes maximum loss, with damage scaling as a power law with model size.

Jessica Chudnovsky, Joshua Kazdan, Noam Levi et al.

2026-06-24 42
cs.LG 2606.19878

On the Oracle Complexity of Interpolation-Based Gradient Descent

Proposes Piecewise Polynomial Interpolation-based Gradient Descent (PPI-GD) achieving oracle complexity of O((p/ε)^{d/(2ℓ)}) for data dimension d=O(log^{0.49}(n)), outperforming classical GD/SGD.

Dongmin Lee, William Lu, Anuran Makur

2026-06-18 164