Open Problem: Separating Geometric and Algorithmic Compression via Cayley-Table Completion

The evolution of statistical learning theory has been dominated by geometric biases such as low-rank, smoothness, and margin maximization, which effectively model continuous data. Early foundational work like compressed sensing and matrix completion demonstrated that convex relaxations of NP-hard properties enable efficient learning of low-dimensional manifolds. However, these geometric biases fall short in capturing discrete algebraic structures like groups, rings, and automata, which are fundamental in logic and formal languages. Recent research highlights the limitations of deep neural networks in learning exact algorithmic rules, especially in tasks requiring length generalization or strict symmetry enforcement. This gap motivates the need for new inductive biases that can natively encode discrete axioms, leading to the exploration of algebraic priors and differentiable measures of structure.

The core challenge is to develop a continuous, differentiable framework capable of exactly recovering discrete algebraic structures, such as group Cayley tables, without relying on combinatorial search. Existing methods either approximate these structures probabilistically or depend on external symbolic engines, which are computationally expensive and lack theoretical guarantees. The difficulty lies in defining a loss function that penalizes violations of algebraic axioms—like associativity—in a smooth manner, enabling gradient descent to converge to exact solutions. Establishing theoretical bounds on sample complexity and recovery guarantees remains an open problem, especially for large or complex structures. Addressing this gap is crucial for advancing the understanding of how neural networks can learn exact mathematical laws.

This work introduces a novel approach that combines operator-valued tensor factorization with a flatness prior, resulting in a differentiable measure of algebraic complexity. The key innovation is the loss function H(Θ), which decomposes into a collinearity term enforcing associativity and an inverse ℓ2 penalty promoting full-rank unitarity. This design ensures that gradient descent naturally converges to structures satisfying the axioms of a group, with theoretical proofs confirming that the global minimum corresponds to the regular representation of the underlying algebraic structure. Unlike prior methods relying on heuristic or probabilistic approximations, this approach provides a rigorous mathematical foundation for the native discovery of discrete axioms via continuous optimization.

• �� Define the structure tensor δ representing the Cayley table of a finite set Q.
• �� Construct an operator-valued tensor T(Θ) with parameters Θ = (A, B, C), where each slice corresponds to a linear operator.
• �� Design the loss function H(Θ) as the trace of the Hessian of the reconstruction error over observed entries, penalizing deviations from δ.
• �� Decompose H(Θ) into collinearity (alignment) and inverse ℓ2 norm components, promoting associativity and full-rank unitarity.
• �� Use gradient-based optimization to minimize H(Θ), with theoretical guarantees that the global minimum recovers the exact Cayley table of a group.
• �� Validate through experiments on finite groups, demonstrating sample efficiency and robustness.
• �� Theoretically prove that the infimum H_inf(δ) equals the algebraic complexity, with the minimum achieved only by the regular representation.

Experiments focus on finite groups like cyclic and symmetric groups. The datasets consist of sampled entries from the Cayley table, with varying sampling ratios (from O(n log n) to O(n^2)). Baselines include traditional matrix completion and heuristic algebraic learning methods. Metrics involve reconstruction error, convergence speed, and sample efficiency. Hyperparameters are tuned for regularization strength and optimization steps. Results show that the proposed method achieves exact recovery with O(n log n) samples, significantly outperforming classical bounds. Additional tests include partial observations, noise robustness, and scalability to larger n, confirming the method’s theoretical and practical advantages.

The experiments demonstrate that only O(n log n) samples are necessary for perfect Cayley table recovery across various finite groups, confirming the theoretical predictions. The method converges rapidly, often within few gradient steps, and maintains high accuracy even with noisy or partial data. Results also show that the recovered structures satisfy algebraic axioms exactly, validating the framework’s ability to natively discover discrete laws. These findings challenge the traditional belief that combinatorial search is essential for such tasks and highlight the potential of continuous optimization in symbolic learning.