A Hamiltonian-Inspired Local-Operator Ansatz for Slimming Large Language Models

The rapid evolution of large language models (LLMs) such as GPT, BERT, and LLaMA has driven parameter counts into the hundreds of billions, enabling unprecedented performance in NLP tasks. However, this growth introduces severe challenges in storage, energy consumption, and inference latency. Early efforts like weight pruning, low-rank factorization (e.g., LoRA), and sparsity have mitigated some issues but often at the expense of interpretability and robustness. Recent advances explore tensor networks and quantum-inspired structures, aiming to impose explicit organization on learned mappings. Despite progress, a fundamental understanding of the structural limits of model simplification remains elusive, especially at billion-parameter scales. This paper builds on these ideas, proposing a Hamiltonian-inspired local operator framework to push the boundary of model compression while maintaining performance.

Current compression techniques are limited by their reliance on low-rank assumptions or sparsity constraints, which restrict the expressive capacity of the approximated mappings. Deep models exhibit complex internal geometries and information flow patterns that are not fully captured by these methods. Moreover, the lack of a unified structural principle hampers systematic control over the trade-off between resource savings and model fidelity. The critical challenge is to design a scalable, interpretable, and executable structure that can approximate global linear maps with minimal parameters, especially at the billion-parameter scale, while identifying the fundamental limits of such simplifications.

This work introduces MixT, a neural linear map architecture inspired by the local decomposition of Hamiltonians in quantum physics. Key innovations include:

• �� Local tensor operators: representing global maps as sums of overlapping local tensors, reducing parameter count exponentially.
• �� Structural control: tuning local term number NT and replacing Transformer blocks NB to balance complexity and performance.
• �� Theoretical grounding: leveraging tensor network (MPO) theory to ensure exact representation and efficient evaluation.
• �� Empirical validation: demonstrating broad recoverability and a sharp performance boundary across multiple large models, revealing the intrinsic structural limits of simplification.

• �� Tensorization: map input features to a set of synthetic sites, defining local support regions.
• �� Local operator construction: create overlapping tensor operators acting on contiguous support supports, with parameters optimized via weight matching.
• �� Parameter control: vary NT to adjust the number of local terms, and replace specific Transformer blocks NB to impose the structure.
• �� Optimization: initialize local tensors by minimizing weight-matching loss, then fine-tune within the fixed structure.
• �� Evaluation: measure accuracy, output entropy, geometric similarity, and resource usage across different replacement depths, identifying the boundary where performance sharply declines.

Experiments replace self-attention and feed-forward maps in selected Transformer layers of Qwen3-8B and LLaMA2-7B models. Using MMLU and GSM8K benchmarks, the study assesses the impact of varying NT and NB on accuracy, parameter count, FLOPs, storage, and internal geometry. The models undergo progressive layer replacements, with performance monitored to identify the structural boundary. Additional diagnostics include output distribution entropy and layer-wise similarity metrics, revealing the internal reorganization associated with structural simplification.

Parameter reduction exceeds 50%, FLOPs decrease by 17-30%, and storage halves at the boundary, with performance loss within 3%. The boundary remains stable across NT variations, indicating a model-level structural limit. Output entropy and geometric drift metrics show abrupt changes at the boundary, correlating with sharp drops in task accuracy. The results validate the local-sum architecture's ability to approximate complex mappings while exposing the intrinsic structural limits of model simplification.

This approach enables efficient deployment of large models on resource-constrained hardware, reducing energy consumption and inference latency. It facilitates model compression for edge devices and accelerates inference without significant performance loss. The method also provides a framework for interpretability, enabling analysis of internal representations and their stability under structural simplification. Long-term, it could inspire hardware-aware model design and multi-modal extension.

The stability of the structural boundary depends on model architecture; its universality remains to be tested. Extreme compression may cause catastrophic performance drops. Support range selection impacts approximation quality, requiring adaptive strategies. Further research is needed to generalize the approach to diverse architectures and tasks, and to understand the theoretical nature of the observed phase transition.