Necessary and sufficient conditions for universality of Kolmogorov-Arnold networks

Kolmogorov-Arnold Networks (KANs) are an emerging neural network architecture inspired by the Kolmogorov-Arnold representation theorem, which states that any continuous function can be represented as a composition of univariate functions. Traditional multilayer perceptrons (MLPs) require a nonlinear activation function at each node, whereas KANs achieve this by assigning univariate functions to edges. This design theoretically enables KANs to approximate any continuous function. However, the universality conditions of KANs have not been systematically studied. With the rapid development of deep learning, researchers have become increasingly interested in the theoretical properties of KANs, particularly their approximation capabilities and expressive power.

The core problem of KANs is their universality, specifically whether they can approximate any continuous function on a given compact set K. The traditional view suggests that multiple non-affine functions are needed to achieve universality. However, this view increases the design complexity of KANs and limits their practical applicability. Therefore, identifying the universality conditions of KANs, particularly conditions that simplify their structural design, has become an important research topic.

The core innovation of this paper is proving that the universality of KANs can be achieved with just one non-affine function. This finding overturns the traditional view that multiple non-affine functions are necessary for universality. Specifically, the author proves that for deep KANs, a single non-affine function σ suffices to be dense in C(K). Additionally, for KANs with exactly two hidden layers, universality requires σ to be nonpolynomial. The author also demonstrates that KANs remain universal approximators even when using the spline-based edge parameterization introduced by Liu et al.

• �� Analyze the properties of KANs' edge functions to determine their universality conditions.
• �� Prove that for deep KANs, a single non-affine function σ suffices to be dense in C(K).
• �� Verify that for KANs with exactly two hidden layers, universality requires σ to be nonpolynomial.
• �� Study the application of the spline-based parameterization method proposed by Liu et al. in KANs, validating its universality.
• �� Introduce a finite set of affine functions, demonstrating that universality can be maintained even in deep networks.

The experimental design of this paper is primarily based on mathematical proofs, lacking large-scale empirical validation on real datasets. The author theoretically verifies the universality conditions of KANs, particularly their performance when edge functions are non-affine. Additionally, the author studies the application of the spline-based parameterization method proposed by Liu et al. in KANs, validating its classical universality. Future research can validate the theoretical results on large-scale real datasets to assess their practical applicability.

The study shows that deep KANs require only one non-affine function σ to be dense in C(K), proving their universality. This finding challenges the traditional view that multiple non-affine functions are needed. Additionally, for KANs with exactly two hidden layers, universality requires σ to be nonpolynomial. This result provides new theoretical insights for designing shallow networks. KANs with spline-based edge parameterization introduced by Liu et al. remain universal approximators, demonstrating their classical universality.

The universality study of KANs provides a new theoretical foundation for their design and optimization in practical applications. By simplifying the structural design of KANs, the results can be applied to various scenarios requiring complex function approximation, such as image recognition and natural language processing. Additionally, the universality study of KANs offers new perspectives on the expressive power of deep learning models, especially in complex function approximation tasks.

The theoretical results are primarily based on mathematical proofs, lacking large-scale empirical validation on real datasets, which may limit practical applicability. Additionally, selecting an appropriate non-affine function σ can be challenging, especially for specific tasks. Future research can validate the theoretical results on large-scale real datasets and explore how to select and optimize the non-affine function σ to enhance KANs performance in specific tasks.