Which Similarity-Sensitive Entropy (Sentropy)?

Information entropy originated from Shannon's work, serving as a core measure of uncertainty in systems. Over time, various generalizations like Rényi and Tsallis entropies emerged, focusing on frequency distributions. However, these traditional measures ignore relationships among elements. Recent advances introduced sentropy, which incorporates similarity matrices, capturing richer structural information. The LCR framework, based on spectral properties of similarity matrices, has been applied in biology and machine learning. The Vendi Score, leveraging eigenvalue spectra, offers a global perspective. Despite their success, a systematic comparison and theoretical understanding of their relationship remained lacking, motivating this study.

The core challenge lies in understanding when and how to choose between LCR and VS for measuring system diversity. Both depend on similarity scales, which are often arbitrarily set, leading to inconsistent results. The lack of a unifying theoretical framework hampers their broad application. Moreover, their performance varies across data types—local versus global structures—necessitating a rigorous comparison. Addressing these issues is crucial for advancing the utility of sentropy in complex data analysis, especially in high-dimensional, heterogeneous, or quantum-inspired systems.

The paper's innovations include: 1) establishing a theoretical bound showing VS as an upper limit for LCR across all q, providing a fundamental relationship; 2) introducing the 'half-distance' parameter to control similarity sensitivity, enabling flexible adaptation; 3) conducting extensive empirical comparisons across diverse datasets, revealing their complementary strengths; 4) extending the framework to negative q values, relevant for certain applications. These contributions deepen the understanding of similarity-sensitive entropy, bridging classical and quantum perspectives, and offering practical guidance for data analysis.

• �� Construct diverse datasets, including medical images and tabular data, ensuring heterogeneity. • Generate similarity matrices using Euclidean-based exponential functions, tuning the 'half-distance' parameter. • Calculate LCR via spectral analysis of the similarity matrix, deriving effective number of elements. • Compute VS from the eigenvalues of the similarity matrix, evaluating Shannon entropy. • Theoretically prove VS bounds LCR for all q ≥ 0, and for negative q when matrices are full rank. • Empirically compare measures across datasets, analyze parameter effects, and validate theoretical bounds. • Use clustering and other metrics to interpret the measures' relevance.

• �� Selected 53 datasets covering vision, medical imaging, and tabular data, with sizes from thousands to hundreds of thousands. • Constructed similarity matrices based on Euclidean distances, adjusting 'half-distance' to examine scale effects. • Calculated LCR and VS for multiple q values, focusing on q=1 for comparison. • Used clustering algorithms (HDBSCAN) to relate entropy measures to data structure. • Analyzed how measures vary with parameters, data type, and complexity, validating theoretical bounds. • Assessed computational efficiency on modern hardware, ensuring practical feasibility.

• �� Quantitative differences between LCR and VS can reach orders of magnitude, with VS often larger. • Theoretical proof confirms VS as an upper bound on LCR for all non-negative q, and for negative q in full-rank matrices. • Empirical results show LCR captures local element differences, useful in heterogeneous biological systems. • VS reflects global structure, especially in high-dimensional or quantum-like data, providing a complementary perspective. • Adjusting 'half-distance' tunes sensitivity, enabling tailored analysis. These insights guide optimal method selection based on data characteristics.

• �� Medical imaging: enhancing diagnostic models by capturing complex structural information. • Bioinformatics: analyzing immune repertoires and microbiomes with detailed diversity metrics. • Machine learning: feature characterization, data augmentation, and model robustness. • Future integration with deep learning for automatic similarity scale tuning and multi-modal data fusion, expanding into quantum-inspired systems for advanced information processing.

• �� Sensitivity to similarity matrix construction and scale parameters, requiring careful tuning. • Theoretical guarantees depend on matrix properties; non-positive semi-definite matrices pose challenges. • Computational costs increase with dataset size, limiting scalability. • Application to dynamic or streaming data remains unexplored, necessitating further research. Future work should address robustness, efficiency, and broader applicability.