TokCom-UEP: Semantic Importance-Matched Unequal Error Protection for Resilient Image Transmission
Proposes TokCom-UEP, integrating non-uniform semantic importance with rateless UEP coding, boosting image transmission robustness.
Key Findings
Methodology
This paper introduces the TokCom-UEP framework, leveraging large multimodal models' semantic understanding. It employs nested expanding windows to partition source tokens based on their semantic importance, assigning higher protection probabilities to critical tokens. The system comprises tokenization via the One-D-Piece model, EWF-based rateless UEP coding for differential protection, and multimodal-guided token restoration using Transformer architectures. The tokenization step converts images into position-dependent semantic tokens. UEP coding applies nested windows with tailored selection probabilities, prioritizing global semantic tokens. At the receiver, a Transformer-based model utilizes contextual cues and semantic priors to restore lost tokens, enhancing robustness. This integrated approach aligns channel protection with semantic importance, significantly improving transmission resilience and efficiency.
Key Results
- On ImageNet validation, TokCom-UEP achieves a 40.54% PSNR increase, a 40.15% LPIPS decrease, and an 11.36% CLIP score improvement at erasure probability p=0.2, outperforming equal protection baselines.
- Across varying channel conditions, TokCom-UEP maintains higher spectral efficiency, especially under low-overhead regimes, demonstrating adaptability.
- Ablation studies confirm the effectiveness of nested windows and multimodal-guided restoration, highlighting their contributions to robustness and efficiency.
Significance
This work addresses the critical limitation of uniform token importance assumptions in semantic communication frameworks. By integrating non-uniform semantic importance into channel coding, it enhances robustness against erasures and optimizes spectral efficiency, paving the way for reliable high-quality data transmission in future 6G networks. The approach bridges the gap between semantic understanding and physical layer protection, offering a new paradigm for intelligent data delivery in complex environments.
Technical Contribution
The key innovation lies in embedding semantic importance into rateless UEP coding through nested expanding windows, enabling differentiated protection aligned with semantic hierarchies. This contrasts with traditional equal error protection schemes, which treat all tokens uniformly. The system's modular design ensures compatibility with existing digital communication standards while significantly improving robustness and spectral efficiency. The integration of multimodal-guided token restoration further enhances resilience, representing a substantial advancement in semantic-aware channel coding.
Novelty
This is the first work to incorporate non-uniform semantic importance into TokCom's coding process via nested window UEP fountain codes. Unlike prior methods that assume uniform token significance, this approach explicitly models semantic hierarchies, enabling prioritized protection of critical tokens. The combination of large model-based tokenization, semantic importance-aware coding, and multimodal-based token repair constitutes a novel contribution to the field of semantic communication.
Limitations
- The system's performance may degrade under extremely high erasure rates where critical tokens are lost irrecoverably.
- Dependence on pre-trained models limits adaptability to unseen data distributions or domain shifts.
- Encoding complexity and computational costs are high, posing challenges for real-time deployment in resource-constrained environments.
Future Work
Future research will focus on adaptive importance estimation methods, reducing computational overhead, and extending the framework to other data modalities. Exploring end-to-end training strategies that jointly optimize tokenization, coding, and restoration modules could further enhance performance. Additionally, investigating lightweight models and hardware-friendly implementations will be crucial for practical deployment.
AI Executive Summary
In the rapidly evolving landscape of 6G networks, transmitting high-quality images reliably over noisy channels remains a formidable challenge. Traditional approaches, such as joint source-channel coding, often struggle to balance efficiency and robustness, especially under adverse conditions. Recent advances in large multimodal models have enabled semantic understanding at unprecedented levels, inspiring new paradigms like Token Communication (TokCom), which converts images into discrete semantic tokens for efficient transmission.
However, existing TokCom schemes typically assume uniform importance across tokens, leading to suboptimal error protection strategies. Critical semantic information, such as global structure, can be lost if all tokens are protected equally, especially in lossy channels. Recognizing this, the present work introduces TokCom-UEP, a novel framework that aligns error protection with the non-uniform semantic importance of tokens.
The core innovation involves partitioning tokens into nested expanding windows based on their contribution to image semantics. High-importance tokens, responsible for global structure, are prioritized during encoding using an EWF-based rateless UEP coding scheme. This approach dynamically allocates protection resources, ensuring critical tokens are more resilient to erasures. At the receiver, a Transformer-guided multimodal model leverages contextual cues and semantic priors to restore lost tokens, further enhancing robustness.
Extensive simulations on ImageNet demonstrate that TokCom-UEP significantly outperforms traditional equal error protection schemes. At an overhead of 1, it improves PSNR by over 40%, reduces perceptual loss by 40%, and boosts semantic similarity scores by 11%, especially under low-overhead conditions. Moreover, it maintains higher spectral efficiency across different erasure probabilities, confirming its adaptability and practical value.
This research marks a substantial step toward intelligent, semantic-aware data transmission in future wireless networks. By integrating semantic importance into physical layer coding, it offers a pathway to more resilient, efficient, and semantically faithful communication systems, with broad implications for multimedia, remote sensing, and AI-driven applications. Nonetheless, challenges remain in reducing computational complexity and enhancing generalization, guiding future investigations into lightweight, adaptive solutions for real-world deployment.
Deep Dive
Abstract
Based on the provided LaTeX code, here is the metadata for the submission form: Title: TokCom-UEP: Semantic Importance-Matched Unequal Error Protection for Resilient Image Transmission Author(s): Kaizheng Zhang, Zuolin Jin, Zhihang Cheng, Ming Zeng, Li Qiao, Zesong Fei Abstract: Token communication (TokCom), an emerging semantic communication framework powered by Large Multimodal Model (LMM), has become a key paradigm for resilient data transmission in 6G networks. A key limitation of existing TokCom designs lies in the assumption of uniform token importance, which leads to the adoption of equal error protection (EEP). However, compressed one-dimensional (1D) token sequences inherently exhibit heterogeneous semantic importance hierarchies, rendering EEP schemes suboptimal. To address this, this paper proposes TokCom-UEP, a novel semantic importance-matched unequal error protection (UEP) framework designed for resilient image transmission. TokCom-UEP integrates rateless UEP coding with the non-uniform semantic importance of tokens by partitioning source tokens into nested expanding windows, assigning higher selection probabilities to windows containing critical tokens to ensure their prioritized recovery. Simulation results demonstrate that TokCom-UEP outperforms EEP schemes in terms of three core semantic restoration metrics and spectral efficiency under low-overhead conditions.