Learning-Based Collaborative MEC for LLM Inference with Soft-Deadline Awareness via Transformer-Enhanced PPO
Transformer-enhanced PPO optimizes MEC collaboration for LLM inference, boosting task completion by 15%.
Key Findings
Methodology
This study develops a Transformer-augmented PPO framework, employing self-attention to model temporal dependencies and cross-server interactions. The system models tasks as DAGs within an MDP, incorporating system states like queue lengths, dependencies, and communication conditions. A normalized reward function guides policy learning, balancing task success and deadline extensions. The approach uses a Transformer encoder with multi-head self-attention, combined with policy and value networks, trained via PPO with advantage estimation. The system effectively captures long-term dependencies, enabling robust task migration decisions.
Key Results
- Simulation shows the proposed method improves task completion rate by 15% over conventional PPO, reaching over 90% success in high load scenarios. System efficiency increases by 20%, with deadline satisfaction rising from 75% to 85%. Ablation studies confirm the importance of Transformer in modeling long-term dependencies, with performance degradation when replaced by MLPs. The method also reduces deadline extensions, conserving resources.
- Across varying user counts (10-50) and request complexities, the model maintains stable performance, with average latency reduced by 12%. It effectively limits deadline extensions, optimizing resource use and system throughput.
Significance
This work addresses the challenge of long-term dependency modeling in dynamic MEC environments, enabling reliable, real-time LLM inference at the edge. It bridges the gap between traditional RL methods and sequence modeling, offering a scalable solution for intelligent edge systems. The approach enhances system robustness, resource efficiency, and task success rates, supporting next-generation AI services with strict latency and privacy demands.
Technical Contribution
The key innovation is integrating Transformer encoders into PPO, allowing the model to learn complex temporal and cross-node relationships. The reward design and system modeling ensure stable training and aligned optimization goals. This framework extends RL capabilities to sequence-dependent, resource-constrained environments, providing a new paradigm for edge AI orchestration.
Novelty
This is the first application of Transformer mechanisms within edge computing RL frameworks for task migration, especially under soft deadline constraints. It uniquely combines long-term dependency capture with deadline extension control, setting a new benchmark for intelligent MEC scheduling. The approach outperforms existing methods that rely solely on instantaneous observations or heuristic rules.
Limitations
- The model's performance heavily depends on accurate system state information; in real networks with high volatility, robustness may decline.
- Training requires significant computational resources and large datasets, limiting immediate deployment.
- Current validation is simulation-based; real-world heterogeneity and unpredictable network conditions pose additional challenges. Future work should focus on robustness, scalability, and real-world testing.
Future Work
Future directions include integrating federated learning for better generalization, extending to multi-task multi-user scenarios, and optimizing energy consumption. Enhancing robustness against network volatility and deploying in real edge environments are key next steps.
AI Executive Summary
The rapid growth of AI applications, especially large language models (LLMs), demands efficient, low-latency inference at the network edge. Traditional single-edge servers struggle with computational loads, especially when tasks depend on complex dependencies and strict deadlines. To address this, the paper proposes a novel framework combining Transformer mechanisms with proximal policy optimization (PPO) to enable collaborative MEC systems.
This approach models tasks as dependency graphs and employs a Transformer encoder to capture long-term temporal dependencies and cross-server interactions. The reinforcement learning agent learns to make migration decisions, balancing task success and deadline extensions, guided by a carefully designed reward function. The system dynamically adapts to changing network conditions, user mobility, and workload fluctuations.
Experimental results in a simulated environment with six heterogeneous MEC servers demonstrate that the proposed method outperforms baseline PPO and heuristic algorithms, achieving a 15% higher task completion rate and a 20% increase in overall system efficiency. The model effectively limits deadline extensions, conserving resources and improving reliability. These findings highlight the potential of Transformer-enhanced RL for real-time, scalable edge AI deployment.
This work significantly advances the state-of-the-art in MEC task scheduling, offering a robust, scalable solution for future AI-driven edge services. Future research will explore multi-task scenarios, energy efficiency, and deployment in real-world networks, aiming to realize truly intelligent, autonomous edge ecosystems.
Deep Analysis
Background
边缘计算作为AI应用的基础设施，近年来得到快速发展。早期工作如Fog Computing和MEC解决了延迟问题，但面对大规模LLM模型，单节点计算能力不足。多边边缘协作成为研究热点，相关方法包括联邦学习和任务迁移，但在信息融合和长时依赖建模方面仍有挑战。深度强化学习（如DQN、A3C）已被应用于调度，但在动态环境中表现有限。Transformer机制在序列建模中的优越性，为边缘调度提供了新思路。
Core Problem
核心问题在于如何在多边边缘环境中，利用深度强化学习实现高效、鲁棒的任务迁移策略，满足LLM推理的严格时延和依赖关系。传统方法难以捕获长时依赖，且在网络波动和任务突变时表现不佳。任务复杂，截止期限有限，迁移决策需考虑多源信息和未来状态，模型的时序建模能力和实时性成为难点。
Innovation
本研究的创新点包括：1）引入Transformer编码器，增强对长时依赖和跨节点交互的建模能力；2）设计软截止期限机制，结合截止期限扩展限制，平衡效率与资源；3）提出归一化奖励函数，确保策略稳定性；4）将任务表示为DAG，考虑依赖关系，优化迁移策略。这些创新在保证任务成功率的同时，减少截止期限扩展，提升系统性能。
Methodology
- �� 建立多边边缘系统模型，定义任务DAG、资源异质性、通信条件和截止期限。• 将调度问题转化为MDP，状态包括任务依赖、队列、通信和资源信息。• 利用Transformer编码器处理历史状态序列，捕获长时依赖。• 设计策略网络和价值网络，输出迁移决策和状态价值。• 采用PPO算法，通过剪切目标和优势估计训练。• 设计奖励函数，结合任务完成和截止期限扩展，平衡效率和资源。• 训练模型，优化迁移策略，提升任务在截止期限内完成率。• 在模拟环境验证性能，比较不同方法的任务成功率和效率。
Experiments
模拟环境包括6个异质MEC服务器，用户数10-50，任务复杂度30-50 tokens。模型在不同负载下训练，比较传统PPO、启发式和本文方法。指标包括任务完成率、截止期限满足率、资源利用率和延迟。超参数调优后，结果显示Transformer增强模型在所有指标上优于对比方法，特别在高负载场景中表现更稳健。
Results
Transformer增强的PPO在任务完成率上比传统PPO提升15%，在高负载场景中成功率超过90%。系统效率提升20%，截止期限满足率由75%提升至85%。 Ablation研究验证Transformer在捕获长依赖中的关键作用，模型在迁移决策中表现出更强鲁棒性，能有效限制截止期限扩展，节省资源。
Applications
该方法适用于边缘端大规模LLM推理、智能制造、自动驾驶等场景，能显著降低延迟、提升隐私保护。部署前需配置多边边缘服务器，确保通信和计算资源充足。未来可结合异构性和动态变化，拓展多任务、多用户调度策略，推动边缘智能普及。
Limitations & Outlook
模型对系统状态依赖较大，在网络极端波动或突发任务时性能可能下降。训练成本高，实际部署面临样本和计算限制。验证仅在模拟环境，真实场景中的异质性和动态性仍需研究。未来应增强模型鲁棒性和泛化能力，降低部署难度。
Plain Language Accessible to non-experts
想象你在一个大厨房准备多道菜。每道菜有不同的步骤和时间，有的还依赖前一道菜完成。厨房里有几个厨师（边缘服务器），每个能力不同。有时，厨师忙不过来，你需要把一些菜转给别的厨师帮忙。你要合理安排，让所有菜都能按时端出。这个过程就像AI帮你决定哪个厨师做什么，什么时候迁移任务，确保每道菜都在时间内完成。Transformer机制就像一个聪明的助手，记住厨房的情况，帮你做更好的安排。最终目标是让所有菜都准时出锅，节省时间和资源。
ELI14 Explained like you're 14
想象你在一个大厨房里准备很多菜。有的菜需要先做完前面的步骤才能继续，有的做得快，有的慢。厨房里有几个厨师（就像边缘服务器），每个厨师的能力不同。有时候，厨师忙不过来，你可能需要把一些菜转给别的厨师帮忙。你要聪明地安排，让所有菜都能在规定时间内做好。这个安排就像AI在帮你决定哪个厨师做哪个菜，什么时候换人，确保每道菜都能按时端出来。Transformer就像一个特别聪明的助手，记住厨房的情况，帮你做更好的安排。这样一来，所有菜都能准时上桌，大家都很满意！
Glossary
Transformer (变换器)
一种基于自注意力机制的序列建模模型，能捕获长时依赖关系。
用于增强任务调度中的时间依赖建模。
PPO (近端策略优化)
一种强化学习算法，通过剪切目标稳定训练过程。
用于训练边缘任务迁移策略。
MDP (马尔可夫决策过程)
描述系统状态转移的数学模型，未来状态仅依赖当前状态。
系统建模和策略优化的基础。
任务依赖图 (DAG)
有向无环图，表示任务间的依赖关系。
建模LLM推理任务的结构。
软截止期限 (Soft Deadline)
允许一定程度的延期，超出后可进行截止期限扩展。
调度策略中的时间约束。
Open Questions Unanswered questions from this research
- 1 如何在真实边缘网络中应对极端网络波动和突发任务，仍需验证模型的鲁棒性和适应性。
- 2 多任务、多用户环境下的调度策略优化仍是未来研究重点。
Applications
Immediate Applications
边缘大规模LLM推理
在智能手机、边缘服务器部署高效推理，降低延迟，保护隐私。
智能制造调度
工业场景中多设备协作，保证生产任务按时完成，提升效率。
Long-term Vision
边缘智能生态系统
实现跨设备、跨平台的协作，推动智慧城市和自动驾驶等行业变革。
Abstract
This paper investigates collaborative mobile edge computing (MEC) servers for large language model (LLM) inference under soft deadline constraints. In this system, to improve the quality of service, computations are expected to be completed within their deadlines. However, due to dependencies among tasks or subtasks, any missed deadline can lead to catastrophic consequences for the entire request. In this context, this work proposes an extended deadline mechanism with constrained flexibility. The main challenges lie in handling large-scale computations under strict latency constraints while limiting the number of allowable deadline extensions, especially in the presence of task dependencies within each request. To tackle these challenges, we develop a transformer-enhanced proximal policy optimization (PPO) framework that enables efficient collaboration among MEC servers. The proposed approach aims to maximize the number of tasks completed within their deadlines while minimizing the use of deadline extensions. By capturing temporal dependencies and cross-server interactions, the transformer improves decision-making for task migration. Simulation results demonstrate that the proposed method significantly outperforms conventional PPO and heuristic-based approaches in terms of task completion rate and overall system efficiency.