Agentic Knowledgeable Self-awareness

Recent advances in pre-trained language models such as GPT-4, BERT, and their derivatives have revolutionized NLP and AI planning. These models have demonstrated remarkable abilities in multi-task learning, reasoning, and decision-making. Notable works like ReAct, Reflexion, and ExpeL have integrated reasoning and reflection mechanisms to improve planning performance. However, these approaches often rely heavily on external knowledge bases or trajectory imitation, lacking intrinsic self-assessment capabilities. This leads to brittle behaviors in novel environments, overfitting to training trajectories, and high inference costs. The challenge remains to endow models with human-like situational awareness, enabling them to decide when to reflect, query, or act directly, thus improving robustness and efficiency.

Despite progress, current LLM-based agents lack dynamic self-awareness, which is crucial for flexible decision-making in complex environments. Existing methods do not enable models to autonomously judge their situational capabilities, resulting in inefficient planning, excessive reliance on external knowledge, and poor generalization to unseen tasks. This gap hampers the deployment of autonomous agents in real-world scenarios where adaptability and resource management are vital. Developing a mechanism for models to internally assess their situational state and regulate knowledge usage remains an open and pressing problem.

The core innovation is the introduction of agentic knowledgeable self-awareness, achieved through: 1) a heuristic situation judgment criterion that classifies trajectories into three states; 2) special tokens that mark these states during data construction; 3) a two-stage training process combining supervised fine-tuning and RPO-based reinforcement learning to instill self-assessment capabilities. This approach enables models to generate situational tokens during inference, guiding resource allocation dynamically. Unlike prior methods that blindly inject external knowledge, this framework emphasizes autonomous judgment, reducing unnecessary knowledge invocation and improving planning robustness. The internal activation analysis further reveals that knowledge and action tokens emerge in the last layers, providing a neural basis for self-awareness.

• ��Knowledge system construction: offline collection of trajectories to form a lightweight knowledge base (K) and selection module (R).•Situation judgment criterion: heuristic rules classify trajectories into fast, slow, and knowledge-dependent states based on predicted vs. gold actions and rethinking outcomes.•Data augmentation: insert special tokens into trajectories according to classification, creating supervised training data.•Two-stage training:• Stage 1: autoregressive fine-tuning with cross-entropy loss to learn initial planning patterns.• Stage 2: RPO loss (Pang et al., 2024) combined with normalized negative log-likelihood to enhance self-awareness.•Inference: model generates special tokens indicating current situation, guiding whether to reflect, query knowledge, or proceed directly, thus enabling autonomous resource regulation.

Experiments conducted on ALFWorld and WebShop datasets evaluate models Gemma-2B and Llama-8B. Metrics include average reward, comparing with baselines like REACT, Reflexion, and ExpeL. The training involves hyperparameters such as learning rates (2e-5, 5e-7), batch sizes (8, 3), and loss weights (β=0.5). Ablation studies remove reflection or knowledge components to assess their impact. Generalization tests involve unseen tasks (Heat, Cool). Results show significant improvements with minimal knowledge (~15%), outperforming baselines, and demonstrating robustness across scales and tasks.

KnowSelf achieves up to 84.33% average reward on ALFWorld, surpassing baselines like REACT and Reflexion, with only 15-26% knowledge usage. It maintains performance on out-of-distribution tasks, indicating strong generalization. Ablation results confirm that removing reflection or overloading with knowledge reduces performance, highlighting the importance of self-assessment. Internal activation analysis shows knowledge and action tokens emerge in the last layers, validating the neural basis of self-awareness. These findings demonstrate that autonomous judgment and resource regulation significantly enhance planning robustness and efficiency.

This framework can be applied to autonomous robots, virtual assistants, and decision-support systems requiring adaptive planning. It enables models to operate efficiently with limited external knowledge, reducing costs and improving robustness. Future applications may include multi-modal environments, dynamic knowledge updating, and multi-task coordination, broadening the scope of autonomous AI in real-world scenarios.

The heuristic judgment rules may misclassify complex situations, affecting accuracy. Special token design requires manual tuning, limiting scalability. The reliance on last-layer activations may not generalize across architectures, necessitating further research. Additionally, computational costs of training and inference remain significant, especially for larger models. Future work should focus on automating judgment criteria, improving generalization, and reducing resource requirements.