Post-Hoc Trajectory-Risk Certification for Modular LLM-Based Security Agents
A spanning-tree overlap bound certifies frozen LLM pipelines; 12 configurations achieved 92.7%±2.4% trajectory coverage.
Key Findings
Methodology
The paper represents stage failures as E_k and defines trajectory risk as TMR=Pr(∪E_k). For two stages it uses inclusion–exclusion; for K≥3 it proves that subtracting every pairwise overlap from the marginal sum is a lower, not upper, bound. The valid alternative is TMR≤Σp_k−Σ_(i,j)∈Tq_ij over any spanning tree T. The theory separates true dependence gain from gain certifiable with finite audits.
Key Results
- Across six open LLMs on CIC-IDS-2018 and RT-IoT2022, removing coarse-to-fine label nesting reduced apparent stage correlation from nearly 1 to 0–0.78, with a mean near 0.30. The same-model, cross-model, and permuted-pairing tests indicate shared sample difficulty rather than shared model representations.
- Once the joint audit reached the theoretically required size, direct trajectory-failure auditing was 13.7% tighter than Bonferroni; with an undersized audit it was worse. When only modular certificates were available, the spanning-tree-style certificate still produced a positive average gain of 0.6%.
- At α=0.10, mean trajectory coverage over 12 configurations was 92.7%±2.4%. Under cross-dataset deployment, single-step miscoverage reached 100% in all 12 tested cells, despite top-1 accuracy as high as 78% and empty prediction-set rates of 0.96–1.00.
Significance
The study formalizes a neglected gap between calibrated individual decisions and safety guarantees for an entire agent trajectory. Bonferroni remains distribution-free, but can create unnecessarily large prediction sets and excessive SOC escalation. Dependence-aware certification can recover part of this efficiency only when the dependence is both real and statistically identifiable. The cross-dataset result also separates accuracy from reliability: distribution shift can destroy calibrated confidence before it visibly destroys classification accuracy.
Technical Contribution
The paper develops a method-agnostic composition framework covering split conformal prediction, conformal risk control, and other marginal risk certificates. Theorem 4 establishes the parity direction of inclusion–exclusion partial sums; Theorem 6 supplies a valid finite-sample-certifiable spanning-tree upper bound for arbitrary depth; Theorems 10–11 give matching upper and information-theoretic lower audit-size bounds for certifying positive gain. It also formalizes label-induced coupling and dependence provenance.
Novelty
Unlike PASC, which jointly calibrates a maximum nonconformity score and requires simultaneous access to every stage’s raw score, this work targets frozen, independently calibrated modules that expose only certificates or pass/fail logs. Its novelty lies in post-hoc dependence certification, the correction of the tempting but invalid K>2 pairwise extension, and the proof that coarse-to-fine label design can manufacture near-perfect correlation without learned dependence.
Limitations
- The empirical pipeline has only two stages: traffic classification and attack-variant attribution. The K≥3 results and spanning-tree construction are theoretically general but are not tested on a real three-stage response system.
- The guarantees rely on exchangeability or a sufficiently representative audit distribution. Cross-dataset deployment violates this assumption severely, and dependence estimation cannot repair a shifted calibration distribution.
Future Work
Future work should combine online audits, drift detection, adaptive α allocation, and time-uniform confidence sequences. Real multi-stage response agents should compare PASC with modular certification under different access constraints. A particularly important direction is joint optimization of statistical TMR, observable escalation rate TER, prediction-set size, and operational cost.
AI Executive Summary
Security operations centers are increasingly considering LLM agents that classify traffic, attribute attacks, and recommend responses. Yet a conformal guarantee at each step does not automatically guarantee the whole trajectory: one uncovered stage can invalidate the decision chain. Bonferroni allocation remains valid under arbitrary dependence, but dividing α across K stages can produce oversized prediction sets and unnecessary human escalation.
Li proposes a post-hoc certification framework for frozen modular agents. For two stages, it uses inclusion–exclusion and an estimated overlap term. For three or more stages, it proves that the intuitive sum-minus-all-pairs formula is a lower bound, not an upper bound, and replaces it with a valid spanning-tree pairwise bound. The theory also distinguishes genuine stage dependence from dependence that a finite audit is large enough to certify. A coarse-to-fine label interface is shown to create near-perfect measured correlation mechanically.
Experiments with six open LLMs on CIC-IDS-2018 and RT-IoT2022 reduce the artifact-corrected correlation to 0–0.78, with mean about 0.30. Adequately sized joint audits improve on Bonferroni by 13.7%, while modular certification recovers a 0.6% average gain. At α=0.10, trajectory coverage averages 92.7%±2.4% across 12 configurations. Cross-dataset deployment, however, causes 100% single-step miscoverage despite 78% accuracy. The message is practical: security agents need audited trajectory risk, not accuracy or isolated confidence alone.
Deep Analysis
Background
Split conformal prediction constructs C_k(x)={y:s_k(x,y)≤q̂_k} from calibration quantiles and provides finite-sample marginal coverage under exchangeability. PASC can guarantee joint coverage using a maximum nonconformity score, but requires simultaneous access to all raw stage scores and joint calibration. Real systems often combine independently trained modules from different vendors or release cycles, leaving only per-stage certificates or audit logs.
Core Problem
Let E_k denote failure at stage k; trajectory miscoverage is TMR=Pr(∪E_k). Marginal bounds p_k≤α_k imply only TMR≤Σα_k, motivating Bonferroni with α_k=α/K. The challenge is to exploit correlated failures without mistaking sampling noise, deterministic label nesting, or model identity for genuine statistical dependence.
Innovation
The work contributes five linked ideas. It provides a general marginal-risk composition theorem; derives the exact K=2 correlation-aware expression; proves that the K≥3 second-order inclusion–exclusion expression has the wrong direction; introduces the spanning-tree pairwise upper bound; and derives oracle-versus-certifiable gain and audit-complexity results. It also proves that coarse-to-fine label selection mechanically couples failure events and uses model-pairing tests to identify shared sample difficulty.
Methodology
- �� Calibration: split each dataset and estimate q̂_k(α_k) for every stage using split conformal prediction.
- �� Risk representation: define E_k, p_k, pairwise overlaps q_ij, and TMR.
- �� Baseline: use Bonferroni, TMR≤Σα_k; for K=2, use p_1+p_2−ρ_12√[p_1(1−p_1)p_2(1−p_2)] as an operational plug-in estimate.
- �� Multi-stage certification: choose any spanning tree T and use TMR≤Σp_k−Σ_Tq_ij; replace q_ij with simultaneous lower confidence bounds when necessary.
- �� Audit analysis: distinguish structural dependence gain from finite-sample certifiable gain and test whether the audit crosses the required threshold.
Experiments
The evaluation uses six open LLMs, CIC-IDS-2018 and RT-IoT2022, three α levels, and five random seeds, totaling 36 configurations, with 12 focal cells at α=0.10. The two-stage pipeline performs traffic classification followed by attack-variant attribution. Baselines include Bonferroni, direct trajectory-failure auditing, pairwise correlation plug-ins, and the modular overlap certificate. Ablations test coarse-to-fine labels, same-model versus cross-model pairs, permuted pairing, and cross-dataset deployment.
Results
Correlation is near zero for CIC-IDS-2018/DoS and reaches 0.58 for RT-IoT2022/Probe, with a mean near 0.30. Adequate joint auditing is 13.7% tighter than Bonferroni, whereas an undersized audit reverses the advantage. The modular certificate gains 0.6% on average. Coverage is 92.7%±2.4% over 12 cells. Cross-dataset deployment yields 100% miscoverage, up to 78% accuracy, and empty-set rates of 0.96–1.00.
Applications
SOC operators can combine per-stage conformal sets, pass/fail logs, and audited overlaps to decide whether to block, quarantine, or escalate an alert. This is particularly useful for heterogeneous vendor modules because it does not require shared training or comparable score spaces. Deployment requires representative joint audit traffic, retained stage outputs, and explicit exchangeability or drift checks.
Limitations & Outlook
The empirical study does not include a real three-stage response pipeline, and TMR is not the same as the observable escalation rate TER. The correlation plug-in is an estimate rather than a distribution-free certificate and can be looser than Bonferroni under negative correlation. Severe cross-domain shift invalidates calibration; practical systems therefore need drift alarms, online recalibration, and cost-sensitive control.
Plain Language Accessible to non-experts
Imagine a factory with three inspection stations. The first asks whether a parcel is dangerous, the second identifies its type, and the third decides whether to stop it. Each station may promise that it rarely misses the truth. But the factory succeeds only when every station succeeds, so the whole line can be less reliable than any individual station.
The safest accounting method gives each station a tiny error budget. That is Bonferroni: dependable, but often so strict that many harmless parcels are sent to human inspectors. The paper notices that hard parcels often confuse several stations together. If this shared difficulty is demonstrated with enough audit examples, repeated errors need not be counted as though they were unrelated.
There is a trap, however. With three or more stations, subtracting every repeated pair can make the calculated danger too small. The authors instead subtract overlaps along a connecting tree, which preserves a safe upper estimate. They also show that a rule linking a broad category to a narrow one can create apparent agreement automatically. High correlation is not always intelligence; sometimes it is just the factory’s paperwork.
ELI14 Explained like you're 14
Think of an LLM security agent as a three-level game. Level one decides whether network traffic is harmless or an attack. Level two guesses the attack type. Level three chooses the action: block, quarantine, or ask a human. Each level can be pretty good, but the whole mission fails if even one level misses the true answer.
The obvious fix is to give every level a very small error allowance. That is safe, but it may make the agent shout “I’m unsure!” too often. The paper asks a clever question: do difficult examples make several levels fail together? If yes, counting those failures as completely separate is overly pessimistic.
But you cannot simply subtract every pair of repeated failures once there are three or more levels. The math can then claim the danger is tiny even when it is not. The authors use a connecting-tree trick to subtract only enough overlap to stay safe. They also test whether high agreement comes from the models or from the task itself.
The answer is surprising: after removing a label-design trick, correlation ranges from 0 to 0.78. And when the model moves to another dataset, it can still be 78% accurate while its confidence guarantee fails 100% of the time. Being right and knowing when you are reliable are different superpowers!
Glossary
Split Conformal Prediction
A calibration procedure that uses held-out nonconformity scores to construct prediction sets. Under exchangeability, it provides finite-sample coverage rather than merely trusting a softmax confidence score.
Each pipeline stage is independently calibrated with C_k(x).
Trajectory Miscoverage Rate
The probability that at least one stage fails to include its true label: TMR=Pr(∪E_k). It is a statistical risk and should not be confused with operational escalation frequency.
TMR is the principal trajectory-level quantity controlled by the paper.
Bonferroni Bound
A union bound that adds stagewise failure probabilities. Setting α_k=α/K guarantees TMR≤α without distributional assumptions, but can be conservative.
It is the distribution-free baseline.
Spanning-Tree Pairwise Bound
An upper bound that subtracts pairwise failure overlaps only along a tree connecting all stages. This preserves validity for arbitrary pipeline depth.
It enables modular certification when joint raw scores are unavailable.
Exchangeability
A condition under which calibration and test examples are probabilistically interchangeable. Distribution shift can violate it and invalidate conformal coverage.
It explains the 100% cross-dataset miscoverage result.
PASC
A joint conformal method based on the maximum nonconformity score across stages. It can be tighter, but requires simultaneous raw-score access and joint recalibration.
The paper contrasts PASC with post-hoc modular certification.
Open Questions Unanswered questions from this research
- 1 The spanning-tree bound has not been empirically tested on a real three-stage response agent. Future work must measure conditional dependencies among analysis, action, and feedback stages.
- 2 The theory controls TMR, not observable escalation rate TER. A practical objective should jointly optimize risk, prediction-set size, analyst workload, and response cost.
- 3 When cross-domain shift causes complete calibration failure, it remains unclear how many online labels are needed to restore guarantees through drift-aware conformal updating.
Applications
Immediate Applications
SOC alert routing
A security team can retain conformal sets and failure logs from traffic classification and ATT&CK attribution, then use Bonferroni or a spanning-tree certificate to decide automatic blocking, quarantine, or human escalation. Representative joint audit traffic is required.
Heterogeneous module validation
Vendors can evaluate independently developed LLM modules without sharing training data or score spaces. Shared audit examples, stage outcomes, and pairwise overlaps reveal joint risk, label-induced artifacts, and whether accuracy-based acceptance is misleading.
Long-term Vision
Auditable autonomous response
Trajectory certificates could attach a defensible risk bound to every automated block, isolation, or escalation. Realizing this vision requires online drift monitoring, adaptive audit collection, dynamic α allocation, and validation on deeper production pipelines.
Abstract
Autonomous security agents operate as staged pipelines, such as classifying network traffic and then attributing attacks to a specific technique. Split conformal prediction gives each stage finite-sample coverage, but deployment requires a trajectory-level guarantee across the full chain. These guarantees do not compose automatically when stages are independently trained and calibrated. Bonferroni allocation is distribution-free but conservative under correlated errors. We show that a natural pairwise-correlation extension to three or more stages is invalid because it gives a lower rather than an upper bound, and derive a valid spanning-tree alternative. We distinguish whether stages are dependent from whether an audit sample is large enough to certify that dependence, and give matching upper and information-theoretic lower sample-complexity bounds. We also show that coarse-to-fine label selection can create near-perfect measured correlation without learned dependence. On a two-stage intrusion-detection pipeline across 6 open LLMs and 2 datasets, removing this artifact reduces measured correlation from near 1 to 0-0.78. A direct audit of trajectory failure becomes 13.7% tighter than Bonferroni once the audit reaches the required sample size, but is worse when undersized. A modular certificate using per-stage certificates and a pairwise overlap bound yields a positive average gain of 0.6%, quantifying the cost of lacking joint access. Same-model, cross-model, and permuted-pairing tests show that residual dependence reflects shared sample difficulty, not shared model representations. Average trajectory coverage across 12 configurations is 92.7% +/- 2.4% at alpha = 0.10. Under cross-dataset deployment, single-step miscoverage reaches 100% even when accuracy remains 78%, showing that distribution shift destroys calibrated confidence before raw accuracy.