Towards 3D Scene Understanding of Gas Plumes in LWIR Hyperspectral Images Using Neural Radiance Fields

High-resolution hyperspectral imaging (HSI) has become vital for environmental and security applications, especially in detecting gases like SF6. Existing methods focus on 2D spectral analysis or simple 3D reconstruction, often limited by data scarcity and computational complexity. Recent advances in neural rendering, notably NeRF, have revolutionized 3D scene modeling from RGB images, but their extension to LWIR hyperspectral data remains unexplored. Synthetic datasets from DIRSIG have enabled preliminary studies, yet real-world validation and spectral fidelity remain challenges. The need for models capable of few-shot learning, preserving spectral details, and accurately modeling gas geometries motivates this research, aiming to bridge the gap between high-dimensional spectral data and volumetric scene understanding.

Reconstructing 3D gas plumes from LWIR hyperspectral images is hindered by high data dimensionality, limited viewpoints, and spectral absorption complexities. Traditional methods struggle with sparse data, leading to inaccurate geometry and spectral estimates. Accurate detection and quantification require detailed scene models, but existing approaches lack the ability to integrate multi-view spectral information effectively. The core challenge is developing a model that can learn volumetric scene properties with minimal training data, capturing both the geometric structure and spectral signatures of gases, which is crucial for reliable detection and analysis in real-world scenarios.

The paper introduces a multi-channel density NeRF that predicts spectral-specific densities, enabling detailed spectral and geometric modeling. An adaptive weighted MSE loss dynamically emphasizes spectral bands with higher residuals, improving absorption feature reconstruction. Geometric regularization extended to multi-channel densities ensures scene consistency, while sampling space annealing stabilizes training with sparse views. The architecture employs integrated positional encoding to capture high-frequency details, and the entire framework is optimized for few-shot learning. These innovations collectively enable high-quality 3D reconstruction of LWIR gas plumes with significantly fewer training samples than prior methods.

• �� Generate synthetic LWIR hyperspectral data using DIRSIG, simulating gas plumes with known ground truth.
• �� Extend NeRF architecture to output multi-channel densities and spectral radiance, incorporating integrated positional encoding.
• �� Design a loss function combining L2, spectral angle mapper (SAM), and an adaptive weighted L2 (AW-L2) component, which adjusts channel importance based on residuals.
• �� Implement geometric regularization by calculating expected depth per channel and enforcing smoothness across patches.
• �� Use sampling space annealing to constrain ray sampling ranges during early training, improving convergence.
• �� Train the model with sparse views (30 images), applying regularization and adaptive loss components.
• �� Render novel views from the trained NeRF, then apply ACE for gas plume detection, evaluating detection accuracy and scene fidelity.

The experiments utilize a synthetic dataset with scenes containing SF6 gas plumes, generated via DIRSIG, with 128×128 resolution. The model trains on 30-50 views, evaluating PSNR, SSIM, and detection AUC. Ablation studies compare multi-channel density prediction, geometric regularization, and adaptive loss components. Baselines include standard NeRF and variants without spectral weighting. Detection performance is assessed by comparing ACE masks from NeRF-rendered images against ground truth, demonstrating the model’s robustness in sparse-view scenarios. Hyperparameters such as regularization weights and sampling ranges are tuned for optimal results.

The proposed model achieves a PSNR of 39.8dB with only 30 images, outperforming baseline NeRF by approximately 50%. Detection AUC reaches 0.821, indicating high accuracy in identifying gas plumes. Adaptive channel weighting reduces errors in high-absorption bands, leading to more accurate spectral reconstructions. Ablation results confirm that multi-channel density prediction and geometric regularization significantly improve scene fidelity and detection performance, especially under limited view conditions.

This approach enables efficient gas plume detection and analysis in environmental monitoring, industrial safety, and defense scenarios. It requires only sparse multi-view LWIR hyperspectral data, making it suitable for UAV or satellite platforms where data acquisition is costly. The detailed 3D models facilitate accurate quantification of gas concentration and temperature, supporting real-time decision-making and early warning systems. The methodology can be extended to other spectral bands and gas types, broadening its industrial and scientific impact.

The model relies on synthetic data for training, and its performance on real-world LWIR data needs validation. Computational costs are high due to multi-channel density prediction and high-frequency encoding, limiting real-time deployment. Data scarcity and sensor noise may affect generalization. Further work is needed to adapt the framework for diverse environments, multi-gas scenarios, and to reduce inference latency for practical applications.