GEAR: Reconstruction of Classical Paintings via Geometry Grounding and Appearance Restitution

The evolution of 3D reconstruction has seen significant progress with neural implicit representations like NeRF and point-based methods such as 3D Gaussian Splatting, primarily applied to natural scenes. Cultural heritage artifacts, especially classical paintings, pose unique challenges due to their stylized, symbolic, and non-physical depiction of scenes. Prior works in digital preservation focus on 2D restoration or style transfer, with limited exploration into 3D modeling. Existing datasets mainly contain annotated images for restoration tasks, lacking explicit 3D structural supervision. Recent efforts in style-aware 3D modeling are limited by the assumptions of physical scene consistency, which do not hold for artistic works. This gap motivates the development of specialized methods capable of handling the stylized nature of artworks, emphasizing the need for geometry-aware and style-preserving approaches.

Reconstructing 3D models from classical paintings is hindered by their stylized depiction, ambiguous perspective, and symbolic layouts. Existing methods rely on physical scene priors that are violated in artworks, leading to unstable depth estimates and style distortions. The core challenge is to develop a framework that can produce geometrically plausible 3D structures while faithfully preserving the artistic style. Achieving this requires decoupling geometric stability from appearance fidelity, as these objectives often conflict in stylized images. The problem is compounded by the lack of large-scale, diverse datasets tailored for this task, making evaluation and generalization difficult.

The paper introduces a two-stage, training-free pipeline: •Geometry Grounding, which employs deep image translation guided by textual prompts to convert stylized paintings into geometry-oriented representations with consistent shading and illumination, stabilizing depth estimation. •Appearance Restitution, which uses multi-view rendering and diffusion-based style editing to recover painterly textures and details, ensuring multi-view consistency and style fidelity. This decoupling allows each stage to optimize for specific objectives—geometric stability and artistic fidelity—without compromise. The approach leverages pre-trained models and spatial regularization, avoiding the need for additional training, and introduces a novel combination of geometric stabilization and style-aware editing tailored for artworks.

• �� Extract scene contours from input painting via edge detection, serving as structural priors.
• �� Use text-guided image translation to generate a geometry-oriented image with coherent shading and illumination cues.
• �� Perform illumination grounding by computing log-ratio fields, multi-scale fusion, and adaptive gain control to enhance spatial illumination consistency.
• �� Reconstruct a grounded linear image combining reflectance and refined illumination, preserving high-frequency details.
• �� Feed grounded image into pre-trained monocular 3D models to generate initial stable Gaussian scene.
• �� Render multiple views from the grounded scene, then apply diffusion models guided by style prompts to restore painterly appearance.
• �� Use multi-view style editing with Gaussian refinement to optimize the final 3D representation, ensuring multi-view consistency and style fidelity.

The evaluation uses the HeriArch dataset, measuring structural consistency, style matching, and user preference surveys. Comparisons with baseline methods like NeRF and 3DGS show significant improvements in depth accuracy (30% error reduction), style fidelity (over 20%), and multi-view consistency. Ablation studies isolate the contributions of geometry grounding and style restoration stages, confirming their importance. Hyperparameters such as illumination regularization strength and style prompt influence are tuned via grid search. Results demonstrate robustness across diverse artistic styles, with qualitative visualizations and quantitative metrics validating the effectiveness of the approach.

GeAR achieves a 15% increase in structural consistency scores, reduces depth errors by 30%, and improves style fidelity by over 20% compared to baseline models. User preference surveys favor GeAR outputs by 80%, indicating high acceptance among experts. Ablation results confirm that geometry grounding stabilizes depth estimation, while style restoration enhances visual fidelity. The method maintains multi-view consistency, accurately reconstructing complex scenes with symbolic layouts and intricate textures. These results demonstrate the framework’s ability to handle diverse artworks, outperforming existing approaches in both quantitative and qualitative assessments.

This technology enables digital museums, virtual reality exhibitions, and cultural heritage preservation by converting 2D artworks into explorable 3D models. It requires only a single painting image, making it accessible for institutions with limited data. Future applications include automated batch processing of large art collections, integration with AR/VR platforms for immersive experiences, and aiding art historians in spatial analysis and restoration. The approach can also be extended to other non-photorealistic domains such as comics or animation, broadening its impact.

Current model performance diminishes with highly stylized or abstract artworks lacking clear structural cues. Computational costs remain high, limiting real-time applications. The framework assumes availability of accurate camera poses for multi-view rendering, which may not always be feasible. Future work should focus on improving generalization to diverse styles, reducing computational overhead, and integrating automatic pose estimation to enhance usability.