CORDS: Continuous Representations of Discrete Structures
CORDS maps discrete objects to continuous fields, enabling exact decoding of variable-sized sets with high accuracy.
Key Findings
Methodology
CORDS employs kernel functions to superimpose object positions and attributes into continuous density and feature fields, forming a bijective mapping. This involves defining positive kernels (e.g., Gaussian), constructing density as a superposition of kernels centered at object locations, and aligning feature fields accordingly. The invertibility relies on total mass encoding object count, kernel matching for position recovery via optimization, and linear projection for attribute reconstruction. Sampling strategies include importance sampling for irregular data and uniform sampling for grid-based data. Neural architectures adapt to data type, using transformers for point clouds and CNNs for regular grids.
Key Results
- On molecular datasets QM9 and GeomDrugs, CORDS achieves competitive or superior performance in molecule generation, with validity scores exceeding 97%, and accurately predicts properties with MAE comparable to state-of-the-art GNNs.
- In object detection tasks like MultiMNIST, CORDS maintains high AP scores (76.8%) across varying object counts, outperforming fixed-capacity detectors, especially in out-of-distribution scenarios with unseen object numbers.
- For astrophysical inference, CORDS effectively reconstructs light curve components, matching the true number of bursts with posterior distributions tightly centered around the actual count, demonstrating robust generalization.
Significance
This work introduces a unified framework for modeling variable-sized sets via continuous fields, addressing longstanding challenges in set cardinality inference. It bridges the gap between discrete and continuous representations, enabling end-to-end learning without predefined capacity constraints. The approach enhances robustness across domains, from molecular design to scientific inference, and opens new avenues for flexible, scalable modeling of complex structures.
Technical Contribution
The core technical innovation is the development of an invertible, kernel-based mapping that encodes object count, position, and attributes into continuous fields. This allows models to operate entirely in the field space, leveraging differentiable optimization for set recovery. The method supports exact decoding, theoretical guarantees, and flexible sampling, representing a significant advance over prior fixed-slot or heuristic-based approaches.
Novelty
CORDS is the first to establish a bijective, invertible mapping between discrete sets and continuous fields, enabling direct modeling of variable cardinality without auxiliary mechanisms. Unlike previous methods relying on padding or heuristics, it provides a unified, end-to-end differentiable framework applicable across modalities, marking a fundamental shift in set representation.
Limitations
- Kernel parameter selection critically influences performance; improper tuning can degrade accuracy, especially in high-dimensional or sparse scenarios.
- Sampling efficiency diminishes with increasing data complexity, requiring advanced strategies for large-scale or dense fields.
- Decoding stability in extremely sparse or noisy environments remains an open challenge, necessitating further robustness improvements.
Future Work
Future research will focus on adaptive kernel selection, multi-scale representations, and integrating learned kernels for enhanced flexibility. Extending the framework to dynamic or temporal data, improving computational efficiency, and applying to broader scientific problems such as particle physics or cosmology are promising directions.
AI Executive Summary
Predicting variable-sized sets of objects is a fundamental challenge across scientific and engineering disciplines. Traditional methods often rely on fixed-capacity models or heuristic post-processing, which limit robustness and scalability. This paper introduces CORDS (Continuous Representations of Discrete Structures), a novel framework that maps discrete objects into continuous, invertible fields. By superimposing kernel functions centered at object locations, CORDS encodes both the count and attributes of objects into density and feature fields. The key innovation lies in the bijective nature of this mapping, enabling models to operate entirely in the continuous domain while guaranteeing exact recovery of the original set.
The approach leverages properties of positive kernels, total mass encoding, and linear projections to ensure invertibility. During training, the model learns to denoise and generate these fields directly, using sampling strategies tailored to data type—importance sampling for irregular data like molecules, and uniform sampling for grid-based data such as images. Neural architectures are chosen accordingly, including transformers for point clouds and CNNs for regular grids.
Experimental results demonstrate the versatility and robustness of CORDS. In molecular generation tasks (QM9, GeomDrugs), it achieves validity scores above 97%, matching or surpassing state-of-the-art GNNs. In object detection (MultiMNIST), it maintains high AP scores (76.8%) even when object counts exceed training distribution, outperforming fixed-slot detectors. In astrophysics, it accurately reconstructs light curve components, correctly estimating the number of bursts.
This work significantly advances the modeling of variable-length sets, providing a unified, end-to-end differentiable framework that is applicable across domains. It opens new possibilities for scientific inference, molecular design, and computer vision, where flexible, scalable, and precise set prediction is essential. Future directions include adaptive kernel methods, multi-scale representations, and broader scientific applications, promising a transformative impact on how complex structures are modeled and understood.
Deep Dive
Plain Language Accessible to non-experts
想象你在厨房里准备一桌菜，每道菜代表一个对象。传统方法就像提前准备好固定数量的盘子，菜多了或少了都得重新调整。而CORDS则像用一块神奇的桌布，把每道菜的香味和位置都画在上面。这块桌布可以根据菜的多少自动变形，显示出所有菜的位置和味道。你只要看这块桌布，就能知道有多少菜，在哪里，味道如何。吃完后，这块桌布还能还原成原来的菜盘，让厨房变得更灵活、更高效。它的神奇之处在于，既能显示所有菜的位置，又能还原出原始的菜盘，不用担心菜的数量超出预设。
ELI14 Explained like you're 14
想象你在玩一个超级酷的游戏，你的任务是找到屏幕上所有的宝藏箱。以前的方法就像提前告诉你最多有10个宝藏箱，然后你在屏幕上找，但如果宝藏箱多了，可能就会漏掉一些。现在，CORDS就像在屏幕上画了一层神奇的隐形网，把每个宝藏箱的位置和大小都藏在里面。你只要看这层网，就能知道有多少宝藏箱，在哪个位置，甚至每个宝藏箱的颜色和形状都能知道。最棒的是，这层网还能还原出原来的宝藏箱，帮你找到所有宝藏！这让游戏变得更公平、更有趣，不管宝藏箱多还是少，都能一一找到！
Abstract
Many learning problems require predicting sets of objects when the number of objects is not known beforehand. Examples include object detection, molecular modeling, and scientific inference tasks such as astrophysical source detection. Existing methods often rely on padded representations or must explicitly infer the set size, which often poses challenges. We present a novel strategy for addressing this challenge by casting prediction of variable-sized sets as a continuous inference problem. Our approach, CORDS (Continuous Representations of Discrete Structures), provides an invertible mapping that transforms a set of spatial objects into continuous fields: a density field that encodes object locations and count, and a feature field that carries their attributes over the same support. Because the mapping is invertible, models operate entirely in field space while remaining exactly decodable to discrete sets. We evaluate CORDS across molecular generation and regression, object detection, simulation-based inference, and a mathematical task involving recovery of local maxima, demonstrating robust handling of unknown set sizes with competitive accuracy.