LLMoxie: Exploring Agentic AI for Scientific Software Development
LLMoxie platform enhances AI agent performance in scientific software development through a three-tier architecture and RSE-Plugins ecosystem.
Key Findings
Methodology
LLMoxie platform employs a three-tier architecture: inference layer supports multi-cloud and on-premise inference, control plane handles authentication and budgeting, and application augmentation layer provides domain-specific workflows via RSE-Plugins. RSE-Plugins encode scientific Python practices and domain knowledge as reusable context through a Plugin-Agent-Skill hierarchy.
Key Results
- Over 20 months, LLMoxie successfully transformed AI agents from generic code generators to domain-aware collaborators, significantly improving auditability and reproducibility of scientific software.
- RSE-Plugins effectively encoded scientific Python community conventions and professional RSE practices into reusable knowledge modules.
- The platform excelled in projects across astronomy, earth and climate sciences, addressing AI agent adaptation issues in scientific software development.
Significance
LLMoxie addresses poor adaptation of AI agents in scientific software development by transforming them into domain-aware collaborators. Its three-tier architecture and plugin ecosystem provide auditable trails of technical reasoning, enhancing software's citability, auditability, and extensibility.
Technical Contribution
LLMoxie offers a new AI orchestration platform through its three-tier architecture and RSE-Plugins ecosystem, integrating retrieval systems, scientific tools, and research knowledge bases. It innovatively transforms AI agents from generic code generators to domain-aware collaborators, offering new engineering possibilities.
Novelty
LLMoxie is the first platform to deeply integrate AI agents with domain knowledge and community conventions in scientific software development. Its RSE-Plugins ecosystem provides reusable context through a Plugin-Agent-Skill hierarchy, significantly enhancing AI agent adaptation.
Limitations
- The platform's adaptation to specific domains needs further optimization, especially in emerging scientific fields.
- Limited capability in handling sensitive data, requiring enhanced data governance mechanisms.
Future Work
Future work includes expanding the RSE-Plugins ecosystem to cover more scientific domains and further optimizing the platform's capability to handle sensitive data. Exploring more applications of AI agents in scientific software development.
AI Executive Summary
The LLMoxie platform, through its three-tier architecture and RSE-Plugins ecosystem, addresses the poor adaptation of AI agents in scientific software development. Existing AI agents often ignore conventions of scientific Python libraries, mishandle sensitive data, and leave decision trails that are hard to reconstruct. LLMoxie transforms AI agents into domain-aware collaborators by providing multi-cloud inference, a LiteLLM/MLflow control plane, and an application augmentation layer, offering auditable trails of technical reasoning.
Over 20 months, LLMoxie excelled in projects across astronomy, earth, and climate sciences, addressing AI agent adaptation issues in scientific software development. RSE-Plugins effectively encoded scientific Python community conventions and professional RSE practices into reusable knowledge modules through a Plugin-Agent-Skill hierarchy.
LLMoxie provides a new AI orchestration platform, integrating retrieval systems, scientific tools, and research knowledge bases, promoting responsible AI integration in scientific research. Future work will expand the RSE-Plugins ecosystem to cover more scientific domains and further optimize the platform's capability to handle sensitive data.
Deep Analysis
Background
Challenges in scientific software development include ensuring software's citability, auditability, and extensibility while maintaining code quality. Existing AI agents, optimized for commercial software, often ignore scientific Python library conventions, mishandle sensitive data, and leave decision trails that are hard to reconstruct.
Core Problem
Scientific software development requires not only high-quality code but also ensures software's citability, auditability, and extensibility. Existing AI agents poorly adapt to scientific software development, failing to meet these requirements.
Innovation
LLMoxie transforms AI agents into domain-aware collaborators through its three-tier architecture and RSE-Plugins ecosystem. Innovations include encoding scientific Python community conventions and professional RSE practices as reusable context through a Plugin-Agent-Skill hierarchy.
Methodology
- �� Inference layer supports multi-cloud and on-premise inference, adapting to different infrastructure needs.
- �� Control plane provides unified API access via LiteLLM and MLflow, supporting authentication, budgeting, and PII masking.
- �� Application augmentation layer offers domain-specific workflows through RSE-Plugins, enhancing AI agent adaptation.
Experiments
Over 20 months, LLMoxie excelled in projects across astronomy, earth, and climate sciences. RSE-Plugins effectively encoded scientific Python community conventions and professional RSE practices into reusable knowledge modules.
Results
LLMoxie successfully transformed AI agents from generic code generators to domain-aware collaborators, significantly improving auditability and reproducibility of scientific software. RSE-Plugins effectively encoded scientific Python community conventions and professional RSE practices into reusable knowledge modules.
Applications
LLMoxie can be widely applied to AI agent adaptation issues in scientific software development, especially in astronomy, earth, and climate sciences. Its plugin ecosystem provides reusable context, enhancing AI agent adaptation.
Limitations & Outlook
The platform's adaptation to specific domains needs further optimization, especially in emerging scientific fields. Limited capability in handling sensitive data requires enhanced data governance mechanisms. Future work will expand the RSE-Plugins ecosystem to cover more domains.
Plain Language Accessible to non-experts
Imagine you're in a kitchen cooking a meal. LLMoxie is like a smart assistant that not only helps you prepare ingredients but also adjusts recipes based on your taste. It understands the characteristics of each ingredient and knows how to combine them to create a delicious dish. RSE-Plugins are like the various tools and seasonings in your kitchen, helping you complete the cooking tasks more efficiently. This assistant not only helps you make a tasty meal but also records each step of the process for future use.
ELI14 Explained like you're 14
Imagine you're playing a super complex game where you have to manage many characters at once. LLMoxie is like a super assistant that not only helps you manage these characters but also gives you the best strategies for different scenarios. RSE-Plugins are like the various tools and skills in the game, helping you complete tasks more effectively. This assistant not only helps you win the game but also records each strategy step for future use.
Glossary
LLMoxie
An AI platform supporting scientific software development with a three-tier architecture and RSE-Plugins ecosystem.
Used to enhance AI agent adaptation in scientific software development.
RSE-Plugins
A Plugin-Agent-Skill hierarchy encoding scientific Python practices and domain knowledge.
Provides reusable context for AI agents.
Multi-cloud inference
Capability to support inference across multiple cloud platforms.
Inference layer of LLMoxie supports multi-cloud inference.
LiteLLM/MLflow
Tools for the control plane providing unified API access and governance mechanisms.
Used in the control plane of the LLMoxie platform.
Scientific Python community
A community of Python developers focused on scientific computing.
RSE-Plugins encode its conventions to provide context for AI agents.
Open Questions Unanswered questions from this research
- 1 How to further optimize the platform's adaptation to emerging scientific fields?
- 2 How to enhance the platform's capability to handle sensitive data?
- 3 How to expand the RSE-Plugins ecosystem to cover more domains?
Applications
Immediate Applications
Scientific Software Development
Enhance AI agent adaptation in scientific software development, especially in astronomy, earth, and climate sciences.
Long-term Vision
Cross-domain AI Integration
Promote responsible AI integration in more scientific fields, enhancing research efficiency and accuracy.
Abstract
In this paper, we describe LLMoxie, an institutional AI platform whose three-tiered architecture supports multi-cloud and on-premise inference, a LiteLLM/MLflow control plane for authentication, budgeting, PII masking, and observability, and an application augmentation layer for AI coding agents. Layered on top, an open-source RSE-Plugins ecosystem encodes accumulated RSE knowledge as a Plugin-Agent-Skill hierarchy spanning scientific Python practice, domain-specific knowledge, a six-phase research-and-implement workflow, and project lifecycle management. Scientific software is judged less by raw code quality than by whether it can be cited, audited, reproduced, and extended. Off-the-shelf AI coding agents, optimized against commercial software benchmarks, are poorly calibrated for this setting: they ignore the conventions of the scientific Python libraries they invoke, mishandle sensitive or embargoed data, and leave decision trails that are difficult to reconstruct after the fact. We report on twenty months of practice at a university-based research software engineering (RSE) center, where RSEs embedded across astronomy, earth and climate science, agriculture, and health projects worked to close this gap. We characterize the recurring infrastructure, governance, and process challenges of adopting Agentic AI inside a multi-domain RSE center, describe the platform and plugin design, and distill operational lessons from real scientific software deployments. Together, the platform and plugins shift AI coding agents from generic code generators into domain-aware collaborators that respect community norms and produce auditable provenance of technical reasoning.