Modelomics References
This is a curated starter set of references that support the ideas behind Modelomics.
The sources do not define the term Modelomics. They support the surrounding premises:
- AI systems create hidden operating and maintenance costs.
- Model and system quality must be evaluated together.
- Cost-aware routing and escalation are real, research-backed patterns.
- Governance and risk management belong in any serious AI practice.
Core References
1. Hidden Technical Debt in Machine Learning Systems
Why it matters:
This is one of the strongest foundations for Intelligence Debt. It shows that ML systems accumulate long-term maintenance costs through system-level entanglement, hidden feedback loops, data dependencies, and brittle interfaces.
2. The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction
Breck et al., IEEE Big Data 2017
Why it matters: This paper supports the idea that serious ML and AI systems need specific tests and monitoring practices rather than vague confidence. It is useful background for disciplined readiness checks and debt reduction.
3. Fixes That Fail: Self-Defeating Improvements in Machine-Learning Systems
Why it matters: This paper shows that improving one model can make the overall system worse because of downstream entanglement. It supports the Modelomics view that allocation decisions should be evaluated at the system level, not only at the component level.
4. Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
Why it matters: Modelomics cares about waste, drift, and silent failure. This paper is a strong reference for why AI systems should detect shift and surface failures clearly instead of failing quietly.
5. The ML Test Score and Related Google Production ML Work
TFX: A TensorFlow-Based Production-Scale Machine Learning Platform, KDD 2017
Why it matters: This is a practical industry reference showing that ML systems need orchestration, validation, and standardized components to reduce fragility and duplicated effort.
6. Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities
Why it matters: This paper quantifies the complexity of real ML pipelines and identifies wasted computation that does not translate into better outcomes. It is a strong support for the idea that over-allocation creates measurable cost.
7. Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Why it matters: This is a major government framework for managing AI risks. It supports the idea that AI systems should be governed as organizational capabilities with risk, measurement, and accountability concerns.
8. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
Why it matters: This is the GenAI-specific companion to AI RMF. It is especially relevant for Modelomics because it treats trustworthiness as something that must be operationalized across the AI lifecycle.
9. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
Why it matters: This is a direct reference for cost-aware routing and cascading across models. It supports the Modelomics idea that cheaper intelligence should be used first when it is sufficient.
10. RouteLLM: Learning to Route LLMs with Preference Data
Why it matters: This work is relevant to progressive escalation and routing decisions, both of which are central to the Modelomics vocabulary.