By Ammar Sabilarrohman 
Three years after his seminal appearance on the Invisible Machines podcast, Jonathan Frankle—formerly the chief scientist at MosaicML and currently the Chief AI Scientist at Databricks—has returned with a diagnosis that is as sobering as it is transformative. In 2021, Frankle introduced the world to the "data mixology" concept, comparing the art of training language models to the precise, iterative process of baking. He cautioned against the industry’s reliance on academic benchmarks like HellaSwag, which, he argued, were poor proxies for what real-world users actually demanded from generative AI.
Today, Frankle’s perspective has evolved from the kitchen to the power plant. His central thesis is stark: The tech industry has successfully achieved "nuclear fusion"—creating raw, massive, and unpredictable intelligence—but has entirely neglected to build the "power lines" required to distribute that energy safely and effectively.
The State of the Industry: A Chronology of Disconnect
To understand Frankle’s current frustration, one must look at the trajectory of the last thirty-six months. In 2021, the focus was on the "mini-cupcake ladder"—the idea that you must prove model performance on smaller, manageable tasks before attempting to scale to massive, generalized systems. It was a time of academic excitement and "lottery ticket" hypotheses.
By 2023, the focus shifted to the "Gold Rush." Companies began stuffing massive context windows with document dumps, operating under the fantasy that training was a thing of the past. If you could just cram enough information into a prompt, the model would become an omniscient oracle.
Now, in late 2024, we are in the era of "Fusion Without Infrastructure." The models are undeniably more powerful, but the gap between "raw capability" and "human intent" has widened. Frankle suggests that while we have reached a level of raw, brute-force reasoning power that was unthinkable three years ago, we lack the engineering disciplines—specification, testing, and modularity—to make these models reliable enterprise assets.
The Engineering Deficit: Why We Need a "Java" Moment
Frankle’s most provocative assertion is that the AI industry is currently in its "Fortran days." In the early decades of computing, programming was a labyrinthine, low-level process that lacked the abstractions necessary for mass adoption. It wasn’t until the advent of structured, human-comprehensible languages like Java, Pascal, and C that software engineering matured into a predictable discipline.
"We really need to invent Java," Frankle argues. Current AI development lacks the rigor of software engineering. There is no mature equivalent for unit tests, integration tests, or regression suites that can reliably verify that an LLM will behave as intended across millions of edge cases.
The Myth of Automated Specification
The industry has attempted to bridge this gap with "automated specification tools," but Frankle remains skeptical. He notes that building an evaluation set is not the same as defining business requirements. Many developers fall into the trap of using model outputs as their own benchmarks, a recursive feedback loop that offers the illusion of progress while masking fundamental flaws in the logic. For Frankle, the hard work lies in the pre-model phase: translating human intention into a testable specification before a single token is generated.
Prompts Are Parameters: A New Framework for Optimization
One of the most critical shifts in perspective offered by this conversation is the realization that "prompt engineering" is simply a form of training. If a developer is working within the confines of a proprietary model like GPT-4 or Claude, they are not merely "talking" to the AI; they are optimizing natural-language parameters.
This creates a paradox: developers are spending vast amounts of time crafting prompts to steer the model, yet they lack the visual back-end or the debugging tools to verify what the model has actually "learned" or prioritized. This is what researchers call the "Marshall McLuhan loop"—using old media (human language and visual UI) to wrap new media (probabilistic, trillion-parameter black boxes).
The implication is clear: until we treat prompts as code—version-controlled, tested, and modular—we will remain in a state of constant, fragile experimentation.
The Data Hygiene Crisis: "Garbage-In, Garbage-Out"
As organizations rush to integrate their proprietary data into Retrieval-Augmented Generation (RAG) pipelines, they are encountering a familiar lesson: the "garbage-in, garbage-out" principle of the 1970s applies to LLMs with ruthless efficiency.
Frankle emphasizes that when companies dump decades of unstructured internal documents into an agent, they are not just providing knowledge; they are introducing "noise." If those documents contain outdated policies, conflicting facts, or unverified draft content, the model will inevitably manufacture falsehoods at scale.
The Future of "Model-Ready" Content
Frankle predicts the rise of a new industry: LLM optimization, which he views as the next generation of SEO. Organizations will soon need to:
- Curate Canonical Facts: Deliberately separating stable, verified "source of truth" data from tentative draft content.
- Structural Sanitization: Removing "distractor" documents that degrade performance during retrieval.
- Strategic Ingestion: Understanding that every document published on a corporate server is now, effectively, a training signal for the next generation of internal agents.
The goal, Frankle suggests, is to separate "knowledge" from "reasoning." We want a faithful reasoner that plugs into a clean, curated set of brand guidelines and factual databases. Currently, however, organizations are forced to live within "trillion-parameter uncertainty," where the model’s reasoning capabilities are inextricably linked to the messy, uncurated data of the past.
Implications for the Enterprise
For the enterprise leader, Frankle’s advice is both pragmatic and demanding. The "mixologist" mindset—a term he coined during his time at MosaicML—is now being applied to twelve thousand enterprises at Databricks.
1. Measure What Success Looks Like
Stop relying on generic leaderboard scores. A model that ranks high on an academic benchmark may be useless for your specific supply chain forecasting or customer support workflow. Define success through domain-specific metrics that correlate with actual business outcomes.
2. Start Small
The temptation to go for a "Big Bang" AI deployment is high, but the risk of failure is higher. Frankle advocates for the "ladder" approach: build, test, and verify small components before attempting to scale the system.
3. Prioritize Impact Over Hype
Frankle maintains a value system that prioritizes real-world impact—such as his early research on the ethics of police facial recognition—over the "AGI tomorrow" hype cycle. He urges practitioners to focus on the "blender in the kitchen"—the immediate, tangible tools that can be built today—rather than chasing the theoretical horizon of artificial general intelligence.
Conclusion: The Road to Mature Infrastructure
The current state of AI is an extraordinary achievement in raw power, but the transition from a research curiosity to a stable enterprise utility requires a fundamental shift in mindset. We have built the engine, but we are still operating without a dashboard, brakes, or a steering wheel.
As Jonathan Frankle concludes, the path forward is not found in bigger models, but in better engineering. It requires a commitment to the boring, difficult, and essential work of software discipline: defining requirements, writing tests, and curating data. Until we build the "power lines" of infrastructure, we will continue to struggle with the instability of the current generation of models.
For those who want to see AI move from the laboratory to the backbone of the global economy, the lesson is clear: Stop looking for the next "lottery ticket" and start building the grid.
Neng Nana 
Sagoh 