By Layla Zulfa 
Three years ago, when Jonathan Frankle—then Chief Scientist at MosaicML—appeared on the Invisible Machines podcast, he introduced the world to the "data mixology" framework. He likened the training of Large Language Models (LLMs) to the meticulous art of baking: a process of testing mini-cupcakes before committing to the full cake. He warned then that academic benchmarks like HellaSwag were poor proxies for the messy, real-world utility that users actually demanded from systems like ChatGPT.
Today, Frankle returns to the podcast as Chief AI Scientist at Databricks. His message has evolved from the granular mechanics of model training to a broader, more urgent systemic critique. While the industry has successfully achieved what he calls "nuclear fusion"—the creation of raw, powerful, and transformative machine intelligence—we have utterly failed to build the "power lines" necessary to deliver that energy to the end user.
The State of the Art: A Chronology of Disconnect
To understand the current bottleneck, one must look at the arc of AI development over the last thirty-six months.
2021–2022: The Era of Scaling Laws. The industry was obsessed with the "more is better" mantra. Researchers focused on parameter counts, GPU clusters, and massive ingestion of public internet data. The goal was to prove that scale alone could unlock emergent behaviors.
2023: The Pragmatic Turn. As LLMs hit the mainstream, the focus shifted to "mixology." Practitioners began to realize that the composition of the training data mattered more than the sheer volume. Frankle’s early work at MosaicML defined this era, emphasizing that quality control in data ingestion was the primary differentiator between a toy model and a production-grade tool.
2024–Present: The Infrastructure Crisis. We have arrived at a point where the models themselves are undeniably capable. We have reached a level of fusion-grade intelligence that was unimaginable a decade ago. However, the software engineering discipline surrounding these models remains in its infancy. We are attempting to run an industrial-scale electrical grid using wiring that belongs in a Victorian house.
Fusion Without Infrastructure: The Engineering Gap
Frankle’s central metaphor in his recent interview is stark: we have created the equivalent of a nuclear reactor, but we are trying to power our cities by dangling wires directly into the core.
The Missing "Java" of the AI Age
"We’re in the Fortran days," Frankle notes. "We really need to invent Java."
In the history of computing, the transition from raw machine code or early assembly to structured, high-level languages like Java, Pascal, or C was what allowed software to move from a research curiosity to the backbone of global commerce. These languages provided developers with a way to describe intent, edit logic predictably, and verify outcomes.
In the current AI landscape, we lack these primitives. When a model produces an output, we are often left guessing why it arrived there. We lack the "specification" layer—a way to formally define what a model should do, test that it meets those requirements, and integrate it into a predictable, composable pipeline.
The Myth of the Benchmark
Frankle is particularly scathing regarding the industry’s reliance on static benchmarks. He argues that building a dedicated evaluation set is not the same as defining business requirements. Many developers treat benchmarks as a "cop-out"—a way to justify model performance without ever having to engage with the difficult work of translating human intention into a machine-testable format.
Supporting Data: The Context Window Illusion
One of the most persistent fantasies in the AI community is the idea that "long context" will eventually render training or fine-tuning obsolete. The logic follows that if we can just stuff a million tokens—an entire library of internal documents—into the model’s window, it will magically understand the nuance of a business’s needs.
Frankle challenges this with the harsh reality of "garbage-in, garbage-out." He notes that while long-context windows are technically impressive, performance often degrades as you increase the data load. Distractors, irrelevant documents, and the model’s inherent lack of omniscience mean that more information does not always lead to better reasoning.
Furthermore, the rise of multimodal inputs—where video and high-resolution imagery add massive token counts—is accelerating this problem. The science of "mixology" holds true: whether you are pre-training, utilizing Retrieval-Augmented Generation (RAG), or performing massive prompt stuffing, if the input quality is poor, the output will remain fundamentally unreliable.
Prompts as Parameters: The New Software Paradigm
One of the most profound insights from the discussion is the re-framing of the "prompt." If you are working within a platform like Claude or GPT-4, you are effectively engaged in training—you are simply optimizing natural-language parameters rather than binary weights.
This brings us to the "Marshall McLuhan loop": we are building sophisticated, high-tech graphical user interfaces whose sole purpose is to help a human write a prompt to generate a result. It is old media wrapped inside new media. The underlying issue is that we lack the fundamental disciplines of software engineering:
- Unit Tests: Can we isolate a specific capability of the AI and verify its performance in isolation?
- Integration Tests: When we chain models together, can we verify that the output of one serves as a valid input for the next?
- Regression Testing: If we update the model version, how do we ensure our existing business processes don’t break?
These disciplines do not yet exist in any mature form for AI. The "blender" of modern AI is sitting in the kitchen, powerful and spinning, but it is effectively unplugged from the systems that would make it useful to a Fortune 500 company.
Implications for Enterprise Content and Brand Strategy
The conversation concludes with a warning for every brand and enterprise content team: the internet is no longer just a library for humans; it is a training signal for the next generation of machines.
Frankle anticipates an emerging industry—a "LLM SEO"—where companies must strategically curate their digital footprint. If an enterprise dumps decades of legacy PDFs, outdated internal documents, and contradictory policies into a model, they are essentially training their agent to manufacture falsehoods at scale.
The strategic imperative for the next five years is the separation of "Knowledge" from "Reasoning." Ideally, a company should provide a curated, stable source of truth (knowledge) and feed it into a faithful, high-reasoning model. Frankle admits we are currently far from this clean split. Until we achieve it, organizations will continue to live inside a state of "trillion-parameter uncertainty."
Official Responses and Future Outlook
The transition from the "MosaicML playbook" to the "Databricks customer experience" has provided Frankle with a unique vantage point. He is no longer just theorizing in a lab; he is seeing the same patterns emerge across twelve thousand enterprise deployments.
The method remains consistent:
- Measure what success looks like: Move beyond leaderboard scores.
- Start small: Avoid the "boil the ocean" approach to data ingestion.
- Prioritize impact over citation counts: Real-world utility is the only metric that matters.
Frankle remains a staunch advocate for a scientific temperament. He famously measures his own career success not by his highly-cited papers on the "lottery ticket hypothesis," but by early, practical work on police facial recognition that influenced actual law.
This value system defines his current stance. He is pushing the industry to move away from the "AGI tomorrow" hype cycle and toward the difficult, unglamorous work of building the infrastructure. We have the fusion; now we must build the power lines. Until we do, the most powerful intelligence in human history will remain a brilliant, but largely unharnessed, experiment.
Ali Ikhwan 
Dwi Wanna 
Nana Muazin 