πŸ“° Key Highlights

French AI startup ZML has officially launched its inference server software LLMD, aiming to let major open-source LLMs run at near-native or beyond-native peak speeds across chips from multiple vendors. Supported hardware covers Nvidia GPUs, AMD, Google TPU, Apple Metal, and Intel Arc. Founder Steeve Morin said that AI inference β€” the process of a model handling user prompts β€” is gradually surpassing model training itself in importance, but the split between the software layer and the architecture layer has enterprises deeply trapped in vendor lock-in. ZML’s core proposition is to let enterprises and cloud providers mix and match chips from different sources, thereby lowering costs or reducing energy consumption, and ultimately making large-scale AI adoption possible.

Morin previously served as VP of Engineering at social app Zenly (acquired by Snapchat in 2017 for a nine-figure USD sum), and leveraged that background to raise $20 million from well-known VCs including 20VC and Kima Ventures. With a lean team of just 20 people, ZML is moving fast on development in Paris. The company has already entered the stage of co-designing silicon with chipmakers, and has named several European emerging chipmakers such as Axelera and SiPearl as potential partners.

The inference infrastructure space is fiercely competitive, with rivals including Baseten (valued at $13 billion), Inferact (founded by the creators of the vLLM open-source project), and RadixArk (the commercial company behind SGLang). Morin emphasizes that ZML is not bearish on Nvidia β€” the two maintain a solid partnership β€” but LLMD’s multi-chip compatibility strategy is giving the market more room to negotiate.

πŸ’¬ JudyAI Lab Perspective

ZML’s LLMD takes aim at the most core contradiction in AI infrastructure: when inference has already surpassed model training in importance, the software-architecture divide keeps enterprises locked into specific vendors, and the cost of large-scale deployment remains stubbornly high.

From $13-billion-valued Baseten to Inferact (founded by the vLLM creators), the competition in the inference infrastructure space is already brutal. ZML’s angle isn’t to go head-on against Nvidia, but to let enterprises mix and match chips from different vendors β€” Nvidia, AMD, Google TPU, Apple Metal, Intel Arc, all fully compatible. What we see behind this direction is the industry’s collective anxiety around “lock-in risk”: when inference costs directly affect the commercial viability of AI services, whoever gives enterprises the most flexibility in chip selection holds the real bargaining chips.

Take a moment to assess the inference architecture you’re currently using: if you had to switch chip vendors tomorrow, how high would the switching cost be? The answer to that question determines how much room you have to maneuver when facing future price hikes or supply shortages.

πŸ“… Source Info

πŸ”— Further Reading