Hot French startup ZML releases free product to speed inference across lots of AI chips

The days of Nvidia’s unparalleled market dominance are not over, but challengers and choices are coming from all directions.

ZMLa popular French AI startup endorsed by Turing Award winner Yann LeCun, has released inference performance software that makes this possible a variety of open-source major language models to run on one variety of chips – including Nvidia’s, AMD’s, Google’s TPU, Apple Metal and Intel Arc.

Of ZML/LLMDthe recently launched LLM inference server, the company’s ambition is to break existing silos and make different chips available for AI use cases at their maximum available speed, and sometimes faster, ZML founder Steeve Morin told TechCrunch.

As AI is integrated into our work and daily lives, optimizing inference — also called cue processing — has surpassed model training in importance, but often feels patchy behind the scenes, with software and architecture barriers leading to vendor lock-in, Morin said.

The promise of delivering top performance on a variety of chips is a technological achievement, but it could also be a market disruptor amid rising fears about AI-related costs.

ZML hopes to offer companies and clouds the ability to use a mix of chips, some of which may be less expensive or use less energy. “The idea is to give people back the power to create their own system and achieve real efficiency gains that make that possible [AI] be spread,” Morin said.

Such software support could help new AI chip makers, many of which happen to be from Europe, Morin noted Axelera, Fragile, Kalray, OLIX, Q.ANT, SiPearl, SpinNcloudAnd VSORA. But more important to him than their region of origin is that ZML can work with them on “things that have never been done before anywhere in the world.”

That doesn’t mean Morin is bearish on Nvidia. He’s notpartly due to the existing supply. He told TechCrunch that ZML has a good relationship with the AI ​​chip giant, which is gearing up for the rise of inference.

Inference has been an area of ​​so intense investment that the trend has become known as the “inference gold rush.” So ZML has competition like Baseten, recently valued at $13 billion; Inferact, from the creators of open source projects vLLM; as well as RadixArkthe commercial company behind SGLang.

Both vLLM and SGLang partially compete with LLMD, but Morin’s ambitions for ZML cover a broader spectrum. “We have reached the point where we are designing silicon together,” he said. He further credited ZML’s lean team of 20 people as the reason why the Paris-based startup has been able to move forward quickly, with more releases in the plans.

It also helped that this small team was well funded for its size. Thanks to his track record as VP of engineering of Zenly, which Snapchat acquired for nine figures in 2017, Morin raised $20 million from venture capital firms including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe and Puzzle Ventures.

Unlike ZML’s first public project, it focused on inference ML framework released in 2024 and updated in MarchZML/LLMD is not open source. But it is being launched as a free product with the aim of learning more about its usage. “I prefer to measure and [then generate revenue] where it is most effective, without stupidly stunting my growth because I have been too greedy from the start,” Morin said.

It is still too early to say when ZML/LLMD may become a paid product and what its adoption will look like. But the startup’s cap table confirms that other founders are paying attention, including Dagger and Docker founder Solomon Hykes, Hugging Face’s Clément Delangue and Julien Chaumond, as well as LeCun, now at AMI Labs. This also strengthens the argument that Europe’s AI startups can now build from home. “I couldn’t do ZML anywhere else but Paris,” Morin said.