ICONIQ surveyed roughly 300 executives at software companies building AI products, and on 13 August 2026 the growth investor published the rule it drew from them: “use the lowest-cost model that reliably clears the quality bar for a task.” The same piece says who should hold the bar. Finance sets the spending envelope and the return hurdle, and “workflow owners own quality, service levels, and cost per outcome.”
The rule assumes that someone has set the bar. In an AI-native scale-up between Series A and Series C, the person best placed to set it is the first senior product hire, and that is the job the brief should describe.
Every cost decision in an AI product hangs off a quality threshold
On 17 November Model ML, the London company that builds personal AI agents for bankers, consultants and investors, announced a $250 million Series B at a $1.5 billion valuation. Its release says it routes each task to the best model for quality, speed and cost, and that a growing share runs on its own fine-tuned open-source models. Those models, it says, run at 2% of frontier model cost and are in most cases five times faster, with comparable or better accuracy on the tasks they handle.
An operator reads the last clause differently. A 98% saving is available only on tasks where someone has defined what an acceptable answer looks like and tested the cheaper model against it. Without that definition the only safe choice is the most expensive model on every task, and the gross margin case for AI products never gets started.
ICONIQ records that multi-step agentic workflows can burn 5 to 30 times as many tokens as a simple chat exchange, and that 68% of the builders it surveyed find the true cost of their own internal AI very or moderately difficult to predict. Its routing instruction makes the same point: send each task “to the lowest-cost model that clears the quality and risk bar.”
The buyer cannot yet see the value, so quality evidence is what sells
The customers of a European AI scale-up are often large companies, and they are struggling to measure what they have bought. Capgemini Research Institute surveyed 500 executives at organisations with more than USD 1 billion of revenue, across 10 countries and 13 industries, in September 2026. In the results published on 12 November, 59% said their AI-related spending was significantly higher than expected. Only a third could confidently estimate indirect costs, including human oversight, rework and governance. Proving enough value to win approval and funding was the biggest obstacle for 57%.
The same survey expects “first-pass success rate” to become the priority metric, and nearly three in five organisations plan to use more open-weight models. Both point the same way. We expect the buyer to move from “which model do you use” to “how often is the answer right first time, and what does a wrong one cost me to fix.” Rework and oversight are the costs the buyer cannot estimate. In our view much of that traces back to a quality bar set too low or never written down.
Capgemini asked the buyers, not the scale-ups. Its numbers describe the conversation a scale-up’s product leader will have with those buyers in 2027.
European AI companies are building evaluation into the product
Parloa, the Berlin company that builds AI agents for customer service, posted a Staff or Principal Product Manager role for its Agent Management Platform on 7 November. The platform the role shapes covers “agent lifecycle management, simulations and evaluations, and AI model and data orchestration,” and the hire is asked to balance “speed, reliability, safety, and exceptional user experience.” Parloa describes evaluation as part of the product it sells.
Ten days later Remanence, a Paris company founded by three former executives of H, announced a $30 million seed round co-led by EQT Ventures and FirstMark, per Tech.eu. It trains specialised models in simulated enterprise environments, where AI agents practise tasks and learn from their mistakes. It turns open-weight models into specialised models for specific tasks. That offer only holds if a customer can see the specialised model meet the standard on its own work.
Pierre-Louis Cedoz, its co-founder and chief executive, gave the aim in the announcement: “to give every company the ability to build proprietary intelligence that improves with use.” Improvement can only be claimed against a measure, and someone has to own that measure.
The first product leader should be hired to set and hold the bar
The first VP Product or Head of Product at an AI-native scale-up should own four things in writing. What a passing answer is for each task customers pay for. The test set and the pass rate that a release must meet. The cost of each passing result, agreed with finance. And the evidence a customer’s buyer is shown when they ask whether it works.
This is not a case for product running evaluation. Engineering and machine learning decide how a model clears the bar. Product decides where the bar sits, because that is a judgement about what the customer is paying for, and it is the workflow-owner role ICONIQ describes.
Roadmap management, discovery and prioritisation still matter. They are the smaller part of the job here, because almost every roadmap decision in an AI product is a decision about where the bar sits and what it costs to clear. A product leader hired mainly to run a roadmap inherits a quality bar set informally by whichever engineer last changed the model, and spends year one discovering it.
The brief should say so. It should name the bar as the product leader’s, list the tasks it covers on day one, and say who settles a dispute between the bar and a release date.
What the recommendation costs
The person is scarcer. Evaluation as a product discipline is young, and many strong product leaders built their records in software where quality was a bug count. A search written this way will see a smaller field, and some of the best roadmap operators in it will not fit.
It slows releases. A product leader who owns the bar will hold back a feature that demos well and fails one task in ten. That is the point, and it will still cost a quarter of momentum at least once, often in front of the board.
It moves a decision away from engineering. Where the CTO or head of machine learning holds the bar today, informally, handing it to product creates a real negotiation, and it fails if the founder does not back it in public.
And the seat will split. By Series C a company with many tasks and many customers will need an evaluation function of its own, and the product leader becomes its customer rather than its owner. The brief should say that this is expected, so the first hire does not read it as a demotion.
Twenty tasks and one table will show whether the seat is covered
A chief executive can test the argument in an afternoon. List the twenty tasks customers pay for most. For each, write down four things: what a passing answer is, the current pass rate, the model that runs it, and the cost of each passing result. Then count how many of the eighty cells anyone in the company can fill without asking an engineer to run a query.
The number of blank cells is the brief, and only the company can count them.
HMN Capital finds the leaders that decide the return on an investment.
Hiring a product leader in a venture-backed business? See CPO and CTO search for VC-backed businesses.
Hiring a product or technology leader in a PE-backed business? See how we run the search.
Running a search for a leadership role like this? Start a conversation with HMN Capital.
Considering your own next move? Tell us where you are.