Skip to content
Why your CRO programme stalls (and how to fix it).
Most teams assume their CRO programme dies from a lack of ideas. You run out of hypotheses, the backlog goes quiet, testing slows down. Fix the ideation problem and the programme comes back to life. Actually, that’s rarely what kills a programme. In over a decade running thousands of experiments across industries, we’ve observed few programmes stall due to lack of ideas. Particularly with the exponential improvement in AI, it’s become more efficient than ever to create a backlog of potential ideas.
Here’s what’s actually going on underneath, and why the fix usually isn’t “generate more hypotheses.” Programmes don’t die in one dramatic moment. They erode, usually across four predictable points:
Win rate drops and nobody explained why it would.
Optimizely’s 2026 analysis of 127,000 experiments found that only 12% of tests win on the primary metric, with average win rates sitting around 20% across all experiments and closer to 10% for anything tied directly to revenue. That’s the benchmark. But if your stakeholders were sold on “every test moves the number,” a run of three or four inconclusive tests reads as failure, not as a mature testing programme doing exactly what testing programmes do.
Dev capacity gets treated as elastic, and then it isn't.
Experimentation velocity often depends on the same development queue as every other roadmap item for in-house teams. The moment a platform migration, peak trading period, or site re-design lands, experimentation is the first thing deprioritised, because it’s the activity without a hard deadline attached to it. A high-converting checkout process revolves around three key elements – simplicity, speed and trustworthiness.
Lack of consistent prioritisation methodology.
More ideas is great but velocity and the associated impact can stall without a clear decision engine to inform what should be tested. The most common approaches for this are based around 3 core principles: Impact, Confidence, Ease (ICE). This is why we built our own weighted scoring model rather than adopting PIE, ICE or PXL wholesale. Every test idea gets scored from 1 (low priority) to 10 (high priority) across seven factors, split into three weighted groups.
→ Impact and evidence, 50% of the score
Change Impact carries 30% on its own, our assessment of how much the targeted element actually influences conversion rate. Evidence carries 10%, the combined strength of quantitative and qualitative signals supporting the idea. CRO Rating carries the remaining 10%, our own impact assessment layered on top. Half the score is earned before ease ever enters the conversation, a deliberately heavier weighting toward impact than PXL‘s ten-question model typically produces.
→ Reach, 40% of the score.
Traffic to the test page accounts for 20%, and Contribution, how much that audience contributes to sitewide conversion, accounts for another 20%. A high-impact idea on a page nobody visits still won’t reach statistical significance in a useful timeframe, so reach gets weighted almost as heavily as impact itself. Every second counts. A slow or overly complex checkout can make customers reconsider their purchase or abandon the process entirely.
→ Ease, 10% of the score.
Technical Ease and Internal Ease each carry 5%. This is the direct opposite of PIE and ICE, where Convenience or Ease can make up a third of the total score, effectively pushing hard, high-value tests to the bottom of the backlog simply because they’re annoying to build. Ease is capped at a tenth of the total so an idea can’t out-rank a genuinely high-impact test purely because it’s quick to implement.
Wins stop compounding because nobody's tracking
A test wins, gets implemented, and the insight behind it evaporates. Six months later a different team member proposes testing the exact same hypothesis. As AB Tasty’s research on knowledge turnover puts it, every time a team repeats a test because it can’t find the previous result, it’s paying twice for the same insight, and every time it loses the context behind a win, it loses the ability to iterate on it. The culture behind a mature experimentation programme works precisely because every test feeds a shared, permanent knowledge base rather than a single person’s memory.
Customers need assurance that their payment information is secure and that there won’t be any hidden surprises, such as unexpected costs.
Why "just generate more ideas" doesn't fix this
If the real causes are stakeholder expectations, dev capacity, institutional knowledge and org resilience, then a bigger backlog of test ideas solves none of it. You can hand a stalling team fifty new hypotheses and watch velocity stay exactly where it was, because the bottleneck was never the ideation stage.
The quick fix is often to invest in workshops to generate more test ideas, when the actual gap is a prioritisation framework, a documented win-rate baseline, and a way to keep momentum visible to the people who fund it.
Contentsquare’s 2026 Digital Experience Benchmarks report found that frustrating experiences lower conversion rates by 6.1%, with almost half of all online visits still suffering from preventable friction. That’s the mechanism in practice: small usability issues don’t cause programmes to visibly fail, they just make each subsequent test slightly less likely to move the needle, which quietly erodes stakeholder confidence over a longer horizon than most quarterly reporting captures. The same pattern shows up in experimentation programmes themselves. The erosion is slow enough that nobody notices until the programme is already dead.
→ What actually protects momentum
The programmes that survive team changes, platform migrations and slow patches share a few structural habits, not a bigger idea pipeline.
→ A documented, shared rationale for every test.
ot just the hypothesis, but why it was prioritised, what evidence backed it, and what was learned regardless of outcome. This is the single biggest predictor of whether a programme survives a personnel change.
→ A win-rate baseline set at the start, communicated honestly.
If your stakeholders know upfront that 70-80% of tests won’t typically move the primary metric, then month three’s run of inconclusive results reads as expected variance rather than a crisis.
→ Non-winning tests can still inform wider strategy.
A test that doesn’t move the primary metric isn’t dead weight. It informs prioritisation of the development backlog with evidence instead of opinion. Every test result, win or not, is a data point that feeds digital decision making.
→ A living heuristic library.
REO‘s own approach, and the reason we built a structured evidence source rather than relying on institutional memory, is that every test result, win, loss or inconclusive, gets folded back into a searchable library of heuristics rather than a person’s recollection.
→ Whoever owns the process, owns the outcome.
Convert’s 2025 Agency Report found that 65% of agencies now see clients bringing more experimentation in-house. The agencies still standing are moving upstream: embedding with teams, designing the experimentation programme itself, and proving business-level outcomes rather than just delivering test wins. In-house or agency doesn’t decide whether a programme survives. Whoever holds the mandate does, provided they’re building the structural habits above.
The uncomfortable part
None of this is exciting work. Nobody gets promoted for building a good documentation habit. But the payoff is real and measurable: Speero’s research across 150+ experimentation teams found that businesses with mature testing programmes are 69% more likely to grow significantly. Teams with the longest-running, highest-compounding testing programmes all did the boring structural work early, before they needed it.
If your CRO programme has lost momentum in the last year, was it the ideas that ran out, or was it something structural that finally caught up with you?
See where your programme actually stands: Maturity Model Assessment
FAQ
Why does an experimentation programme lose momentum even when the backlog is full of ideas?
Because ideas were rarely the bottleneck in the first place. Momentum usually erodes through four structural issues: a win rate stakeholders were never calibrated on, dev capacity that quietly gets treated as flexible, no consistent way to prioritise the backlog, and test learnings that disappear the moment a test ends. A bigger backlog doesn’t touch any of these.
What is a good win rate for an A/B testing programme?
Optimizely’s analysis of 127,000 experiments found only 12% of tests win on the primary metric, with average win rates around 20% overall and closer to 10% for tests tied directly to revenue. REO’s own programme runs at a 48% win rate, well above that industry benchmark, but even a programme finding one or two winners in every ten tests at the 20% benchmark is performing normally, not underperforming.
Why do stakeholders lose confidence in a testing programme that's actually performing normally?
Usually because nobody set an honest win-rate baseline at the start. If a programme was pitched on “every test moves the number,” a completely typical run of three or four inconclusive tests reads as failure instead of expected variance, and that expectation gap is what actually gets budgets cut.
What's the difference between ICE, PIE, PXL and REO's own prioritisation model?
ICE, PIE and PXL all let ease or convenience carry up to a third of a test’s priority score, which can push hard, high-value tests to the bottom of the backlog simply because they’re annoying to build. REO’s model caps ease at 10% and weights impact and evidence at 50%, so prioritisation follows business impact rather than implementation convenience.
Why do testing programmes stop compounding results even after finding wins?
Because the reasoning behind a win often isn’t recorded anywhere. A test wins, ships, and six months later a different team member proposes testing the same hypothesis because there’s no record it already ran. Programmes that keep compounding results fold every test result, win, loss or inconclusive, into a shared, searchable heuristic library instead of relying on institutional memory.
How do you stop a testing programme from losing momentum during a busy quarter?
Protect a minimum cadence rather than letting testing be the first thing that flexes when a migration, re-design or peak trading period lands. Programmes that survive team changes and slow patches share a few structural habits: a documented rationale for every test, an honestly communicated win-rate baseline, and clear ownership of the outcome, regardless of whether the programme sits in-house or with an agency.
