Most of the use cases we’re asked to build are legitimate. Some are not, the data isn’t there, the process isn’t stable enough, or the manual cost is lower than the build and maintenance cost over two years. We run this filter before we scope anything, because finding that out after you’ve committed budget is a worse outcome for everyone.
Most AI prioritisation frameworks are written by the vendors selling AI. The framework always ends with “deploy AI.” This one doesn’t.
Why Most AI Projects Fail Before They Start
The RAND Corporation tracked AI initiative outcomes in 2025 and found that 80.3% failed to deliver intended business value. One-third were abandoned before reaching production.
These aren’t failed experiments, they’re funded, scoped, and partially built projects that ran out of signal, budget, or internal will. The failure usually traces back to the same root: no hard filter before the first dollar was committed.
The Gap Between Pilot and Production
A pilot is designed to succeed. You pick a controlled dataset, scope out the awkward edge cases, and run it for six weeks under close supervision. It works. Then production hits.
In production, data is inconsistent. Staff use the tool differently than the test group did. Edge cases appear. Maintenance costs accumulate. The 57% of I&O leaders who reported an AI failure told Gartner their initiative failed because they “expected too much, too fast.” That expectation is set, or not set, during scoping.
What “Operational Readiness” Actually Means
Operational readiness isn’t about whether your team can use AI. It’s about whether your data infrastructure, staff capability, and change management capacity can sustain an AI system after launch.
85% of AI projects fail due to poor data quality or missing data, not bad models. You can buy the most capable model available; it cannot fix fragmented CRM records, inconsistent input formats, or data that lives in three spreadsheets nobody owns.
The Five Criteria That Matter for Operations Use Cases
Before scoring, confirm you’re evaluating a real use case, not a solution looking for a problem. A real use case has a defined workflow, a current manual process, and a measurable output.
Business Impact (Not Business Excitement)
Quantify the impact in time or money before building. “It’ll make things faster” is not a criterion. “Our invoicing team spends 11 hours per week on data entry that produces 3–5 input errors per batch” is a criterion.
Ask: what’s the current cost of the problem? What’s the projected cost of the fix? Only proceed if the delta justifies build cost plus maintenance cost over 24 months.
Data Availability and Quality, The Gating Factor
This is the single most common disqualifier. Before scoring anything else, answer three questions: Is the data structured and consistent? Do you own it? Can you get 1,000+ labelled examples, or equivalent historical records, without a data cleaning sprint?
If any answer is no, score this criterion zero. A zero here disqualifies the use case entirely, don’t let higher scores in other criteria compensate for it.
Time-to-Value vs. Time-to-Maintenance
Some use cases produce value in six weeks. Others take six months to train, deploy, and stabilise, then require monthly retraining to prevent performance drift. Factor both.
The Deloitte AI ROI Study (2025) found that only 6% of organisations achieved AI payback in under a year, and those that did concentrated investment in a small number of use cases with clear, pre-agreed success criteria. Scope creep and underestimated maintenance costs were the most common reasons others didn’t.
Integration Complexity and Total Cost of Ownership
Every API call, every data pipeline, every user permission layer adds complexity. Map the integrations required before scoring this criterion. A use case that touches your CRM, your billing system, and your email provider is three times the integration surface of one that reads from a single source.
If you’re running a custom WordPress development environment or a heavily customised WooCommerce backend, factor in the middleware work, AI outputs rarely map cleanly to existing systems without a translation layer.
Staff Capability and Change Tolerance
AI tools fail when staff don’t trust the output, can’t interpret confidence scores, or route around the tool to avoid uncertainty. 75% of SMBs cite insufficient internal AI knowledge as a major adoption barrier, this isn’t a training problem alone. It’s a change management problem.
Score this criterion based on your current state, not post-training assumptions. A team that has never worked with probabilistic outputs will need process redesign, not just onboarding.
A Plain-Language Scoring Model
Score each of the five criteria on a 1–5 scale. Weight them:
| Criterion | Weight |
|---|---|
| Business impact | 25% |
| Data availability and quality | 30% |
| Time-to-value vs. maintenance burden | 20% |
| Integration complexity (lower = better) | 15% |
| Staff capability and change tolerance | 10% |
Multiply each score by its weight, sum the results. Maximum is 5.0.
Minimum Threshold Scoring, What Disqualifies a Use Case
A weighted score below 3.0 doesn’t pass. More importantly: any score of 1 on data availability or quality disqualifies the use case regardless of total score. Do not average past this, you will build something that degrades within six months and requires more time to maintain than it saves.
Portfolio Balance: Quick Wins vs. Strategic Bets
Aim for a 70/30 split: 70% of investment in use cases scoring 3.5–4.5 (reliable ROI, manageable risk), 30% in strategic bets scoring 4.5+ with longer payback horizons. Don’t fill your portfolio with 3.0 scores, they cluster around “possible but probably not worth it.”
The Cases Where You Should Not Proceed
The honest output of a rigorous prioritisation process is sometimes “don’t build this.” 73% of failed AI projects had no agreed definition of success before they started, Gartner, 2026. Most of them would have been filtered here.
Red Flags in Your Own Data Readiness
- Data lives in more than two systems with no unified schema
- No one owns the data quality process
- Historical records exist but haven’t been audited in 12+ months
- Ground truth labels require manual review by a subject matter expert who is not available for a scoping sprint
Any of the above should trigger a data readiness sprint before AI scoping, not alongside it.
Red Flags in Vendor Proposals
A vendor-supplied prioritisation framework is a conflict of interest. They get paid when you build; “don’t build” costs them a project. Watch for:
- Use cases scored on capability fit (what the vendor can build) rather than business fit (what your operations actually need)
- Frameworks that don’t include a disqualification threshold
- Pilot proposals that delay the data quality conversation until after contract signature
- ROI projections that don’t account for maintenance cost in year two and three
Ask any vendor: “What’s your process for recommending against a project?” If they don’t have a clear answer, you’re dealing with someone who won’t filter bad use cases for you.
Applying the Framework in Practice
This scoring model works in a two-hour working session with three to five people: the process owner, someone with data access, and someone with budget authority. You don’t need a consultant in the room.
Running the Scoring Session with Your Team
Start with the use case definition, not the scoring. If the team can’t agree on what problem is being solved and how success is measured, stop. Document that gap and schedule a problem definition session first.
Once the problem is defined, score each criterion independently before sharing scores. Anchoring bias, where one person’s score influences others, is the most common source of inflated totals. Average the independent scores, then discuss outliers.
Documenting Decisions and Setting Success Criteria Up Front
Every use case that passes threshold scoring should produce a one-page brief with: current state metrics, projected state metrics, success definition, data source and quality status, integration map, and a named owner. 88% of organisations use AI in at least one function, only 39% see any EBIT impact (McKinsey, 2025). The gap is almost entirely explained by absent pre-deployment success criteria.
This brief is what we build from at Designodin. If a client can’t produce it before we scope, we help them write it, because scoping without it produces a contract, not a solution.
Frequently Asked Questions
What is an AI use case prioritisation framework?
A prioritisation framework is a structured scoring model that evaluates potential AI projects against defined criteria before any build work begins. The goal is to separate use cases with genuine operational fit from those driven by vendor enthusiasm or internal hype. A good framework produces “no” outcomes as often as “yes” outcomes.
How do you score AI use cases for operations?
Score each use case across five weighted criteria: business impact, data availability and quality, time-to-value versus maintenance burden, integration complexity, and staff capability. Weight data quality at 30%, it’s the most common failure point. Use a 1–5 scale per criterion and set a minimum threshold score (3.0 weighted) below which no project proceeds, regardless of other scores.
What percentage of AI projects actually deliver ROI?
Gartner’s 2025–2026 survey of I&O leaders found only 28% of AI infrastructure and operations projects fully meet ROI expectations. RAND Corporation data shows 80.3% of AI projects fail to deliver intended business value. Only 6% of organisations achieve payback in under a year, and those concentrate on a small number of tightly scoped use cases with pre-agreed success criteria.
How is AI prioritisation different for SMBs vs. enterprises?
Enterprises can absorb failed experiments at scale; SMBs typically cannot. The data infrastructure gap is wider: SMBs rarely have dedicated data engineering, labelled training sets, or ML Ops capacity. For SMBs, the data availability criterion should be weighted even more heavily, and any use case requiring custom model training (rather than API-based inference) should be disqualified unless data infrastructure is already in place.
What should be in a minimum viable AI use case brief?
Six items: the current process and its measurable cost, the projected outcome and its measurable benefit, a definition of success that can be checked at 90 days, the primary data source and a quality status assessment, a high-level integration map showing which systems are touched, and a named owner who is accountable for adoption after launch. Without all six, you’re scoping blind.
When should you not build an AI solution at all?
When data quality is poor, when the problem is unclear, or when the manual process it would replace costs less than the build and maintenance cost over 24 months. “Not building” is a legitimate output of a rigorous prioritisation process. If a vendor’s framework never produces that output, the framework is designed to sell, not to advise.
If you want to talk through what this looks like for your operation, start a conversation. We’ll tell you honestly whether the use case passes the filter, before any commitment is made.