ERP Has a Measurement Problem: Are We Tracking the Wrong Things?
ERP programmes measure almost everything. Budgets are tracked, milestones are monitored, defects are counted, testing completion is reported, data migration is measured and system availability is watched closely. By the time an ERP programme reaches deployment, leadership can be surrounded by an enormous amount of data.
Yet one surprisingly difficult question can remain unanswered: is the transformation actually making the business better?
It exposes an uncomfortable problem within ERP. Organisations have become extremely good at measuring whether a programme is being delivered, but considerably less consistent at measuring whether the transformation itself is working.
That distinction is becoming increasingly important as ERP programmes move beyond technology replacement and towards broader changes in processes, operating models, data and decision-making. OXYGY's approach to IT-enabled transformation offers an interesting perspective on this problem by connecting three dimensions that ERP programmes have historically struggled to measure together: People, Process and Performance.
Perhaps ERP does not suffer from a lack of metrics. Perhaps it suffers from measuring the wrong things.
When a Green Dashboard Doesn't Mean Success
Traditional programme metrics serve an important purpose. Executives need to know whether an implementation is on schedule, while programme teams need visibility into testing, defects, migration and readiness. Without these measures, managing a major ERP implementation would be almost impossible.
The problem appears when project performance becomes confused with transformation performance.
An ERP programme can hit its implementation milestones while operational performance deteriorates. The system can be available while employees struggle to complete processes. Training can reach 100% completion while users remain uncertain about their responsibilities. A process can be technically standardised while teams quietly create workarounds outside the system.
All of those things can exist at the same time.
OXYGY's transformation approach attempts to close this gap by connecting system implementation with people's behaviour, process execution and measurable business performance rather than treating them as separate conversations.
That changes what an ERP dashboard needs to tell leadership. It is no longer enough to know whether the programme is progressing. Leaders increasingly need to understand whether the organisation is progressing with it.
Completion Is Not the Same as Performance
Consider training. One of the easiest measurements available to an ERP programme is training completion. If 98% of the target population has completed its assigned learning before deployment, the dashboard looks reassuring.
But what does that number actually tell us?
It confirms that people completed the training. It does not necessarily tell us whether they can perform their role successfully in the new environment.
The same problem appears elsewhere. A test can be completed without proving that a business process will operate efficiently at scale. A system can successfully record transactions without demonstrating that users are following the intended process. A data migration can achieve technical accuracy while underlying ownership problems continue. A go-live can happen on schedule while productivity falls sharply afterwards.
The distinction is subtle but important. Activity measures what happened. Performance measures what changed because of it.
ERP programmes need both.
What Should ERP Leaders Actually Measure?
A better measurement model starts by asking what the transformation was supposed to achieve in the first place.
If the investment was justified by faster processes, measure process speed. If it was intended to improve inventory accuracy, measure inventory accuracy. If automation was supposed to remove manual work, measure whether that work has actually disappeared. If standardisation was supposed to eliminate local workarounds, measure whether those workarounds continue.
And if employees are expected to operate differently, organisations need some way of understanding whether those behaviours are actually appearing.
This sounds obvious, but in practice it is considerably harder than tracking whether a project milestone was completed.
OXYGY's approach brings these different dimensions together. People measures can help organisations understand readiness, confidence and capability. Process measures can reveal whether new workflows are becoming part of everyday operations. Performance measures can then connect those changes to the outcomes the transformation was intended to produce.
The real value is not necessarily any individual metric. It is understanding the relationship between them.
The Weeks After Go-Live Can Reveal More Than Go-Live Itself
One particularly interesting area to measure is the period immediately after deployment.
ERP programmes often treat this primarily as a period of stabilisation. But it can also provide some of the richest information available about whether the transformation is actually working.
How quickly does the business return to normal productivity? How many users require additional support? Where are workarounds appearing? Which processes generate the most exceptions? How quickly are issues being resolved? Are teams becoming increasingly independent, or does support demand remain consistently high?
OXYGY has highlighted this type of measurement through its own transformation work. In one global FMCG ERP rollout, reported outcomes included business recovery time reducing from six weeks to three, hypercare reducing from four months to two and user adoption during the first two weeks increasing from 60% to 90%.
These measures tell us considerably more than whether a system was technically deployed on the planned date.
They tell us what happened to the organisation afterwards.
ERP Value Rarely Sits Within One Department
There is another reason ERP performance is difficult to measure: ERP value rarely belongs neatly to one function.
Imagine a transformation that improves master data quality. Better data may reduce procurement errors. That could improve inventory accuracy. Better inventory information could improve production planning. Improved planning could increase on-time delivery, which could eventually influence customer satisfaction, working capital or margin.
Where exactly did the ERP create value?
The answer is across the process.
Yet organisations are often structured and measured functionally. Finance measures finance. Procurement measures procurement. Operations measures operations. IT measures system performance.
ERP connects these areas, but the measurement model may not.
This creates a strange situation where organisations implement an integrated enterprise platform while continuing to evaluate its success through fragmented metrics.
As ERP becomes increasingly connected across the enterprise, end-to-end performance measurement needs to become more important too.
Workarounds Could Be One of ERP's Most Valuable KPIs
There is another potential ERP metric that receives surprisingly little executive attention: the workaround.
Spreadsheets, offline approvals, manual reconciliations, duplicate data entry, unofficial reports and processes completed outside the intended system are often treated as inconveniences.
Collectively, however, they can provide an extraordinarily useful signal about transformation health.
If employees repeatedly leave the ERP environment to complete a task, something is happening. Perhaps the process is poorly designed. Perhaps the system does not reflect operational reality. Perhaps users do not understand the new process. Perhaps local requirements were overlooked. Or perhaps the organisation never truly stopped using the old way of working.
From a measurement perspective, that suggests something interesting. Instead of simply trying to hide or eliminate workarounds, ERP leaders could actively measure them.
Where are they appearing? Why are they happening? Are they increasing or decreasing? What does their existence tell us about the underlying process?
Sometimes a spreadsheet sitting outside the ERP system can reveal more about transformation performance than a green programme dashboard.
AI Makes the Measurement Question Even Bigger
This issue becomes more significant as AI and automation move deeper into enterprise systems.
Traditional ERP environments largely required people to execute processes through software. Increasingly intelligent ERP environments can recommend actions, identify exceptions, automate decisions and potentially execute parts of processes autonomously.
That changes the measurement challenge.
If AI automates a task, organisations should not simply measure whether the AI capability was used. They need to understand what happened to the process as a result.
Did cycle time improve? Were fewer exceptions created? Did decision quality improve? Was manual effort genuinely removed or simply moved somewhere else? Did employees trust the recommendation? How frequently did they override it? Most importantly, did the automation improve the business outcome it was introduced to influence?
As enterprise AI becomes easier to deploy, measuring usage will be tempting.
But usage is not value.
The same lesson ERP has learned with system adoption will increasingly apply to AI.
The Business Case Shouldn't Disappear After Approval
Most ERP programmes begin with a business case. Efficiency gains are identified, costs are modelled, productivity improvements are forecast and potential benefits are used to justify the investment.
Then implementation begins.
Too often, the business case gradually becomes less visible as programme delivery takes over. The transformation starts being managed against scope, time and budget rather than against the outcomes that justified the investment in the first place.
There is an opportunity to change that.
Instead of treating the business case primarily as a document used to secure investment, organisations could treat it as the foundation of the transformation's measurement model.
Every significant promised benefit should have an associated measure. Every measure should have an owner. Those measures should then survive long after the implementation programme itself closes.
This is where OXYGY's emphasis on People, Process and Performance becomes particularly relevant. Connecting what people are doing, how processes are operating and what is happening to business performance provides a clearer way of understanding whether transformation is actually creating value.
It turns benefits realisation from something organisations review retrospectively into something they can actively manage.
ERP Needs Two Definitions of Success
Perhaps the answer is not to abandon traditional programme metrics. It is to recognise that ERP requires two different definitions of success.
The first is delivery success: did we build and deploy what we intended to?
The second is transformation success: did the organisation become better because we did?
Both matter, but they are not interchangeable. An ERP programme can succeed at the first and fail at the second.
For CIOs, CFOs and transformation leaders, that distinction should change the questions being asked in steering committees and boardrooms.
Not only, are we on schedule? But, are processes improving?
Not only, how many people completed training? But, can people perform effectively?
Not only, is the system stable? But, is the business performing better?
And not simply, did we achieve go-live? But, are we achieving the outcome that made the transformation worth doing in the first place?
ERP has never lacked data.
The next challenge is making sure organisations are measuring what actually matters.