Are you measuring the right things to achieve the right outcomes?
What gets measured gets managed — right?
This mantra is a key reason why Net Promoter Score (NPS) and Customer Satisfaction (CSat) are used so widely to measure the effect an organisation’s activities have on customers and users.
But, if you want to influence the right CX, outcome metrics like NPS and CSat are the last metrics to focus on.
A customer’s experience is obviously driven by the quality of interactions across a range of business functions including finance, distribution, operations, marketing, production, technology and logistics, so, it’s often down to how measures and targets are set for these functions, that decides what your customers and users experience. As a result, it’s vital to understand the relationship between those operational metrics and customer outcomes, and consider them in overall service design and CX management activities.
Customer focused targets don’t guarantee a customer focused outcome.
What can appear to be a great customer centred target, such as less effort, faster delivery, cheaper product and better performance, can sometimes drive the wrong organisational behaviours and ultimately outcomes that are actually wrong for customers.
This is often known as Goodhart’s law, “When a measure becomes a target, it ceases to be a good measure”. In other words, when a specific goal is set, people will go for that objective regardless of consequences.
There are many examples where targets have led to unintended consequences for product and service experience.
Back in the late 1960s, Ford set goals to deliver a car that weighed under 2000lbs and cost under $2000. Sounds great — something lots of customers would want. The result was the Ford Pinto.
But, in chasing the design and production targets (it was designed and produced in almost half the time normally taken for cars at the time), quality and safety suffered and, in the end, 53 people died due to the poor design and fitting of the gas tank. (See crash test picture above).
In another example, what appeared to be a wise approach to reducing air pollution in Mexico City, in fact ended up delivering the exact opposite. In order to reduce car usage the city banned cars for one day per week, depending on the last number of the reg plate. Initially the scheme achieved great results, bringing down CO2 emissions by 11%. However, the scheme also had the unintended consequence of enlarging the 2nd hand car market. Residents started buying second cars (often inefficient ones) to get around the ban. This in part drove a long-term rise in CO2 of 13%.
More recently, in the early 2000s, the UK Govt set all sorts of public service performance targets, with the worthy intention of improving outcomes for all. But, in some circumstances, the targets chosen resulted in problematic behaviour shifts and systematic gaming, which resulted in the very opposite of initial intentions, a poor experience.
For example, some hospital trusts overlooked key patient care in order to hit wait time targets. Schools focused on the use of less challenging subjects to deliver results, at the expense of key subjects like English and Maths. In order to hit overall exam grade targets, some school resources were re-directed to focus on kids who were on the cusp of meeting exam grades, to the detriment of other student needs. There were even some cases, in police services, where victims were pressured into dropping claims, in order to achieve target clear up rates.
Independent research has also been undertaken, into how targets can bend behaviours. For example, Harvard Business School found that stringent goal setting can lead individuals and groups to cheat, in order to meet their target objective and I’m sure many of us have seen this kind of behaviour in action, even with simple exercises, such as collection of NPS or CSat responses, where customers have been pressured into giving particular scores, in order for employees to meet targets.
Another impact of poorly applied targets can be employee mental health. Although stretch targets and challenging deadlines are an everyday factor within work, aggressive targets, if applied poorly, can have a real adverse impact on those that have to work under them.
Good targets, however good, are never good enough.
No matter how good targets are, or how well aligned they are across the organisation, they are never an answer, in themselves, for driving the right behaviour and outcomes.
The process of creating a system of metrics that meets all needs within complex organisations, is incredibly hard and rarely foolproof, therefore targets should always be applied alongside an effective qualitative management framework.
Of course, the consequence of misapplied targets may not literally be a car crash, but they can, as we have seen, have other significant, albeit unintended, consequences.
So, are you really measuring the right things?
To ensure that your metrics don’t lead to unintended consequences and poor impacts on your employees, organisation and customers, ask yourself…
- Do you know what key aspects of your organisational performance drive your customer metrics?
- Do your organisational targets align to what your customers value?
- Do your strategic ‘balanced scorecard’ targets include voice of the customer in addition to the normal voice of finance, sales and operation?
- Are your targets compatible across the organisation? In other words, are they considered as part of a joined-up system or do targets in one part of the organisation conflict with those in another?
- Are targets driven by operational demands first and customer demands second?
- Do your people understand and buy into the targets?
- Can you illustrate the consequences of certain metrics, if followed blindly without consideration on the impact for end users and customers?
- Can measures and targets be changed easily if issues are identified?
- Are the measures and targets you use really needed — can more effective results be delivered in other ways, such as more focused leadership and coaching?
If you can’t answer those questions or your answer is ‘no’ to one or more — you may be at risk of creating unintended consequences and with that creating problems in delivering the kind of experience you want and that your customers and users need.
