Key Experience Indicators

How do we know if our products and services are delivering the desired outcome?

Do we even know what the desired outcome is?

Do we strive to capture what is important to measure, or instead opt for what is easy to measure?

As part of World Usability Day 2019, we considered these questions as we looked at the whole area of KPIs for UX, or Key Experience Indicators (KEIs) as they are often referred to.

This subject falls into the area of Strategy & Governance, an often-overlooked area of human-centred design. Strategy & Governance seeks to bring the necessary rigour and structure to human-centred design, to compliment the creativity and agility often associated with it. Unfortunately without it we end up with products and services that, while potentially well designed, are inconsistent, misaligned and not providing a cohesive experience that meets both the needs of the end user and the business. Bridget van Kralingen, Senior VP at IBM, was quoted as saying; “There’s no longer any real distinction between business strategy and the design of the user experience”. Her message (as I read it) emphasises how the users’ experiences of products and services is now so central to the success of the organisation. However, I also believe that part of the message is that we have to be better at applying the rigour of business strategy to the design of those experiences. That means giving over more time and energy to the “What” and “Why” of design (the Strategy), but also the “How”, “Who” and “When” (the Governance). And a big part of this Strategy & Governance is understanding what your design goals are and how to measure them. We should be putting the same importance on Key Experience Indicators in organisations, as we do to Key Performance Indicators.

It was in this context that I had many interesting conversations on World Usability Day as I shared our approach to devising appropriate KEIs. This approach is based around a framework for choosing/positioning out KEIs and a process for defining them.

At its highest level (please get in touch if you would like us to go into more detail with you), the framework concentrates on three broad categories:

1. Facility – Can people use the product/service/feature?
2. Engagement – Do people use the product/service/feature?
3. Satisfaction – How do people feel about using the product/service/feature?

As we unpack these three areas, it provides us with a structured way to decide which things we want to measure. Once these are identified, we can then go through the following process for developing these KEIs, considering the Goals, Signals, and Metrics that will enable us to effectively measure our KEIs.

This process should ideally involve all the stakeholders in the organisation who will have responsibility for collecting, reporting or acting upon the final Key Experience Indicators. This shared view will help to ensure that the KEIs are focused, useful and measurable, and that the insight from the KEIs can and will be acted upon. The KEIs after all will only point in the direction of a potential issues. They will rarely, if ever, uncover the underlying problem. This is where qualitative research is needed to fully understand the behaviour and attitudes that are driving the indicators.

This area prompted a great deal of discussion during World Usability Day, including the challenges of choosing the right things to measure, being rigorous in only measuring things that can be changed, measuring behaviour vs attitude, and the challenge of NPS-type metrics that are limited in their usefulness. An encouraging theme however was the growing acceptance in industry that measuring the experience is of increasing importance and stakeholders getting more on board with pro-actively measuring the effect of the users’ experience.

For more information about how User Vision could help you to devise Key Experience Indicators to drive continual UX improvement in your organisation, please get in touch.