If you’ve been struggling to adopt useful metrics in your user experience team – you might want to consider the HEART framework designed by Kerry Rodden, Hilary Hutchinson and Xin Fu, from Google’s research team.

The idea is a simple one; to deliver a series of user-centred metrics that allow you to measure the user experience on a large scale. These metrics can then be used for decision making in the product development process.

An Introduction to the Heart Framework

Measuring user experience on a small scale is relatively easy. It’s what user experience designers do on a daily basis. You can observe users, talk to them, ask them questions, etc. and get rapid feedback.

What the research team from Google noted was that while small scale frameworks were common place measuring the experience on a large scale via automated means had no framework in place. Thus the Heart Framework is specifically targeted at that kind of measurement. However, the principles are equally useful at a small scale level; though the methodologies used to derive measurements at a smaller scale are likely to be substantially different.

The Heart Metrics

There are five metrics used in the HEART framework:

Happiness

Engagement

Adoption

Retention

Task Success

The research team notes that not all projects will require all the metrics and that metrics should be chosen (in combination) based on the outcomes required from the metrics. The example they cite in the research report is engagement; a measurement of limited value in enterprise contexts because many users are not choosing to use a system – they’re required to use one as part of their job. Engagement is thus of little relevance when examining such systems.

Author/Copyright holder: Google Inc. Copyright terms and licence: All rights reserved Img source

Happiness

As you might expect this is a measure of attitude or satisfaction. You’re most likely to record satisfaction on large scale projects through some sort of user survey. An example cited in the report shows that change can impinge on happiness and that an initial drop in happiness following a change does not necessarily have long-term implications.

As with all metrics, a snapshot in time is not enough to base decision making on. Long-term observation of metrics will provide better data for decision making on projects.

Engagement

This is a measure of how much a user interacts with a product, of their own volition. As we’ve noted already – it may be a poor metric for enterprise systems because there’s no optional element in usage patterns. If you’ve got to do a job; you’ve got to do the job – whether you like the tool you’re doing the job with or not.

Measurements in this category might examine the regularity of use, the intensity of use or the overall level of interaction over a period of time. The right metrics will vary from product to product. A weather app, for example, is unlikely to see the same depth on engagement or intensity of engagement as an e-mail client (except in very unusual circumstances).

Adoption

Adoption is defined as the number of new users over a certain time frame. It’s a measure of how successful you are at attracting new business. It might be argued, and we can certainly see how, that this is less a measure of user experience and more a measure of customer experience.

Certainly, there’s a strong case that much of adoption will not be down to user experience but rather to sales and marketing activity. It’s entirely possible that for a short period of time a heavy investment in sales and marketing can overcome user experience problems. However, in the longer-term – a poor user experience is likely to discourage new users as they read reviews and speak to friends, peers, etc. about the product.

On this, we think it’s best to be cautious and if you are measuring adoption – you might want to share some of the credit with sales and marketing; unless you want to create a lot of bad feeling between them and the UX team in your company.

Author/Copyright holder: Fiserv Inc. Copyright terms and licence: All rights reserved Img source

Retention

Retention, on the other hand, is a matter of keeping your existing users for x amount of time. That might be an indefinite amount of time for products with long-term utility. However, you’re probably going to want to look at other time scales to work out where drop out from a service is most pronounced so that you can tackle the UX issues that lead to that drop out. A week, a month, a quarter, a year, are all perfectly reasonable intervals as are any other intervals that you can justify as relevant to your business.

Both adoption and retention metrics are going to be valuable when you roll out a new release or make substantial changes to the way a feature works. If the product isn’t evolving particularly rapidly it would be reasonable to assume that some sort of plateau will eventually take place where these metrics deliver fairly constant results.

Task Success

The final metric “task success” can be broken down into more subtle components. You might want to examine time spent on any given task (can the process be improved?) or the percentage of successful completions of a specific task once it has begun (e.g. checkout processes or registration processes).

Remote usability testing and benchmarking studies are recommended for measuring these on a large scale.

Author/Copyright holder: Optimal Workshop. Copyright terms and licence: All righs reserved Img source

Summary

The HEART framework is a useful framework because it’s simple and easy to understand. That makes communicating the reasons for selecting it easy across teams. While the framework may be designed for large projects; there’s nothing to stop it being implemented on smaller projects either – it’s just that the methods used for data collection are likely to vary.

If you’d like to learn more about the HEART framework and how to implement it in your organization; there’s a link to the original Google research paper at the end of this article. We hope you find it as interesting and as useful as we have.

Reference:

The HEART Framework Original Research Can Be Found Here: https://ai.google/research/pubs/pub36299

Header Image: Author/Copyright holder: UX Matters. Copyright terms and licence: All rights reserved Img