Case study · OVO Energy · User research

Usability testing the OVO Energy app

After a platform upgrade changed the app and moved key metrics, I planned and ran usability testing to find out whether the app still worked for customers.

My role
Research planner and interviewer
Method
Remote, moderated usability testing with a live feed for the team
Participants
5 OVO Energy customers
Impact
Findings led to the home screen redesign
A Trello board from the usability testing, with a list of research participants and two lists of notes captured during the sessions.
Why research

Platform-led changes had moved the metrics

The OVO Energy app had gone through several rounds of changes, mainly because of a back-end platform upgrade. These changes were feature-led, based on what the new platform could do, rather than what customers needed.

Analytics then showed that some of the metrics we tracked had been affected. So I set out to test whether the new app was still fit for purpose.

Research goals

Agreeing what to learn before testing

I started by creating a research canvas and sharing it with stakeholders. It told them what I planned to research and what recruitment I needed. It also gave them a chance to suggest changes or additions to the research goals.

A one-page research canvas in Google Slides titled Is the Orion app fit for purpose?, with sections for audience, responsibilities, context, what we want to discover, research goals, methodology, participants, budget and expected output.
The research canvas set out the goals and recruitment, and gave stakeholders a chance to shape them.
Recruitment

Preparing the participants

I wanted participants to know what the research was about and what was expected of them. So I sent each of them a research agreement. It covered who I was, what the session would cover, the technology they needed and the compensation for taking part.

The participant research agreement in Google Docs, with the OVO Energy logo and an information sheet that introduces the researcher and explains the purpose of the research.
Each participant got an agreement explaining the session, the technology needed and their compensation.
Discussion guide

Writing and stress testing the guide

I shared the discussion guide with the team while I was editing it, so they could see and shape what we would ask. I also ran mock sessions to stress test it before the research day.

The discussion guide in Google Docs, titled Orion App research, February 2020, opening with a five-minute introduction script for the session.
The discussion guide, shared with the team and stress tested in mock sessions.
Research day

Bringing the whole team into the sessions

The research was carried out remotely, with a live feed for the rest of the team to follow. I wanted everyone to see customers using the app first-hand, so the whole team built a shared understanding of the problems rather than relying on my summary.

Analysis

From live notes to themes

Everyone captured notes in Trello, which kept everything online and shareable. Each participant had a card, and notes were sorted into areas of the app, such as the home screen, payments and usage. Once the day was over, Trello also made affinity mapping the notes much easier.

A Trello board titled OVO App usability testing, round 3, with a list of participant cards and lists of notes for General, Home, Payments and Usage.
The team’s notes in Trello, grouped by area of the app and ready for affinity mapping.