Company IBM Cloud (Billing & Cost Management)
Timeline Spring 2019 - March 2021
My Role Project Design Lead
Collaborators Tracey King, Kristin Holifield, Hannah Moyers, Haley Lucey
Project background
The IBM Cloud Billing & Cost Management team had completed our State of Billing UX research project (view case study) the year prior, as well as 2 additional billing-related studies in collaboration with our UX researcher. This project references findings from each of these studies.
We were well underway in delivering a vastly improved invoice reconciliation experience for our billing users (view case study). With the invoice improvements underway the team was ready to begin addressing the other key billing-related issues identified in the research studies referenced above.
The problem
Outside of the reconcile invoice experience, the most prominent billing-related item we consistently received negative feedback on was our monitor usage experience.
We found that many of our IBM Cloud Billing users were struggling to understand their spending and usage across the platform.
- ~73% of users unable to successfully locate a specific charge in a given project.
- ~73% of users unable able to successfully locate a prediction for next months spend.
- 100% of users unable to view their PaaS and IaaS charges in a single place (must reference and total charges from multiple pages on and off platform).
- 5 experience breaking pain point themes encountered by the majority of our users.
- System usability score of ~41. Even accounting for a 8 point margin of error, this is 19 points below the industry average SUS score of 68.
Issues elaborated on in further detail below in the Understanding the problem section.
Project overview
Team goal
Increase the overall TCR (task completion rate) and SUS (system usability score) for core billing-related jobs, by addressing the UX gaps and technical issues prevalent on the IBM Cloud Usage dashboard (where usage monitoring occurs).
Design objectives
Understanding the user
This effort is primarily focused around meeting the needs of the Cloud Cost Analyst, one of 3 key Billing personas initially identified in our State of Billing UX research project and later expanded upon in our Business Parter Learnings study.
Who is the user?
While the jobs of the Cloud Cost Leader and Cloud Cost Advocate personas do sometimes overlap, the majority of the responsibility around usage monitoring and cost optimizations typically falls on the Cloud Cost Analyst.
What do they do?
Cloud Cost Analysts work to...
Expand the sections below to see examples and learn more.
Choosing a starting point (determining scope)
It quickly became clear after initial scoping discussions with PM, Engineering and Research that each of the tasks above were large enough to be their own project. Of the 4 billing-related tasks the Cloud Cost Analyst is responsible for we decided to zero in on the first one:
Understanding their organization’s costs
Why? If billing users are unable to do something as fundamental as understand their organization’s costs, then all of the other tasks they’re responsible for essentially become impossible to complete. It made the most sense to start here.
What are their needs (use cases)?
From the 120+ interviews and NPS feedback we received, our IBM Cloud Billing users were not afraid to express their specific needs related to understanding usage and spending details:
Here are a few of the most commonly vocalized needs...
As elaborated on below, the IBM Cloud Usage dashboard experience does not support most of these use cases.
Understanding the problem(s)
As stated above, IBM Cloud billing users struggle to understand their spending and usage across the Platform. When compared to our competitors we lack most of, if not all of the features they include as standard elements of their usage monitoring experiences.
Where is spending and usage tracked today?
Ideally users would be able to track their spending and usage from a single page, the IBM Cloud Usage Dashboard located in Billing & Usage.
IBM Cloud Usage Dashboard.
Unfortunately, this is not often the case...
Due to technical restraints in how we can surface spending and usage data, most users have to toggle through a minimum of 3 different pages to get a complete view of their usage/spending.
IBM Cloud billing users lack a single view of their usage and billing data across their IaaS and PaaS offerings.
Are there other pain points?
Yes, many. 😢
Outside of not being able to slice and dice spending and usage charges from a single page, there are innumerable pain points across the Usage, Billing items, and Invoices pages (many of which are detailed in the State of Billing UX research project).
Pain point themes
Expand the sections below to see examples and learn more.
On top of that, the Usage dashboard was not consistent with IBM Cloud branding standards and had been dinged for many basic visual, layout and page hierarchy issues.
Anything else?
Design explorations
With an understanding of the user, their use cases and existing pain points, I got to work on my design explorations.
What makes a good usage dashboard?
[Coming soon: include quotes, examples]
Sketches & iterations
[Coming soon: Include Brainstorming, Sceanrios]
Filter patterns
[Coming soon]
User testing
[Coming soon]
Finalizing designs
[Coming soon]
Outcomes
[Coming soon]
What I would do different / next
[Coming soon]
Investigate the remaining 3 tasks.
Why?