Cloud Billing Usage Dashboard

🚧
Case study under construction

Company IBM Cloud (Billing & Cost Management)

Timeline Spring 2019 - March 2021

My Role Project Design Lead

Collaborators Tracey King, Kristin Holifield, Hannah Moyers, Haley Lucey

‣
Expand section to see my roles and responsibilities

Project background

The IBM Cloud Billing & Cost Management team had completed our State of Billing UX research project (view case study) the year prior, as well as 2 additional billing-related studies in collaboration with our UX researcher. This project references findings from each of these studies.

‣
📚  Expand section to see research studies referenced

We were well underway in delivering a vastly improved invoice reconciliation experience for our billing users (view case study). With the invoice improvements underway the team was ready to begin addressing the other key billing-related issues identified in the research studies referenced above.

The problem

Outside of the reconcile invoice experience, the most prominent billing-related item we consistently received negative feedback on was our monitor usage experience.

We found that many of our IBM Cloud Billing users were struggling to understand their spending and usage across the platform.

  • ~73% of users unable to successfully locate a specific charge in a given project.
  • ~73% of users unable able to successfully locate a prediction for next months spend.
  • 100% of users unable to view their PaaS and IaaS charges in a single place (must reference and total charges from multiple pages on and off platform).
  • 5 experience breaking pain point themes encountered by the majority of our users.
  • System usability score of ~41. Even accounting for a 8 point margin of error, this is 19 points below the industry average SUS score of 68.

Issues elaborated on in further detail below in the Understanding the problem section.

Project overview

Team goal

Increase the overall TCR (task completion rate) and SUS (system usability score) for core billing-related jobs, by addressing the UX gaps and technical issues prevalent on the IBM Cloud Usage dashboard (where usage monitoring occurs).

Design objectives

🔍
Understand the problem Understand the root causes of low TCR, SUS scores, and experience breaking bugs preventing our users from understanding their spending and usage.
🤝
Drive alignment around user outcomes Drive alignment around user needs and outcomes with collaborators across 3 distinct stakeholder organizations.
📐
Design a solution Design a revamped Usage dashboard solution (in tandem with the updated invoices) that addresses the billing user’s core needs and pain points.

Understanding the user

This effort is primarily focused around meeting the needs of the Cloud Cost Analyst, one of 3 key Billing personas initially identified in our State of Billing UX research project and later expanded upon in our Business Parter Learnings study.

image

Who is the user?

While the jobs of the Cloud Cost Leader and Cloud Cost Advocate personas do sometimes overlap, the majority of the responsibility around usage monitoring and cost optimizations typically falls on the Cloud Cost Analyst.

image

What do they do?

Cloud Cost Analysts work to...

Expand the sections below to see examples and learn more.

‣
💰  Understand their organization’s costs
‣
🔍  Identify optimization opportunities
‣
👩🏻‍💻  Take action on those opportunities
‣
📈  Track the impact of the action taken

Choosing a starting point (determining scope)

It quickly became clear after initial scoping discussions with PM, Engineering and Research that each of the tasks above were large enough to be their own project. Of the 4 billing-related tasks the Cloud Cost Analyst is responsible for we decided to zero in on the first one:

Understanding their organization’s costs

Why? If billing users are unable to do something as fundamental as understand their organization’s costs, then all of the other tasks they’re responsible for essentially become impossible to complete. It made the most sense to start here.

What are their needs (use cases)?

From the 120+ interviews and NPS feedback we received, our IBM Cloud Billing users were not afraid to express their specific needs related to understanding usage and spending details:

Here are a few of the most commonly vocalized needs...

1️⃣
Single view of all usage View usage for all their services in a single place, tracking their spend over time and which services contribute the most to their overall usage.
2️⃣
Slice and dice usage details Drill into usage specific details at various levels of granularity. Isolate usage metrics by specific parameters like service, creator, project, resource group, tag, region, etc.
3️⃣
Custom time ranges Set custom time ranges for their usage views to monitor how usage declines or spikes during specific periods of time (like a product release).
4️⃣
Data visualizations Create share data visualizations to better understand spending and usage data.
5️⃣
Export usage data Generate and export usage reports in CSV or Excel format to perform reporting analysis in a separate system or tool used by their financial group.
6️⃣
Save and share Save usage views for easy access at any point in time. Share usage reports with other stakeholders using a single link.

As elaborated on below, the IBM Cloud Usage dashboard experience does not support most of these use cases.

Understanding the problem(s)

As stated above, IBM Cloud billing users struggle to understand their spending and usage across the Platform. When compared to our competitors we lack most of, if not all of the features they include as standard elements of their usage monitoring experiences.

image

Where is spending and usage tracked today?

Ideally users would be able to track their spending and usage from a single page, the IBM Cloud Usage Dashboard located in Billing & Usage.

image

IBM Cloud Usage Dashboard.

Unfortunately, this is not often the case...

image

Due to technical restraints in how we can surface spending and usage data, most users have to toggle through a minimum of 3 different pages to get a complete view of their usage/spending.

IBM Cloud billing users lack a single view of their usage and billing data across their IaaS and PaaS offerings.

Are there other pain points?

Yes, many. 😢

Outside of not being able to slice and dice spending and usage charges from a single page, there are innumerable pain points across the Usage, Billing items, and Invoices pages (many of which are detailed in the State of Billing UX research project).

Pain point themes

Expand the sections below to see examples and learn more.

‣
💻  Users lack a single view of usage across all their services (IaaS & PaaS)
‣
🧾  Users cannot match invoices received to their usage data
‣
🗓 Users cannot accurately calculate their next bill for their accounts’ payable team
‣
📈  Users cannot easily inspect service details to identify the source of usage spikes
‣
💸  Users do not have a reliable & accurate way to be made aware of current overages

On top of that, the Usage dashboard was not consistent with IBM Cloud branding standards and had been dinged for many basic visual, layout and page hierarchy issues.

Anything else?

image

Design explorations

With an understanding of the user, their use cases and existing pain points, I got to work on my design explorations.

What makes a good usage dashboard?

[Coming soon: include quotes, examples]

Sketches & iterations

[Coming soon: Include Brainstorming, Sceanrios]

Filter patterns

[Coming soon]

User testing

[Coming soon]

Finalizing designs

[Coming soon]

image

Outcomes

[Coming soon]

What I would do different / next

[Coming soon]

Investigate the remaining 3 tasks.

Why?