Improving engagement through product education: Building and implementing a template library for UsabilityHub
đź‘‹

Improving engagement through product education: Building and implementing a template library for UsabilityHub

Company
UsabilityHub
Created
Jan 4, 2022 7:59 AM
Role
Director of product
Tags
DiscoveryUX ResearchStrategyDelivery managementProject managementProduct management
When
March 1, 2020 → June 30, 2020

Goal

UsabilityHub had just established their product team and had done their first ever round of OKR planning, with two specific quarterly objectives identified:

  • Increase engagement through customer education
  • Build a major feature.

The goal to build a major feature was simply to get the product team forming (then hopefully storming, norming and performing). Prior to this the whole company had been working on a repricing project and there was no product roadmap or strategy yet.

My role

As Director of product my role here was not only to lead the strategy but to act as product manager and lead designer in this small team. I was supported by two engineers, the CEO and CTO, and the Director of CX who assisted primarily with discovery research and acted as a subject matter expert.

Achievements

  • In lieu of a roadmap or product strategy, solicited ideas from around the business using a “Product Bet” template that encouraged anyone to contribute ideas framed within the constraints of hypothesised business value, customer value and risk.
    • Scored various proposals collectively and then made the final decision from the combination of group feedback, meaningfulness to leadership and product teams and my own call as Director of product.
  • Developed a reusable high-level product project template (albeit a little waterfall/linear but helpful to outline the shape of design thinking at a team implementing it for the first time)
    • Identify an opportunity (hypothesise a problem/solution match)
    • Explore the problem
    • Define needs and goals
    • Explore ideas
    • Define the solution
    • Develop the solution
    • Ship the solution
    • Measure the solution

Identify an opportunity

  • Once templates had been establised as the key feature (an idea proposed by the CEO), worked closely with leadership team to understand the hypothesis (Lean canvas, project canvas
  • Ranked and categorised hypothesised solutions and outcomes with relevant hypothesised customer needs)
  • Eventually narrowed down the goal of the feature - focusing on first-use education rather than power-user reuse/governance

Explore the problem

  • Conducted an audit of existing example library (in collaboration with the broader product team) in order to assess whether the content was eligible for use in this feature
    • Ranked example tests/content according to traffic obtained from Google Analytics
    • Assessed each piece of content for quality of advice, quality of example designs, test construction, freshness of designs and relevance to the current product suite
  • Conducted exploratory research to understand the first-use experiences of various customers via remote moderated interviews
    • Discovered two distinct usage patterns and archetypes — customers simply exploring the product without having committed to it, and customers who had prepared resources and were ready to run a test

Define needs and goals

  • Worked with key stakeholders to choose an archetypes needs to focus on
    • Decided the solution should support both archetypes with a lean towards those who were still evaluating the product
  • Discovered improvements for the marketing site to add to the backlog based on this research which could be actioned at a later date

Explore ideas

  • Ran a competitor research/inspiration gathering session with the broader team, asking them to bring template/onboarding examples from various experiences (both close competitors and other more oblique solutions)
    • Identified strengths and weaknesses of various approaches
  • Ran a sketching ideation workshop with the broader team to generate ideas for how the templates might be incorporated into the product at a high level
    • Used the Design Sprint “Art Museum” technique to crit

Define the solution

  • Results from the ideation workshop uncovered that templates could be introduced at various points in the user journey — created user journey maps to evaluated the relative strengths and weaknesses of each intervention point
    • Decided on a “welcome screen” pattern for the first iteration — introducing the templates as soon as a new use had signed up for the product instead of a blank screen

Develop the solution

  • Rapidly iterated through mid-fidelity wireframes in Balsamiq in collaboration with front-end developer
    • Tested wires quickly with unmoderated design surveys via UsabilityHub
    • Refined in high fidelity in code by pairing with front-end
    • Further tested designs at different levels of fidelity
  • Simultaneously worked with Director of CX to construct subject matter expert content
    • Iteratively tested and refined content using unmoderated design surveys
  • After discovering customers would benefit greatly from seeing the test-taking experience, created short videos of the tests in action to include as part of the template experience.

Ship the solution

  • Worked closely with engineering team to develop an extensible foundation allowing templates to grow over time
  • Developed the solution in a matter of weeks, not months
  • Worked closely with engineering to ship the solution iteratively by building and shipping to production in stages in order to reduce risk
  • Wrote the blog post and in-product comms to launch the feature
  • Ran the launch campaign on social media.

Measure the solution

  • Evaluated responses using Intercom for in-situ feedback
  • Evaluated experience in moderated research interviews to understand whether it met user expectations
  • Monitored outcomes via metrics on a Metabase dashbord.

Results

image

image
image
image

image
image
image
image
image
image
image
image
image
image
image

Results

  • Customer feedback was 95% positive (as measured via Intercom)
  • Template foundations/infrastructure in code was used in further projects
  • Audit for content on the example library provided a brief for future improvements of that body of work or integration with the template library feature
  • Minor increase in acquisition — not as much as we’d hoped for!
  • Decrease in support queries about how to set up tests.
image

Testimonial

“Milly has a way of clarifying the complicated. The process of iterating on templates, to a point where adoption and utility rates were terrific, was driven by her ability to crystallise customer and business needs to deliver a feature of incredible value to UsabilityHub customers.”

— Amanda Warmerdam, (formerly) Director of CX at UsabilityHub