Staying competitive with a flagship feature: Deciding, designing, developing and launching prototype testing for UsabilityHub

Goal

My role

Achievements

• Developed a product strategy based on business goals, user needs and technical constraints.
• Worked with leadership to identify business priorities and broader strategic direction
• In collaboration with Director of CX, ran and analysed (open source Excel/Google Sheets formulae based on the official analysis methodology) a KANO survey with 30+ high net worth customers to understand relative importance of various features
• Groomed and analysed extensive customer knowledge database in ProductBoard to understand the patterns of customer feature requests to date (10+ years of data)
• Conducted extensive and in-depth competitor review for the user research tooling product space to build a SWOT model and in particular identify opportunities
• Finalised and presented a cohesive product strategy based on the above activities, which was then presented to the broader leadership team (Director of Sales, Director of CX, CEO, CTO, COO)
• Ran a series of strategic workshops with the product team to present business context and strategic research findings to then identify technical and user-focused priorities and solidify a tactical roadmap
• Identified two tactical priorities (Prototypes and Screeners) for the year (each a ~6 month project by early estimations)
• Set up two streams of work to investigate both features with the initial intention of building both in parallel
• Leading exploratory research for both features including in-depth anaylsis of customer feedback database (Productboard) and extensive competitor analysis at a feature level
• Formulated a template for and finalised “Bets” for both features, outlinining the business importance, customer importance, hypothesis and risks for the feature ideas
• Kicked off both features with the broader cross-functional and product teams presenting initial scoping hypotheses and product bets
• Eventually deprioritised Screeners after discovering some concerns regarding the viability of the feature after the scoping process (and learning as part of the Agile process that the scrum team did not have bandwidth for two projects in parallel, especially with only one designer to service both).
• Identified two streams of concurrent research to be done by the product team: gain a deeper understanding of customer needs and gain a deeper understanding of technical requirements
• Conducted the first stream (in collaboration with the product designer) by soliciting research from the customers who had requested the feature in ProductBoard
• Initial research included a survey (using UsabilityHub) to help understand the relative importance of various possible features (and why)
• Briefed the technical lead on the technical research goals — primarily understanding the opportunities and limitations provided to us by the Figma API and identifying any other foundational technological pieces required to execute on the product hypothesis
• Developed a high level feature overview (led by the Senior product designer) to flesh out the initial scope hypothesis and align with the technical lead based on technical research
• Led project management activities to slice the work into streams to be conducted concurrently across various parts of the user experience journey (test builder, test taking and test results)
• In collaboration with design and development leads developed a project plan to deliver an end to end MVP (to be shipped to internal customers for research)
• Set product/business goals for the feature including:
• Feature adoption (20% of tests to use this feature within the first three months)
• Competitive parity (provides features that our key competitors do in order to stop losing customers to those competitors)
• Competitive differentiation (provides features that our key competitors don’t in order to attract more customers)
• Ran the Scrum process for the product team (as effective Scrum Master/Product Owner) during the course of the project including standups, review, planning and retrospectives as well as ongoing backlog grooming
• Co-ordinated broader communication with the company outside the product team to share updates about project decisions and progress
• Led the product marketing strategy and project management
• Set up a Slack workspace for research participants in order to better manage research ops and findings as well as begin to build a customer advisory community
• Conducted various small-scale evaluative research projects during design stages to understand the merits of various design approaches (via UsabilityHub click tests, five second tests, and design surveys)
• Designed, led and coordinated internal end to end QA
• Included key stakeholders from across the business including support, marketing, executives
• Created documentation to structure and collate feedback across the various aspects of the UX of the MVP
• Trained the participants on how to give effective QA feedback and how to best record their results
• Analysed and synthesised QA feedback in collaboration with design and technical leads in order to prioritise inclusions for the internal MVP research test
• Ran the remote unmoderated evaluative research at MVP delivery via Lookback
• Recruited participants from our internal customer feedback tool and broader customer feedback panel — as well as recruitment in-app via Intercom
• Wrote the study instructions and outline (in collaboration with Senior product designer)
• 17 participants over the course of 1.5 months
• 2 Lookback sessions per participant (Screen recordings of usability testing with narration from participants)
• In collaboration with development team, leveraged the pre-existing test templates technology to develop a template (initially served by query parameter) to allow the MVP to be deployed on production but only available to the subset of users we included in our study
• Constructed the study to allow participants to run a test based on user-generated content that included a set of questions to also gather feedback from test participants (recruited by research participants) in order to evaluate all parts of the MVP UX
• Collated the data from the Lookback study by involving stakeholders and team-members from across the business in initial analysis (comments on Lookback videos)
• Synthesised the research extensively (in collaboration with Senior product designer) in order to identify key findings for next iteration of work (launch to customers)
• Worked with stakeholders (business, technical lead, design) to rank, estimate and prioritise findings from research to scope the final deliverable
• Project managed the final iteration — launch to customers
• Created centralised document repository to summarise features included in various iterations and what will be ready for launch
• Worked closely with support and marketing to identify collateral that needed creation and align on timelines
• Identified required help centre articles and skeletoned them with knowledge from the product team
• Worked directly with illustrator to create marketing imagery for the feature
• Wrote (in collaboration with Product marketer) marketing materials including website copy, blog post, customer announcement via Headway, social media posts
• Organised rewards and launch party for the product team
• Identified key metrics to measure in order to evaluate the project after launch
• Developed templates, role descriptions and resources to be used on various other projects including:
• Definition of technical lead role description
• Product bet template
• Product initiative template
• Research study outline template
• Centralised and collated all research in research repository (Dovetail).

• https://io.usabilityhub.com/introducing-figma-prototype-tasks-with-usabilityhub-8d10b163725e
• https://usabilityhub.com/product/prototype-testing
• https://help.usabilityhub.com/en/articles/5758862-troubleshooting-prototype-tasks
• https://help.usabilityhub.com/en/articles/5758858-how-to-best-set-up-your-prototype-in-figma
• https://help.usabilityhub.com/en/articles/5442240-prototype-task-sections

Results

• Project was delivered on time (although timelines did shift a couple of times during the project due to team members joining and leaving)
• Feature achieved the goal of 20% adoption (as seen below in the dashboard)
• Product team completed the most complex project in the history of the business, successfully shifting the positioning of the product
• Delivered feature parity with competitors (Figma integration, clicks, paths, misclicks, participant test times recorded)
• Delivered feature differentiation with competitors (prototype tests without path goals, results pages that display “common paths” in order to suggest patterns in data to test creators).