MasterClass: User Research for a New Product Launch
Overview
As employee number three I helped create an entirely new type of online education platform, starting with solid user research.
The Challenge
Conduct brand assessment and user research on the first three classes to finalize pricing, validate the class content, and set a baseline of user responses for future growth.
Team
Starting at MasterClass as Experience Manager while the company was in stealth mode, I validated the product in a series of research tasks before launch. I worked independently under the direction of the cofounders and Art Director.
Audience
Life-long learners and subject matter enthusiasts looking to learn from luminaries in their fields such as Serena Williams, Dustin Hoffman, and James Patterson.
Constraints
I had only a few weeks to get feedback on the homepage, class landing pages, and ten chapters from each class. Working with celebrities and exclusive launch day promotions meant we needed to keep this quiet before the release date.
Research Process
I started with usertesting.com for feedback on the homepage and class landing pages and used Craigslist to set up in-person or Skype interviews for the class content. Both types of testing utilized NDAs for privacy and compensation, though users didn’t know what kind of product they would be using.
Setting the demographics for the usertesting.com reviews to capture a broad audience, I focused the questions on perception of the brand and class value. This research confirmed some of our assumptions and led to copy refinement for clarity. The data I collected enabled the content team to make final edits to the class videos and for the cofounders to establish the final pricing.
The in-person interviews recruited people interested in the subject matter. For James Patterson Teaches Writing, I spoke with five of his fans and five aspiring writers. Each meeting lasted between one to two hours, starting with a series of questions about their knowledge and interest in the subject. I took notes based on their reactions to each chapter, evaluating their response to the video. I closed with inquiries related to pricing, using the Van Westendorp model.
Research Questions
Knowledge & Interest Evaluation Questions
Do you read crime or mystery novels?
Tell me about your experience, if any, with online education.
On a scale of 1 to 5, how interested would you be in purchasing an online course on writing taught by James Patterson?
Viewing Response Questions
What did you like? What was the best part?
What did you not like? What was the worst part?
What did you learn (if anything) from this chapter?
Was anything confusing about this chapter?
Van Westendorp Pricing Questions
At what price would you think the class is a bargain – a great buy for the money?
At what price would you begin to think the class is getting expensive, but you still might consider it?
At what price would you begin to think the class is too expensive to consider?
At what price would you begin to think the class is so inexpensive that you would question the quality and not consider it?
Retrospective
The research was most valuable when it came to pricing, as the class purchase price still reflects the one I helped establish. Most of the feedback from usertesting.com was flat, coming from jaded “professional testers” who are used to giving product feedback. A different approach would be first researching new testing options on Product Hunt. The in-person interviews were instrumental in helping us make necessary edits and solidify brand positioning for professionals and enthusiasts.