Qualitative research at scale
As we head into the final stretch of 2020, the second week of November in New York saw unseasonably warm weather, time spent collectively honoring our nation’s veterans, and Smart Design’s second Virtual Smart Salon, a series of conversations with diverse leaders on today’s most important design-related issues.
The invigorating and inspiring conversation, led once again by our own Richard Whitehall and Leah Cabrera-Fischer, convened three outstanding panelists from varying industries and backgrounds. Together, we comprehensively explored how design teams can effectively combine the depth of qualitative research with the breadth of quantitative research.
Watch the entire session featuring (clockwise from the left) Smart Design hosts Leah Cabrera-Fischer and Richard Whitehall with Ashley Graham, Juliette Melton, and Rachel Lehrer.
Virtual and in-person events
Join us for our upcoming events
Our Panelists
Ashley Graham, a former Design Research Manager at IBM, Ashley currently helps lead user research and service design practices at Spotify as a User Researcher. Together with her team, she works to equip businesses with experiences to build, learn, and grow as they meet the needs of people at work and at play.
Rachel Lehrer, as the Associate Director of Design & Innovation at Airbel Impact Lab—the International Rescue Committee’s Research and Innovation team—Rachel leads global, multi-disciplinary teams to design and test life-changing, scalable solutions for those affected by conflict and disaster.
Juliette Melton, a former lead of Design Research at IDEO, Juliette now spends her days as Managing Director of User Insight & Strategy for NYT Cooking—a division of The New York Times— where she bridges qualitative and quantitative methods to inform, inspire, and shape a new generation of news products.
Here are three key takeaways for what researchers should be focussing on
Aligning on the goal before determining a method
When gathering new insights on a user group, teams often fall into the trap of deciding on research methodologies before aligning on learning objectives. As researchers, it’s our job to debunk these presumptions, and instead refocus efforts on the question, “What do we want to know?” instead of “How are we going to find out?”
“Whenever I work with stakeholders to design a research plan, the method is always the very last question” asserts Juliette Melton, Managing Director of User Insight & Strategy for NYT Cooking. “I have a process of steps I go through just to make sure that we’re all on the same page: First, I want to understand, what do you currently believe? What are your current hypotheses? What does success look like? Who are the stakeholders? What are the things you really want to know? Who do you want to know those from? And then we get to the method.”
“That’s a really important piece of it,” she continues, “not to assume. When someone comes to me and says, ‘We want to do a survey’, I respond, ‘Wait just a second!’ I don’t ever let conversations start with the [research] method, because it’s just not helpful.”
Combining methods to get the full picture
It takes a combination of both qualitative and quantitative data for teams to build a comprehensive understanding of not only what customers do, but also who they are as people and the things that matter to them. Mixing the right methodologies to strategically gather data and marrying the two datasets together will give researchers and designers the individualized, yet full picture required to getting the full picture required to make informed strategic and product decisions?
“I think one of the things that I enjoy about insights at Spotify is the scale, and really being able to see how trends in the world affect our business and user experience,” explains Ashley Graham, User Researcher at Spotify, “And so in terms of quantitative research, it’s really about seeing how big a trend is, where’s that trend showing up? Really having that concrete evidence that we can have confidence in to make business decisions.”
“But qualitative data” she continues, “really helps us understand the whole of the person, or the segment of people that we’re studying, and helps us really tie together both what we understand from their data, and what’s happening in the rest of their lives.”
Adapt researching methodologies to the context
Companies across the world are evolving their research methodologies, with many moving toward more uniform, standardized practices. But a flexible approach is needed to gather qualitative and quantitative datasets, one that caters to both context and culture, when engaging communities who might have limited access to certain technologies.
“Some of the things that we do pretty regularly to just get quick qualitative data.” reflects Rachel Lehrer, Associate Director of Design & Innovation at Airbel Impact Lab. “And in a much less rigorous way. We’re being really scrappy, but we’re also really trying to connect with people on platforms in ways that they’re already comfortable with.”
“We really try to use the right kind of data,” she concludes, “but we change the method of how we collect that data to match what is available for the people we need to be in touch with.”
Conclusion
As designers and researchers, it’s our responsibility to understand the benefits of gathering and analyzing both qualitative and quantitative insights, combining them in advantageous ways to benefit our end users. By pushing the boundaries of how we obtain, distill, and blend this data, we can continue to create novel solutions and experiences that delight people and companies around the globe.
Contributors
Jamie Munger, Strategy director
Yodai Yasunaga, Visual designer
Danny Dang, Visual design intern