Qualitative Inquiry Can Make Quantitative Work More Meaningful: But Which Comes First?

A case study presented for Quant UX Blog

A few weeks back I was invited to guest-author a post on Chris Chapman’s Quant UX Blog. Chris is a highly respected member of the quant UXR community so I was thrilled to be asked. Below is a summary of that content, but please visit the blog here for the full story.

In the data-obsessed world we live in, companies are constantly reaching for numbers as the foundation for improving the performance of their products and services. A commitment to gathering and analyzing user data is a critical component for building an understanding of how, and whether, we’re meeting customer needs. Harnessing this information allows teams to move beyond assumptions, identify pain points, and strategically design solutions that enhance usability, satisfaction, and ultimately, business outcomes. We know in UX research that numbers are often only part of the story.

• Starting with qualitative research before quantitative studies ensures you’re measuring the right things and that metrics accurately reflect real-world behaviors
• A loan servicing company discovered their call center metrics were misleading: agents marked 80%+ of calls as “make a payment” when many were actually customers checking on payment status due to system delays
• This inaccurate data led to wasted investment in improving an already-functional online payment system instead of addressing the actual issues: payment processing delays and poor messaging
• Qualitative research revealed agents had no appropriate category to describe calls where reassuring customers was the primary focus, forcing them to select “make a payment” as the closest option
• Leading with qual research would have let the team identify blind spots, refine data collection instruments, and ensure the quantitative categories matched reality before large-scale implementation
• The decision of which method to use first depends on team confidence in understanding the problem, timing constraints, and the risk of collecting inaccurate data
• Secondary datasets can sometimes provide useful context, but direct qualitative observation often reveals critical insights that metrics alone cannot capture
• Rotating between qual and quant methods creates a powerful research approach, with each method strengthening the other when properly sequenced