Beyond the Surface: Why User Research Must Look Past What People Say and Focus on What They Do

By Smashing Editorial

In the fast-paced world of digital product development, companies routinely claim to possess a deep understanding of their user base. Organizations spend millions on surveys, focus groups, and Net Promoter Scores (NPS) to capture consumer sentiment, convinced they have successfully mapped out the exact desires, needs, and decision-making pathways of their audience. Yet, according to a growing consensus among user experience (UX) researchers and design leaders, these efforts often rely on a dangerous foundation of big assumptions and unverified hunches.

What people say, feel, think, and do are frequently entirely different realities. To build products that genuinely resonate and avoid costly missteps, product teams must learn to look past surface-level feedback. They must decode hidden motivations, identify root causes, and embrace mixed-method research frameworks that span multiple layers of human behavior.

Main Facts: The Illusion of Direct Feedback

At the heart of modern product design lies a fundamental flaw: the belief that asking users direct questions is the most reliable way to uncover what they need. Industry experts and foundational thinkers in UX argue the exact opposite.

Direct questioning is often the least effective way to secure actionable insights. Human beings are notoriously unreliable narrators of their own behaviors. We frequently lack conscious awareness of our true motivations, project our own subjective contexts onto ambiguous questions, and heavily exaggerate our needs. Furthermore, individuals naturally skew toward edge cases, construct unrealistic hypothetical scenarios, and heavily prioritize short-term goals over long-term utility.

For instance, if a user insists in a survey that they desperately require a complex, multi-column comparison table to choose a product, it does not mean they cannot successfully reach their underlying objective through a simpler, more streamlined interface. Relying strictly on verbalized preferences leads to bloated software and misdirected engineering efforts.

Compounding this problem is the linguistic ambiguity inherent in human communication. Words that seem precise often carry vastly different numerical and psychological interpretations from person to person. Studies exploring verbal probability terms reveal that while extreme language maintains some consistency, words like “possible,” “maybe,” “uncertain,” or “likely” trigger a wide spread of subjective interpretations. Listening to words alone is an inherently flawed strategy.

Chronology: The Evolution of Customer Understanding Frameworks

To trace how the industry arrived at this methodological crossroads, it is helpful to look at the progression of UX research philosophies over the past decade:

• The Era of Direct Surveys (Early 2000s): Companies heavily relied on quantitative self-reported metrics like satisfaction surveys and the Net Promoter Score (NPS). The prevailing logic assumed that if a customer stated they liked a product, the design was automatically successful.
• The Shift Toward Empathy and "Think-Aloud" Protocols (2010s): Recognizing that surveys missed emotional nuances, the industry embraced usability testing featuring the "speak-aloud" protocol. Researchers asked users to narrate their thought processes in real-time while completing digital tasks.
• The Realization of Disruption (Recent Years): UX practitioners noticed a critical flaw in the speak-aloud method: forcing users to verbalize thoughts simultaneously while navigating a complex interface altered their natural behavior. Emotions remained hidden behind the cognitive load of speaking.
• The Framework of Four Levels (Present Day): Innovators like Hannah Shamji formalized a comprehensive structure—the Four Levels of Customer Understanding—urging teams to triangulate data across what users say, what they think or feel, what they do, and why they do it. Concurrently, thought leaders began pushing the industry to abandon passive "validation" in favor of rigorous, objective behavioral diagnosis.

Supporting Data: The Four Levels and the Spectrum of Empathy

To achieve an unbiased, realistic view of customer needs, modern research frameworks mandate exploring behavior across four distinct, nested layers:

1. Level 1 — What they say: The outermost layer, captured via interviews, surveys, and support tickets. Highly accessible, but deeply unreliable due to human bias and social desirability.
2. Level 2 — What they think or feel: The internal emotional state. Difficult to extract, requiring specialized tools like the Emotion Wheel to move beyond simplistic labels like "good" or "bad."
3. Level 3 — What they do: The observable reality. Tracking actual user actions, click paths, hover states, and hesitation without external interruption.
4. Level 4 — Why they do it: The innermost core. Uncovering the root motivations, jobs-to-be-done, and fundamental drivers behind the behavior.

+-------------------------------------------------------+
| LEVEL 1: What they say (Surveys, Interviews)          |
|   +-----------------------------------------------+   |
|   | LEVEL 2: What they think or feel (Emotions)   |   |
|   |   +---------------------------------------+   |   |
|   |   | LEVEL 3: What they do (Behavior)      |   |   |
|   |   |   +-------------------------------+   |   |   |
|   |   |   | LEVEL 4: Why they do it       |   |   |   |
|   |   |   | (Root Motivations & Drivers)  |   |   |   |
|   |   |   +-------------------------------+   |   |   |
|   |   +---------------------------------------+   |   |
|   +-----------------------------------------------+   |
+-------------------------------------------------------+

Compounding this structural approach is Sarah Gibbons’ Spectrum of Empathy, which charts the journey from pity and sympathy to true empathy and compassion. While emotional resonance is vital, some UX researchers argue that an obsessive focus on absorbing user emotions can distract from the core objective: solving practical problems. As design critic Alin Buda notes, the work is about addressing user pain and mess, not performing emotional theater.

However, emotional signals remain critical indicators of product success. When observed naturally, user hesitation, micro-expressions of confusion, or aesthetic engagement act as diagnostic telemetry for how well a system serves its audience.

Official Responses and Industry Perspectives

The shift away from traditional, validation-seeking research has sparked significant debate across the UX community.

Industry advocates for behavioral observation emphasize that corporate reliance on "validation testing" is frequently a euphemism for confirmation bias. Too often, teams initiate user testing not to learn the truth, but to secure approval for pre-existing design decisions. UX leaders argue for a complete vocabulary shift—replacing the word "validate" with words like research, investigate, assess, evaluate, examine, and learn.

Furthermore, experts caution against treating metrics like the Net Promoter Score as a silver bullet. Because NPS aggregates complex human sentiment into a single, often volatile number, it fails to diagnose structural usability failures or reveal the underlying mechanics of customer churn.

Instead of deploying expensive, overly complex research operations, industry veterans advocate for lightweight, high-impact visibility tactics. Distilling user testing down into short, raw video clips of genuine user struggles or compiling monthly insight newsletters ensures that entire cross-functional teams—from marketing to engineering—keep authentic user pain points at the forefront of their daily work.

Implications: Moving from Assumptions to Real Research

The implications of adopting a multi-layered, observation-first research model are profound for product-led organizations.

• Reduction of Waste: By replacing expensive, misleading focus groups with direct behavioral tracking and observational research, companies can avoid building features based on edge cases and false customer claims.
• Cross-Functional Alignment: When user struggles are made visible across departments through raw session highlights and diagnostic data, engineering, product, and design teams unite around objective reality rather than executive hunches.
• Trust and Relationship Building: True customer understanding cannot be extracted via drive-by surveys. It requires building sincere, trustworthy relationships where users feel secure enough to expose their real workflows, doubts, and vulnerabilities.

Ultimately, organizations must decide whether they want to build products based on what they hope users want, or what rigorous, triangulated research proves they actually need. Without a commitment to observing actual behavior and asking the right diagnostic questions, everything else remains an expensive, unmitigated assumption.