Creating Space for Surprise in User Research

The most valuable research doesn’t just answer the questions we ask. It reveals the ones we didn’t realise we should be asking.

"Why didn't we think of that?"

In user research, these moments often reveal blind spots in how teams understand customer behaviour, product decisions and service design challenges.

This reaction from a client, after hearing an insight from a user research session we were running, can feel like failure – as if something obvious was missed, or the team's ways of working prevented them from seeing what was right in front of them. But more often, such moments signal the opposite. They show that research has done what it is meant to do: not just confirm what was already known, but reveal what had been overlooked.

The mechanism behind that shift is nearly always surprise.

Why surprise matters in user research

In its simplest terms, surprise is the experience of something unexpected. We tend to think of it as an emotion, but instead it amplifies whatever emotion follows. It captures attention and holds it there, so that what comes next lands harder than it otherwise would: curiosity, discomfort, recognition, urgency.

This is what makes surprise valuable in research. It exposes what had been invisible and makes it hard to unsee. And when it lands in the right room, at the right moment, it shifts not just what design teams know but how they understand the problem. That's the moment that makes someone ask: "Why didn't we think of that?"

How teams design surprise out of research

Surprise is rarely lost through carelessness. More often it is quietly designed out through perfectly reasonable processes: research scripts become tightly controlled, briefs are framed around predefined hypotheses, and timelines and deliverables prioritise clarity and efficiency. All of these are necessary. But together, they create invisible rails that shape what gets asked, what gets noticed, and what gets reported back.

Over time, teams become fluent in their own systems, the jargon feels natural, and workarounds stop being problems and become “just the way things are”. This familiarity creates a kind of learned blindness: we assume we understand the user because we understand the product. But familiarity with a product is not the same as understanding the people who use it.

Within these constraints, certain insights struggle to surface, not because they are complex, but because they sit just outside the frame we have defined.

Creating ‘spaces for surprise’

We cannot plan for surprise directly, but we can create spaces that make it possible. That means deliberately creating conditions where blind spots can be exposed and teams are able to recognise and respond to them.

Some of this can be designed structurally:

• Discovery research that starts from genuine questions leaves more room for the unexpected, whereas research designed only to validate a hypothesis can introduce confirmation bias and limit new insight.
• Interviews benefit from a degree of openness - through open-ended questions and unscripted time that allows participants to go beyond what was anticipated.
• Where possible, stakeholders should have the opportunity to observe research directly, so they encounter the unexpected firsthand rather than through a summary.
• Sometimes, creating distance helps. When researchers are deeply embedded in a product, its assumptions become harder to see. Independence, whether through external researchers or simply through deliberate distance from internal thinking, can make certain observations easier to notice - not because external perspectives are inherently better, but because they are less shaped by what the answer is supposed to be.
• Methods like ethnographic user research extend this further by observing behaviour in context and over time, revealing what people actually do rather than what they say they do. The gap between the two is often where the most meaningful surprises live.

The role of the researcher

But structure only goes so far. Surprise also depends on how research is conducted in practice:

• It requires curiosity – the willingness to follow an answer beyond the script.
• It requires comfort with silence, where participants often say more than they had planned.
• It requires tolerance for ambiguity: the ability to resist explaining something away too quickly.
• Perhaps most importantly, it requires openness to being wrong. When validation becomes the goal, research will tend to confirm what is already suspected. When validation is treated as a possible outcome, rather than the objective, the process remains capable of producing something genuinely unexpected.

Not all research should optimise for surprise

There are moments in a product lifecycle where surprise is not what is needed. For instance, late-stage usability testing is about whether a product works, not what that product should be. At that point, surprise is costly. Similarly, when scaling a proven idea, validation plays an essential role. In practice, generative research benefits from openness and discovery, while evaluative research benefits from clarity, focus and control.

There is also a role for confirmatory research in building trust. When organisations are still developing confidence in research, findings that align with expectations can establish credibility, making more open-ended exploration possible later.

When surprise matters – an example

A well-known example of surprise paying off comes from Airbnb in its early days. The team initially struggled to understand why listings were not being booked. There was no obvious product flaw and no clear signal of what was going wrong.

So instead of analysing the data from the safety of their desks, the team decided to get close: they went to people’s homes, sat in their living rooms, and spoke to hosts who were not receiving bookings. What emerged through closer observation was unexpectedly simple: many of the listing photos were low quality. Hosts were uploading dark, unappealing images taken on basic cameras. Guests weren’t necessarily rejecting the locations; they weren’t trusting what they saw.

This wasn’t a hypothesis the team had set out to validate – it was a detail that sat just outside their frame of attention. Acting on it was equally simple. When the Airbnb team started taking professional photos of listings themselves, bookings increased significantly. The insight was simple, but it changed how the team saw the problem and the direction of the business.

It is a good example of how user research can uncover issues of trust and perception that are not obvious in analytics alone.

From surprise to action

Not everything unexpected is meaningful. The valuable surprises are those that lead to action: they might challenge an assumption, reframe a problem, or alter a decision.

Reporting is a key part of that – it is not enough to state what was unexpected. Good reporting makes clear what changed, why it matters, and what needs to happen next.

But it also requires the right organisational mindset – a willingness to question assumptions, change direction, and let insights change the thinking about the product or service that is being designed.

If research only confirms what teams already believe, it does very little work. Its real value lies in showing product, design and service teams what they have missed — and changing what they do next. Creating space for surprise is how we make that possible.

Want to uncover what your team might be missing?

Want research that uncovers what your team cannot yet see? Talk to User Vision about discovery research, usability testing and insight-led design to turn unexpected findings into better decisions.