Navigating User Research in the Age of AI

You’d be living under a rock if you haven’t noticed how quickly artificial intelligence (AI) technologies are changing the way industries operate. It’s fascinating to see how they’re reshaping everything, isn’t it? And that includes User Experience (UX) design and research.

In particular, generative AI (e.g. ChatGPT, Microsoft Copilot, Gemini) brings unique capabilities with its language and image processing abilities, enhancing work productivity, and novel capabilities in the UX industry. On the other hand, it poses challenges to the process, addressing some issues such as system biases, outcome accountability and ethics.

Considering these two sides of current AI tools, how do you envision them impacting the future of user research? What potential benefits or concerns do you foresee in integrating these tools into your user research process?

As the research team at 55 Minutes, here are some of our learnings that look into the benefits you’ll gain from using such tools in the user research process, particularly: problem exploration, market research, and qualitative data processing, as well as what limitations/challenges you might encounter, and how to mitigate them.

1. Leveraging AI tools to understand a problem space

Picture this: you’re diving into a fresh topic area requested by a client, and it’s a bit like navigating uncharted waters. Maybe it’s a new market region, unfamiliar demographics, or a cutting-edge field you’re not quite an expert in yet. In this case, using AI tools would definitely be beneficial for you to quickly understand and explore a problem space.

Let’s say your client wants to develop a digital solution to support the mental health of secondary school students in Singapore. However, you are not a domain expert, not a Singaporean, and you do not know what would be the major pain points and problems to solve for secondary school students in Singapore.

In this situation, you could tap on a generative AI tool to kickstart your research process:

Gemini’s answer to the prompt: “I’m looking to understand the most common problems that secondary students in Singapore face when it comes to mental health support. What do you know about their struggles?”

After you gain the overall understanding, you can narrow down to scope a specific problem space that might be relevant to your team or your client.

Gemin’s answer to the prompt: “Among the common problems you suggested: 1) Academic pressure, 2) Social issues, 3) Family issues, can you prioritise or help me to select one most critical problem for designers to solve that will significantly impact secondary students in Singapore.”

The above examples show the great potential of using AI tools to understand a problem space, which provides the benefits of:

✅ Helping the team feel more confident about new topics by providing a broad overview as well as some potential problem areas the team could look into

✅ Saving time, by helping us get relevant and useful sources a lot more efficiently than if we were to do it ourselves on Google Search

✅ A good adviser to suggest what design goals the team can set!

However, it’s essential to recognise the inherent limitations 🚫 in the outcome of AI-generated content. Given that, we have ✏️ suggestions on how to attain truly valuable insights in your design process, while you gain the benefits of using AI tools:

🚫 They do not indicate the content source (ref/citation). You’ll only be informed of the summary outcome without providing source references and origins

✏️ You need to prompt to get references. A manual fact-check is required in case they provide a fake reference or citation

🚫 The content generated could be outdated/not accurate depending on their system updates (e.g. current database in ChatGPT 3.5 is updated in 2021). System bias is mainly discussed where the system is built with a database accumulated via limited contexts and demographics

✏️ Verify the outcome with the most recent resources and trends

✏️ Find a domain expert (with real people — at least one or two) in the topic area. If possible, conduct an interview to validate the major problems suggested by AI tools

✏️ Find a relevant target audience to validate the major problems and if possible, conduct an interview to validate the major problems suggested by AI tools

🚫 The problem statements or How Might We statements wouldn’t be specific/it’ll be generic

✏️ Discuss with your team, leveraging the ideas suggested by AI, but specifying the relevant/accurate problem statements/HMWs that the team thinks makes sense to tackle

2. Market/product research using AI

As part of the user research process, we often conduct market research as it provides the insights to help stakeholders make informed business decisions. This would involve digging into the competitive landscape to understand who the current competitors are, their strengths and weaknesses, as well as the needs and pain points of the target audience we should address with our product.

This is often a tedious process of searching for competitors and researching them one by one, from their unique selling points to their target audiences.

Thankfully, this has become a much more efficient process now that we have AI tools on our side!

Let’s dive into an example below, using the same scenario of a client seeking to develop a digital solution to support the mental health of secondary school students in Singapore.

After identifying the major problem areas, such as academic pressure, your team might want to understand the current product landscape in mental health support solutions for students. What major products are available, their pricing, as well as their respective strengths and weaknesses?

Gemini’s answer to the prompt: “What are the digital solutions targeted at supporting the mental health of secondary students in Singapore?”

Gemini’s answer to the prompt: “Could you also analyse the strengths and weaknesses of each product?”

Gemini’s answer to the prompt: “What about the pricing of these products? I would like to compare their prices so as to complete my market research.”

The benefits of using AI in market research is pretty similar to that for exploring problem spaces, but some benefits specific to market research would be how AI tools are able to:

✅ Instantly put together a shortlist of competitors, instead of you having to manually search for competitors to look into, one by one

✅ Gather comprehensive data on competitors, from product features to pricing, all at a speed that manual research can’t match

✅ Providing a second opinion on your own analyses of the strengths and weaknesses of competitor companies, so as to provide another perspective and avoid internal biases

Similarly, the downsides of using AI tools in market research are the same as those for exploring problem spaces.

However, what is most important to take note of when it comes to market research, is the currency of the information given by the AI tool. We would recommend to always do a quick search on your own, so as to:

✏️ Ensure that the competitors shortlisted by the AI tool still exist in the market

✏️ Check if there are any promising, up-and-coming products that were missed out by the AI tool

3. Qualitative data processing using AI

Next, picture that you’ve just wrapped up interviews or focus groups, leaving you with multiple qualitative data to sift through for coding, thematic analysis, and insight generation. The process can take anywhere from a few days to weeks depending on the volume of data.

Fear not, because AI tools are here to lend a helping hand! You can expedite the process, juggling multiple projects, gaining valuable insights to enhance user experiences and design.

Let’s imagine that your team just completed conducting interviews with several secondary students who have had the experience of receiving mental health support before. Now your team is scheduled to conduct focus groups with parents the next day for the same project. Before going to another session of research study, it would be good to quickly summarise the key findings and note down major patterns.

‘Generate a short overview’ feature in Miro AI assistant — it generates a short summary of each student’s interview (for five students).

Miro AI Assistant’s answer to the prompt: “Can you summarise what would be common challenges in receiving support across students? Give me the summary with themes and supporting points, and representative quotes by indicating which student says.”

‘Create a presentation’ feature in Miro AI assistant — it generates a slide with themes and key points summarised by the previous prompts.

The qualitative data synthesis demonstrated by one of the AI products we tested shows that it’s able to provide a quick summary with texts and visual presentation. AI can help by:

✅ Becoming your extra hands. AI tools can assist in generating key findings and identifying common patterns among various data points, particularly when handling numerous interviews

✅ Working like magic! AI can summarise and synthesise the interview data in a matter of seconds

However, attaining truly valuable insights from the qualitative analysing process is not about getting an overview and processing the data quickly.

Below are some of the limitations 🚫 you may encounter while processing the data, with a counter tip or suggestion ✏️ on how to address it:

🚫 The synthesis and summary generated by AI tools are too generic. Accordingly, the results wouldn’t be insightful enough to take away, potentially overlooking crucial details that could provide significant insights

✏️ You’ll need to conduct a thorough review. Use AI for synthesising your data, but make sure to conduct a thorough review to confirm comprehensiveness

🚫 The AI tool is unable to capture micro-nuances and non-verbal cues. This means, it won’t be able to interpret the exact meaning of the feedback

✏️ Include your observational insights in the report as well. Your detailed observations might impact overall findings and insights

🚫 Its qualitative data analysis lacks depth. In qualitative research studies, we pursue uncovering insights, even from a single statement made by a participant, rather than simply identifying common patterns or providing a key summary

✏️ Do not stop at getting a quick summary. Discuss the key findings with your team. What aspects stood out the most and why? Probe into the ‘what’ and ‘why’ to delve deeper into understanding of users’ needs, motivations and pains

Our key learnings

Our testing and experiments with AI tools help us broaden our understanding on the current state of these tools — including the advantages and limitations when integrating them into research tasks. While these tools serve as invaluable assistants, they also shine a light on three distinct human qualities that remain crucial and irreplaceable.

❤️ Building empathy with our users lies at the heart of practice, understanding their needs, motivations and pain points.

True innovation flourishes when we authentically engage with our users, discerning their pain points and needs. This understanding doesn’t solely rely on text; it’s gleaned from facial expressions, observations of the person in their space and spontaneous responses in the interview questions.

⏳ It is about understanding users’ experiences, not about a quick answer which inherently can be false.

Often, true innovation requires investing time in hearing users’ unique stories and journeys from seeking solutions to experiencing products in their context. Observation plays a part here — immerse yourself in users’ experiences, observing their unique behaviours within specific contexts and scenarios. Asking “why” about users’ behaviours further enhances understanding.

🚀 Tap on your team’s creative spirit. Don’t rely on AI for ALL the answers for crafting a unique product value proposition

Remember, AI-generated answers can be accessed by competitors as they draw from the same database. Particularly for product innovators or design thinkers, avoiding conventional solutions that are easily replicated is key. Forster your team’s creative spirit to unearth unique innovative opportunities.

—

Hye Yoon is the UX Researcher Lead at 55 Minutes. She has a Bachelor’s in furniture and spatial design from Seoul, South Korea, and has a Master’s in Helsinki, Finland. She lives in Singapore, observing her surroundings from the lens of a tourist from time to time. She loves nature and goes cycling every weekend in the Northeast region of Singapore.

Lynn is the energetic UX researcher at 55 Minutes, whose curiosity fuels her work and personal life. She believes that research is not just about gathering data; they’re about connecting on a human level to ensure that products truly resonate.