Unlock the Potential of Your Product with Beta Testing

As a business owner, do you think one test (an alpha test) is enough to validate your assumptions about how your customers will find your product?

In our previous article, we talked about alpha tests and what to look out for when you’re conducting one on your own. Let’s now look at beta testing.

The alpha test that we did for our client, Nine Tales, a storytelling app to connect families, was an invaluable process. It helped us find out if the experience of using Nine Tales was even delightful to begin with, spot user experiences that might need improvements, and identify initial bugs in the product — all at the same time.

Conducting an alpha test helped validate the product to a certain extent. However, it’s definitely not 100% validated yet! 😅 Testing it within the team is not enough to validate a product for launch, as we are not Nine Tales’s primary target audience. How do we know if the product would appeal to parents and children (the primary target users)?

Time to test the product with the primary target users?

We designed a beta test to test the product with our target users and see if our improved designs and features would work well in real users’ context.

There are usually two options to run a beta test — a closed beta test and an open beta test:

• A closed beta test is conducted with a small group of selected users. They are usually your primary target users chosen by your team. This means that your product is tested under a controlled environment.

• An open beta test is available to the general public, where the test is open to any user who is willing to test out the product.

We decided to conduct a closed beta test for Nine Tales. This gives us the chance to make further improvements after the test to ensure that the final product is useful, usable and meaningful for our target audience before we make it accessible to the public.

Our learnings from beta testing

Here are three main insights that surprised us:

👉🏻 Insight 1: As children’s reading abilities vary significantly by age, it’s important to provide options to cater to different reading ability levels

During the beta test, we discovered notable differences in reading abilities between a three-year-old and a six-year-old child. For example, the three-year-old was able to read fewer words than the six-year-old. This was brought up by parents as they prioritise their child’s learning development. Our current design did not address the varying reading abilities at different ages. While the voiceover helped children to understand and learn the pronunciation of each word, they might not be able to catch the words as quickly.

This insight prompted us to introduce two design improvements:

• Based on user’s activity, we adjusted the logic of story recommendation on the homepage to match the child’s reading level

This personalised user experience allows parents to browse and easily select stories for their child. Furthermore, should a child be using Nine Tales by themselves, this would ensure that they are browsing the stories best suited to their reading ability.

• We introduced playback speed and text size control features while reading a story

Playback speed enables children to read and learn words and sentences at their own pace. The text size control feature makes reading easier and facilitates more interaction between parents and children. For example, parents are able to point at a word clearly and prompt the child to read that word aloud.

Accessing playback speed and text size controls while reading a story. Source: 55 Minutes.

👉🏻 Insight 2: As children’s attention spans are short, the storytelling process has to be more interactive to keep them engaged in reading

Parents observed that their children were excited to use Nine Tales at the start, but the excitement trailed off shortly after. This got us thinking — how can we sustain the child’s interest in reading and using Nine Tales?

We decided that it would be a good idea to allow children to record a story on their own. Recording a story helps children to comprehend and articulate words they’ve read and listened to. Once a child records their own story, they are able to learn the correct pronunciation of words by listening to the default voice over and comparing it to their own. It also serves as a motivational factor as they are able to listen to their own recorded story after.

Allowing children to record their own story. Source: 55 Minutes.

👉🏻 Insight 3: There was a need to add more interactive elements for children to build connections with family members

When parents were asked to what extent their children had built connections with family members through Nine Tales, the response was slightly below average. They felt that listening to stories recorded by family members did not help build connections.

This insight was crucial, as it implied that users did not experience what Nine Tales aims to achieve — helping children learn meaningful life lessons, build emotional intelligence through stories, and connect with their loved ones.

How can we better connect family members with children when using Nine Tales, while keeping the goal of helping children learn at the same time?

We introduced two new features to help nurture the connection between a child and their loved ones while improving their learning experience:

• Interactive prompts throughout the story

With the goal of deepening and building connections in mind, the prompts will help parents and other family members facilitate conversations with the child while reading the story, allowing for more interaction between them. This is similar to how the reading experience would look like while reading a physical book.

Interactive prompts throughout the story help family members facilitate conversations with the child while reading the story, allowing for more interactions between both parties. Source: 55 Minutes.

• A discussion segment after the story as a self-reflection exercise for children

To keep the goal of helping children learn at the same time, we added a discussion segment at the end of the story that would help children understand and summarise what they’ve just read and reflect on their learning points.

Similar to interactive prompts, the discussion segment at the end of the story helps children to reflect and summarise what they’ve just read. Source: 55 Minutes.

Summary

The insights we gathered from the beta test were very different compared to the alpha test (in a good way). The findings were deeper and more insightful as the testers were real users and our primary target users. There was so much to unpack and to learn from, and to find out how we can better design the product to meet their needs.

This is not to say that the alpha test was a waste of time; both tests are different in their objectives, and we learned so much while conducting both tests. So if you decide to conduct a user test next time, remember to first have your objective(s) in mind. This will help you decide on the type of test(s) to undergo, and the steps to follow.

We hope you enjoyed reading through the entire process of designing Nine Tales! Even though the product did not eventually make the cut to launch, this partnership taught us so much on the ins and outs of researching and designing a product for a group of users we were unfamiliar with.

If you had first stumbled upon this article and would like to read up more on the work we did for Nine Tales, you may follow this sequence:

1. Our branding work for Nine Tales

2. Initial user research

3. Our first app designs for Nine Tales

4. Initial findings we got from conducting an alpha test

5. Our beta test learnings (this article)

All in all, partnering with Nine Tales from the start allowed us to understand their business goals and customer needs more deeply, ultimately enabling us to create a better product together. This is when we do our best work — when we work closely with partners, beginning with a thorough understanding of their needs and objectives. With a shared goal in mind, we put forth our very best effort and brought all of our strengths to the table, aiming for nothing less than the best possible outcome.

Whether you are a startup with a visionary idea or an enterprise looking for a fresh take, we’d love to work with you to bring your vision to life! Feel free to reach out to us at hello@55mins.com 😉

This article is co-written by Elaine and Hye Yoon.

—

Elaine does UI/UX design at 55 Minutes. Inspired by architecture, art, fashion, nature, photography, and people, Elaine’s approach to design is to tap into what makes us human — our emotions. She believes good design involves understanding who she’s designing for, and how design makes them feel. For her to be pixel-perfect in her work, she needs a good breakfast. Coffee is a must too.

Hye Yoon is a UX researcher at 55 Minutes. She has a Bachelor’s in furniture and spatial design from Seoul, South Korea, and has a Master’s in Helsinki, Finland. Currently, she lives in Singapore, observing her surroundings from the lens of a tourist from time to time. She loves nature and goes cycling every weekend in the Northeast region of Singapore.