How Alpha Testing Validates Your Product Design

There are a ton of tests to be completed in a developmental process when you are launching a product. You may have heard of a usability test, end-to-end testing, or a beta test, but have you heard of an alpha test?

What is an alpha test, and why is it important?

Alpha testing is one of many user acceptance tests (UATs). As the word “alpha”suggests, it is the first type of acceptance test that’s performed. It is carried out to validate key functionalities and/or features of a product, which is tested out by internal staff, including stakeholders.

More often than not, most teams skip the alpha test. Some clients may look at it as an unnecessary additional step to the whole developmental process, which may potentially cause a delay in the launch of a product. You may also ask, why should we run a test ourselves when we should get feedback from real users instead? They are ultimately the ones using the product - isn’t it more important to know how and what they think?

These points and concerns are valid, however, by having your team and stakeholders test it out yourselves, you get to experience first-hand how key functionalities and/or features work and gain empathy for your users. An alpha test also helps to:

• Save initial costs and time of recruiting participants to test your product

• Spot flaws in user flow and user interfaces early, allowing the design team to improve on them before moving on to run a beta test

• Allow stakeholders to identify major green and/or red flags of any features and functions in the product early on before any time and cost is wasted (on recruiting participants)

  
  

An alpha test not only validates the design plan; it also surfaces features that need improvement so the team can confidently build the product and not waste time working on features that will neither serve nor delight the users.

When we were designing an initial prototype for Nine Tales, a startup focused on helping children build emotional intelligence through storytelling, the idea of conducting an alpha test came up when we had some doubts about certain key features and functionalities of the product.

What better way to clear those uncertainties than to conduct an alpha test?

3 Learnings from Conducting an Alpha Test

After conducting an alpha test for Nine Tales, here’s what we think you should look out for when running a test of your own:

1. Set your objectives

Before running through this idea of an alpha test with your stakeholders, it’s important to justify conducting the test by setting objectives.

What are you trying to find out?

What key functions and/or features are you specifically testing?

Without setting objectives, running the test would be pointless witht no premise to fall back on.

With Nine Tales, we questioned if certain key features and functionalities would be easy to use and understand:

• Would assigning characters in stories be enjoyable and easy to use?

• How would the different recording flows (assigning of characters, round-robin, and recording on your own) work on different stories?

• Would recording a story be intuitive and enjoyable?

• Are there any initial bugs and issues we should be aware of?

  
  

These objectives help set a premise and at the same time, help determine your next steps in crafting the test.

2. Determine the test plan, time, and resources needed

A key to a successful alpha test (or any test for that matter) is planning. With a detailed plan, the test has a higher probability of running smoothly and efficiently. It also reduces risks and might help avoid potential issues that might arise.

Two factors to take into consideration while crafting a test plan are time and resources needed. You might ask:

• How long will the whole test take?

• With the set of features and functions to test, how are we going to execute the test?

• How many testers will be needed?

• How should we gather our feedback/insights? (e.g. surveys, interviews)

  
  

How we split the tests, roles, and responsibilities for Nine Tales’s alpha test. Actual names of testers are hidden due to confidentiality.

As we wanted to test how family units work together in real-time to record a story on Nine Tales, we designed the alpha test to be conducted over a week. Every person in the team, including stakeholders, was involved. All of us were split into different family units, with each family unit testing two stories. There were three different tests, based on the objectives we had set. The one-week duration allowed us to record stories at our own pace, resembling the reality of how busy each family member might be on a daily basis.

3. Prepare for possible outcomes

As a client who has agreed to an alpha test of your product, it’s important that you are prepared for possible outcomes.You might expect the test to be completed in a week when it might take two weeks or more (depending on what your objectives are), or you might expect to move to beta testing immediately after. Maybe some amount of time will be needed to improve on the designs before conducting another test. These assumptions are valid from a business standpoint, so being prepared for all possible outcomes goes a long way. Should insights gathered be unpleasant, the advanced preparation would allow everyone to expect the necessary steps that will follow.

Key findings from Nine Tales’s alpha test

Fortunately and thankfully, Nine Tales was immediately on board with the idea of an alpha test. They valued the benefits that came with it and had already expected several issues to arise from the test, which both of our teams sat down together to resolve. In general, here are our summarised findings from the alpha test:

• 100% agreed that assigning characters was fun!

• 67% of the testers found it easy to record for the first time, and 100% agreed that it gets easier the second time onwards.

• 91% of the testers found the process of recording stories enjoyable.

  
  

We also tweaked and added some features to elevate the whole experience of using Nine Tales. Below are a few graphics on some improvements we made:

Improvement 1: Since the feature of assigning characters was a hit, we decided to make the assignments viewable to all family members.

Improvement 2: Introduce “Recording in Progress” section on Home Page.

Improvement 3: Introduce a step-by-step tutorial for users recording a story for the first time.

I hope our learnings convinced you how worthwhile it is to conduct an alpha test. It is the first step to ensure your product meets your business requirements and functions, and how you can redefine it to better serve your customers. It is often overlooked, but it is as important as other UATs because of the many benefits that come with it.

—

Elaine does UI/UX design at 55 Minutes. Inspired by architecture, art, fashion, nature, photography, and people, Elaine’s approach to design is to tap into what makes us human — our emotions. She believes good design involves understanding who she’s designing for, and how design makes them feel. For her to be pixel-perfect in her work, she needs a good breakfast. Coffee is a must too.