3 unexpected things we learned about Beta Testing 🤔

THANKS 👏: I appreciate the many claps most my readers leave because it takes me a long time to write these free articles.

The product roadmap was clear, we needed 100 users by summer.

6 months in — Onboarding starts:

After 6 months of developing the NetZero Open Banking app which lets people track the carbon footprint of their purchases, we were ready for launch. Our aim is simple:

Instead of building a “good app”, we want to make a “great app” for just 100 people.

This way we could take in all their feedback into new versions and make the product better. But of course, every launch has its unexpected learnings...

My reaction after the first 10 users — “Hmm, never seen that before”

Here’s what we learned in our first Beta Testing process so far:

1. ) Onboard slowly (in batches)

We found onboarding 10–15 people every few days was optimal to gather feedback and make sure new users suggested new feedback that the others did not.

This way our users were able to give new knowledge instead of onboarding everyone at the same time. Furthermore, it gave us so many more product iterations to test, improve and launch every few days.

ℹ️ Top Tip: Make sure to put everyone’s feedback in a document and record what device and settings they used if they experienced any errors (this will save you a lot of painful time in the future).

2. ) Message your users personally — and followup

We had tried gathering people’s thoughts by giving them surveys and on the phone before in previous projects– but found this wasn’t an engaging or personal way to keep a long term conversations with our users.

So we instead decided to maintain informal communication with our users through LinkedIn and Messenger. We found this was more personal and let our users feel more comfortable to share their thoughts with us and maintain us in the loop in the long term.

We don’t want to just have feedback, but long-lasting connections with our users.

3.) Beta testing is the start, not the finish

We foolishly thought that everything was pristine before Beta.

Only when 100 different people with different devices, different accounts, different internet speeds are using your app do you truly understand its flaws. And that’s great! More to improve :).

Accepting that our app is always a work in progress helps us set up our cycle of improving, launching, and testing every few days with new users.

Thanks a lot for reading this article, these have been our Beta Testing lessons so far and we are super grateful all our Beta Users and their awesome feedback. We will keep you updated with more lessons we learn in the process! 🚀

Please share your thoughts on what you think makes a successful Beta or prototyping launch. 😃