Mobile App Testing Best Practices for 5 Star Launch
As we start the new year, many customers are deploying new systems into production to hit strategic goals. We here at VCG helped one of our customers release a mobile app on both the iOS and Android platforms in December. That piqued my interest to think about and talk about what it takes to successfully and efficiently deploy a mobile app into production.
So what does a product manager think when she’s in the quality assurance phase of testing her product and she’s ready to move it into production? What planning tasks for mobile app development are going to help her be successful?
One of the most essential skills of management is anticipating — thinking about what’s going to happen next. Wherever you currently are in a project, it’s important to think about what to do next.
Mobile App Testing Best Practices
Here are some tips to keep in mind regarding mobile app testing best practices as you get ready to deploy your mobile app into production.
Alpha Testing
The app should be tested — unit tested and QA tested — before you think about alpha testing. Alpha testing is really testing the app with a small number of trusted users. Typically, it would be a group of 2 to 10 customers.
The main goal and objective is to confirm that the app can consistently work well, with no fatal errors. You should expect your alpha testing group to have a stabilized and sustainable user experience before deploying it to a larger group of users.
There is always an internal conflict between perfecting the product and getting it to market as soon as possible. It’s important during the alpha testing phase to look at product release as an iterative process.
During alpha testing, it’s important to outline acceptance criteria for your product to be released and promoted. Usually, the most important factor is, “Is the system stable and without fatal errors?”
Once you have a stable system that can work for 1, 2, and 10 users consistently, it’s important to continue to promote the app through the development lifecycle. You can then continue testing through the beta testing process and continue the iterative approach to product development and product release.
Some Recommendations during Alpha Testing
During alpha testing, remember to employ an internal code repository tool that also has an issue tracking system. A few good choices are GitLab, GitHub, and BitBucket, for example. These tools will help support an iterative product development model.
Consider giving something to your alpha and beta testing customers as a reward for being part of the testing process. Freebies go a long way towards building a relationship with your customers. It also sets the expectation you want, which is that this is an early release of the app, and there may be issues or defects to capture and fix. You include them, in effect, on your team.
When you’re in the alpha testing phase you want to use an internal deployment mechanism so you’re not relying on the stores to deploy your app. Sometimes the iOS store can take up to a week for approval, and when you’re in alpha testing mode you don’t want to have to wait a week between iterations. So use a service-like installer. You should be in rapid feedback and development cycles to get the best app out through this process.
Beta Testing
The whole objective of the beta testing phase is to test, test, and test again. Developers and QA specialists are going to be able to test the app, but only so much. They may not be experts in the industry and may only know how the app is supposed to behave based on an outline or a noted specification document. Getting actual users and actual user data is a key component to the success of beta testing.
Beta testing should still be conducted with a trusted user group, but it’s a wider audience. Now you’ll be looking for 10 to 100 users to use your app, which you have now categorized as a stable build. That’s what alpha testing phases are for.
Now, in beta, your goal is to make sure you have no performance issues with your app. Beta testing involves evaluating the app’s usability and getting feedback. Beta testing also includes getting new feature requests and prioritizing those requests for future releases. But remember: the core idea here is to test, test, test.
When it comes to getting user feedback, there are several tools designed for that purpose. It’s important to have a virtuous feedback cycle, where you are getting user feedback and having that feedback drive your next iterations of features.
A couple of tools to consider:
- Jira Service Desk: Atlassian, a company which has become ubiquitous within the collaboration software space, has a product called Service Desk. Service Desk will allow you to implement a service tracking system for product issues and customer service.The system allows you to report an issue, get IT help, and facilitate all the communications that take place, from detecting an issue to isolating it to resolving it and finally releasing the fix. The two main functions that this tool provides are to:
- Allow you to automate repetitive tasks so your customer service agents can focus on the hard work of solving problems; and
- Automate the reporting processes to help improve performance by catching bottlenecks before they happen.
- Surveys by SurveyMonkey: Surveys are an important tool in gaining a good picture into the user experience. Surveys allow customers to provide direct feedback about products. SurveyMonkey is a free and premium tool that allows you to build custom surveys and send them out to your user base. Surveys allow you to make better decisions, and SurveyMonkey is considered the #1 survey tool in the space.
Some Recommendations during Beta Testing
Once you’ve started the beta testing phase, it’s time to use the store to deploy your app. A week before beta testing, you should go through the verification and approval process between Android and iOS stores. Each of them have different requirements and will take different amounts of time, so it will be important for you to manage this.
Again, sometimes the iTunes store takes a little longer so factor that into your project schedule. Using stores for deployment can be a bit complicated, so it’s important to iron out these issues during beta testing.
Make sure the updates are working properly. Consider whether you want to force updates as they come along or whether you want to leave it up to the users to update their app. Consider any data ramifications or version control issues that might arise.
Using the stores, you’re now ready to complete the beta testing phase. Once you’re done with beta testing and ready to go to full production, there are a few things you want to keep in mind.
Support Systems and Tools
How Long Should Testing Take?
There is no testing cycle length that fits all projects. Many factors come into place in deciding how long a product texting cycle should last. For instance, is it a mission-critical system? Are lives or important resources at stake? Or is this a game people play on their mobile phones?
Generally speaking, if you have an enterprise mobile app that took 4 to 12 months to develop and you’re looking to deploy a version 1.0, then your alpha testing cycle is measured in days and weeks and your beta testing cycle is measured in weeks.
So, for example, an average alpha testing cycle would be 1 to 3 weeks and a beta testing cycle may be something like 2 to 6 weeks. This is a very general statement, and it will depend on your particular situation. The important factors are that your product is stable, the features meet the designed user experience, and the users’ needs are well considered and reflected in the app’s functionality.
Ready for Primetime
Now that you’ve followed mobile app testing best practices, you’re ready for prime time — time to make your app available to the estimated 4.4 billion mobile users around the world. The goal is to start getting any production support tools in place.
You’ll want to be able to monitor the user data and the application. You’ll also want to set up any alerts to make sure your web services are up and running.
You should have all tools in place so that you can be proactive in the management of your app and not reactive to customer complaints. Being able to measure, analyze, and continuously improve is going to be the key factor to having a successful mobile app in production.
Good luck!
Jake Ryan
Jake Ryan is founder and president of the Venice Consulting Group which is an interactive web portal consulting and development company specializing in user experience and process optimization with a big focus on Nearshoring. Jake leads the firm with sales, outsourced marketing, consulting services and operations. He is experienced in delivering strategic technical consulting as well as product implementation to his clients.
Originally published at blog.veniceconsulting.com on February 12, 2016.
