3. Preparing for Release: Test Tracks

Written by Fuad Kamal

In the previous chapter, you created an Alpha release through the Closed testing release track. In this chapter, you’ll dive deeper into the app publishing process and discover various ways to test and distribute your app within your team. You’ll also go through the Alpha and Beta testing process to make sure your app is ready to share with the world. In the next chapter, you’ll learn how to make your app go live to the general public and the issues around production releases.

Note: You don’t have to release your app through Alpha and Beta channels. You’re free to take your initial release straight to production!

Play Console overview

Developing and publishing your app is just the beginning. In 2020, Google launched a completely revised Play Console for better managing and deploying apps. The new Play Console introduced a major UX facelift, improved team management, a modern interface using Google Material design language and a new menu structure.

As you saw briefly in the previous chapter, the Play Console is grouped into several main areas by function: Release, Grow, Quality, Monetize and Policy. In this chapter, you’ll get an overview of the entire Play Console. You’ll deep dive into some of these sections in this and the next chapter.

Google Play Console gives you the power to scale with access to billions of users. It also gives you the tools to acquire them, like the ability to test the text and graphics of your store listing to drive more installs.

The Play Console gives you the power to iterate. Testing tools and pre-launch reports help you identify issues before they affect your users so you can release with confidence. It lets your users beta test new features and then roll them out in stages to ensure a safe launch.

The Play Console lets you make data-driven decisions with access to insights about your app, such as metrics, and what your users are saying through reviews and ratings. With the Play Console, you can monitor your app’s policy status so you can stay compliant.

Grow

Grow contains everything you need to supercharge your user acquisitions, from pre-registration to store listing experiments. It also features acquisition reporting.

Quality

Google designed the Quality section for QAs and team leads. It contains all the signals and features that tell you how your app performs in the wild, including both its technical performance with Android vitals and how people feel about it through ratings and reviews.

Monetize

Monetize brings together Play Console setup, reporting and optimization tools to help you grow your business through in-app purchases and subscriptions.

Policy

Policy helps you release confidently on Google Play. At the time of this writing, the author experienced multiple Play Console rejections on internal releases due to Google Policy compliance issues. The reasons you provide for various app permissions matter, and Google has become increasingly strict about Policy reviews. For more information about Google Policy, see https://play.google.com/console/about/policystatus/.

Release

Designed with Release managers in mind, Release collects all Google’s tools to upload and distribute you apps, review your testing and production tracks and check how your latest updates perform.

Releases overview

The Releases overview shows you an up-to-date snapshot of your testing and production tracks to quickly see which versions are being tested, by how many users and in which countries.

Private vs public apps

There are two main types of mobile apps, regardless of the platform. The most common type is public apps, which are generally available to everyone. Developers typically distribute these through an app store, such as Google Play or Amazon Appstore.

The other type of apps are private apps. These are also known as enterprise apps or Enterprise Mobility Managed, or EMM, apps. While this book mainly focuses on public apps, everything you learn here is also applicable to EMM.

You can use the Play Console to manage both your EMM and public apps. For more information on EMM in the Android ecosystem, as well as the Google Play EMM API, refer to the Android Enterprise documentation page for integration with your CI/CD toolchain found here: https://developers.google.com/android/work/overview. You’ll learn more about continuous integration and delivery in section three of this book.

Test tracks

Before you publish your app to the world, it’s important to test it with a small group of users. Testing not only catches bugs but also helps with scaling and rolling out updates faster.

The key is to distribute a test version of your app to a small group of trusted users. This group could be your wider team, product managers, a QA team or whoever can give you early feedback.

You could directly email or otherwise share an APK with the testers, but that can be cumbersome and places an undue burden on them. Also, you lose the benefits of using an app bundle: APKs need to be universal for all the supported devices. Therefore, the installation package is much larger than if the users receive only the files they need for their particular installation through an app bundle on the Play Console.

A track is a way to define a group of users to whom you can publish your app. Google provides three different tracks: Internal, Closed and Open. Additionally, Google designates two release types: Alpha and Beta. The Closed track only has the Alpha release type while the Open test track only has the Beta release type.

The Alpha and Beta release types provide an excellent way to get feedback to make sure your final release is as polished and stable as possible. The only requirement for testers is a physical Android device and a @gmail, G Suite or Google account.

Alpha and Beta release types are designed for testing at scale, with potentially millions of users. Because of this, they’re also a bit more involved and tedious to deploy.

For fast, iterative testing Google introduced the Internal test track.

Internal test track

The Internal test track is for direct distribution to your quality assurance, or QA, team. Deployment is much faster than the other tracks. The first time you deploy, it may take up to 48 hours. However, after that your builds will be available to your testers within a few minutes.

With the Internal test track, you can have up to 100 testers, which you organize by email address. There are no country limitations for distribution. While they can install Paid apps for free, testers still must pay for in-app purchases unless you add them to a license testers list. Any device exclusion rules you define for your app won’t apply to internal testers.

When Google first introduced the Internal test track, they said it wouldn’t be subject to Play Policy or security reviews, but later they reversed their stand on this. Beware as this might change in the future since the goal behind internal testing was to make it fast and easy to deploy to your most trusted testers.

Testers don’t need a special app or permissions to install your test builds with the internal test track. All they need is a link, which you’ll find in the Play Console’s App bundle explorer.

The updates for internal testers are also different than they are for Alpha and Beta testers. Users can be either alpha/ beta testers or internal testers: They must opt-out of one to join the other.

The Play Console publisher API and many Play Console publisher plug-ins support the Internal test track. That means you can easily automate publishing to the internal test track from your existing toolchains and workflows.

The internal test track is great for quick iteration during development, catching bugs early, getting fast feedback from your team and testing Play integration. Because your app is still published through Play, it supports all the Play features: LVL testing (https://developer.android.com/google/play/licensing/setting-up), in-app purchases, the Android app bundle and dynamic developer features.

Exploring the internal test track

In the previous chapter, you deployed a build of your app to the Play Console using an Internal Test Track. Go to the Play Console now and select All apps from the menu. You’ll see a list of apps under a title heading, which includes the number of apps you currently have in the Play Console.

You may have only one app at this time, but as the number of apps in your Play Console grows, you can highlight the most important ones by pinning them. Click the pin icon in your app listing to add it to the Pinned apps list:

When you have multiple pinned apps, you can drag and drop them to arrange their order. You can see summary KPIs for pinned apps without going into the dashboard for each app.

Click View app to go to the dashboard for your app. Then click App bundle explorer from the Play Console menu on the left to go into the App bundle explorer.

In the App bundle explorer, you can see all the deployment details for your current app bundle. Spend some time exploring and become familiar with all the information here. Some things of particular note are:

• The permissions that your app lists in its manifest.
• Which API levels your app is compatible with.
• The number of Android devices, specifically which devices your app should work on.
• How many active releases you have.
• Which types of releases you have.

You can also see the size of the download for your users to install the app.

Next, click the Downloads tab in the App bundle explorer. Here you’ll find the link you can share with your testers to install the internal test build.

Installing an internal test track build

So you’ve deployed your build to the internal test track, and you sent out the shareable link to your testers. You check with your testers to see how it’s going, only to find out they weren’t able to install it.

Some of them say they got a 404 Error when they clicked the link. Others say when they click the link on their device, they get a popup message that says Internal app sharing is turned off…. What have you done wrong?

Actually, you haven’t done anything wrong. It turns out there are some extra steps your testers need to take to install builds from the internal test track.

First, if you restricted the testers to only people on your selected email lists, they must sign in to their Google account with the respective email address on the device on which they are trying to install the build. If they aren’t signed in with the correct email address, they may get a 404 server error on their web browser when they tap the install link.

Alternatively, you can open up testing to anyone who has the link. You can check this in the Play Console under Release -> Setup -> Internal app sharing.

Second, the testers need to perform a bit of black magic, similar to what developers do before they can debug on a device. That’s why they got the popup message.

In the google play app, the tester should click the menu in the navigation drawer, then Settings.

Then they should Enable Developer Options. This is similar to enabling Developer Options within the Android Settings app. Tap seven times on the Play Store Version.

Once they get the You are now a developer! prompt, they’ll see the Internal App Sharing option pop up on their device. Enabling internal app sharing will display a warning regarding the internal test nature of the apps they can now download.

Once they’ve enabled internal app sharing they can click the download link again and see your internal test build available to install.

Closed test track

Once you have your app tested internally with up to 100 users on the internal test track, you’re ready to have your app tested by real-world users. Typically, however, you’re not yet ready to open up to everyone. For example, you might be looking for trusted feedback on a confidential set of features.

Closed testing lets you scale well with various testing scenarios. For example, what if you would like to test your app with a track for each feature branch? Or what if you would like to create a new track for each new milestone?

With Closed testing, you can create multiple closed tracks. You can have one production track, one open testing track and any number of closed testing tracks, plus an internal test track.

You can choose any name for the test track, and that same name is used in the Play Console publisher API. This is great for user studies, country-specific variants, feature branches and testing in incrementally larger groups. These are a few examples of the many ways you can use closed testing tracks.

In the Closed test track, there are only Alpha builds: There is no Beta. Beta is only for the Open test track, which you’ll learn more about in the next chapter. You can promote your existing Internal test track build to an Alpha build in the Closed test track or upload a new app bundle to the Closed test track.

You’ll try the former approach next.

Promote your internal build

In the Google Play Console menu, under Release, click Testing -> Internal Testing. Under the Releases tab, click Promote Release -> Closed Testing -> Alpha.

On the next screen, click Review release.

Note the warning and error messages displayed, and that the Start rollout to Alpha button is grayed out.

Since you’re creating a new test track, you need to select the users who will test this track and the countries or regions where it’ll be available. Remember that while tests for the Internal test track can be in any region, Closed and Open test tracks testers must be from the regions you specify.

If you click Edit release to add these details, you go back to the previous screen where you can only edit the release name, the release notes and upload new app bundles. There’s no obvious way on this screen to add the countries or testers. Hopefully, this is a UX oversight Android will fix in a future update to the Play Console! But for now, take a look at where to update this information.

In the Play Console menu, click Closed testing. Note under Active tracks there is a heading for Alpha that says There is no release on this track. To the right of that, click Manage track.

Here, at last, you’ll find the Countries/regions tab and Testers tab, where you can edit this information.

Click each tab and update those sections for your Alpha build. If you want to add all countries and regions click the check box in the first row titled Country/region.

When you finish adding testers and countries, click Save changes at the bottom of the Testers tab. Note that you’ll also find the links you can share with your testers to join the test group and download the builds at the bottom of the tab.

Again, unlike the Internal test track, you deploy your Closed and Open tracks through Google Play. However, with the Closed track, Alpha builds are only visible to people in your test group, while with the Open track, Beta builds are visible to the general public.

On the same page, click the Releases tab. Then click Edit for your Alpha release.

This brings you back to the page title Create closed testing release. Click Review release again. This time, you get rid of the warnings and errors and enable the Start rollout to Alpha button. Click it and you’ll see a confirmation dialogue:

Click Rollout to launch your first Alpha release.

Congratulations! Your release now appears under the list of Alpha releases in the Closed track. Note that it may appear listed as under review until Google completes the app review for your release.

If you go back to the Internal track on the Play Console, you’ll see your Internal release is also still available. It might be exactly the same build as the Alpha release, but the two are treated as separate and distinct by way of the test tracks.

Create multiple closed tracks

The process to create additional closed tracks is quite similar.

1. Select Create track.
2. Select whether you want to manage testers by email or Google Groups.
3. Copy the Opt-in URL to share with your testers.
4. Next to Feedback channel, add an email address or URL to collect feedback from testers. Testers will see your app’s feedback channel on your tester opt-in page.

Country targeting for testing tracks

With country targeting, you can control the availability for each track individually. You can use this for both open and closed testing. It also lets you run a test in a different set of countries than those in which your production app is live.

Key points

Whew! You’ve covered quite a bit in this chapter! Here’s a quick recap of some of the salient points:

• You can use Play Console to manage and deploy both enterprise and public-facing apps.
• Play Console has several sections to help you through all the stages of app deployment, from testing, release and monitoring, to debugging issues, monetization, license management and Google Policy compliance.
• You can have multiple closed tests, a single open test and a single internal test, concurrently.
• Internal testers can’t access open or closed tests at the same time. They must opt-out of one to access the other.

Where to go from here?

For more information about closed testing, refer to the Play Store documentation: https://play.google.com/console/about/closed-testing/. In the next chapter, you’ll learn more about open testing, deployment strategies for teams of all sizes and automated device testing.