How to Validate Applications Using Test Base for Microsoft 365

Test Base for Microsoft 365 is a platform which allows you to script and automate your application compatibility testing. Hosted in Azure, and based on Azure virtual machines, this service allows the easy onboarding of applications through a dedicated area of the Azure portal.

To validate apps in Test Base, you should:

1. Sign up for the Test Base service in the Azure management portal.
2. Create a new Azure Resource Group and specify a Test Base account.
3. Zip and upload the application dependencies, and then upload them to Azure to create a new test package.
4. Specify the scripts you will use to perform an Out of Box test.
5. Launch the tests by completing the wizard.
6. Review the test results.

Test Base for Microsoft 365 in public preview

Test Base for Microsoft 365 was originally released as the “Update Staging Lab” in 2020. At Ignite that year, it was renamed to Test Base for Microsoft 365 and given a fresh set of features courtesy of direct feedback from IT admins.

Initially the platform was intended and available only to critical software vendors of antivirus, VPN and security apps, allowing them to gain vital insights into how their applications will perform on multiple versions of Windows. Since July 2021, Test Base for Microsoft 365 has been in public preview, with access to the platform available to all Azure customers via a simple service creation page.

Getting started is easy

It’s incredibly easy to get started with an account for Test Base for Microsoft 365. It can be accessed by searching for the service from the Azure Portal search bar.

 

Once found, the process for setting up an account is also trivial. Simply pick or create your Azure Resource Group, and specify a new Test Base for Microsoft 365 account – this is a free text field.

 

Create a Test Package

Microsoft has provided package guidelines for test binaries, dependencies, and scripts.

Package

The package binaries, dependencies, and scripts should be zipped before being uploaded.

Tasks

For all apps, an Out of Box Test is required, which has 4 core scripts

In my example, installation of VLC, the following files are used to generate a simple out of box test for VLC Media Player.
install.ps1:
Start-Process -FilePath "vlc-3.0.16-win64.exe" -ArgumentList "/S"

Launch.ps1:
Start-Process -FilePath "C:Program FilesVideoLANVLCvlc.exe"

Close.ps1
stop-process -Name VLC -Force

Uninstall.ps1
Start-Process -FilePath "C:Program FilesVideoLANVLCuninstall.exe" -ArgumentList "/S"

Once uploaded, it’s required to specify which scripts perform specific actions.

Launching the process

As soon as the files are in place, and the wizard is complete, the platform starts to validate the package and script. This is not a preparation or validation phase – all tests are performed during this stage, meaning it can take a very long time to see any results.

Reviewing results

The results of each app test are very detailed. The overall Test Summary screen gives a great overview of how each applciation performed, and more information can be gathered by clicking See Details.

Within the detailed results screen, it’s possible to see the outcome of each individual test, and any specific exit codes that were produced during execution of the scripts.

Advanced testing with functional tests

Whilst the default Out of Box Test is required for all apps being tested, Microsoft recently added the capability to add specific Functional Test scripts to Test Base for Microsoft 365.

Functional Tests allow customers, ISVs, and Vendors complete specific use-case driven functional tests that will verify additional elements of an application.

Visibility from Microsoft Endpoint Manager

Since October 2021, Test Base for Microsoft 365 has been offered as a suggested “next step” for admins creating Win32 apps in the Microsoft Endpoint Manager Admin Center.

Add Test Base for Microsoft 365 to your workflow

As Win32 apps typically require some scripting to install, it would only be a small amount of additional effort to add Test Base for Microsoft 365 to your app creation workflow, giving both current and future insights into application installation, launch, and uninstall issues that may occur.

Test Base Pricing

Whilst Test Base for Microsoft 365 is in public preview, no charges are being made for using the service. Once the service becomes generally available, charges will be applied for the use of the service.

Pricing information is not publicly released until the service is generally available.