Economic Impact Study Says Outlook Mobile is Good. How Surprising!

What can we make of the Forrester Total Economic Impact™ study of Outlook mobile in the enterprise released on February 26?

Microsoft trumpets the financial analysis showing that organizations which deployed Outlook mobile achieved cost savings, increased user productivity, and better security. Who wouldn’t like to gain benefits of $4.4 million (NPV) over three years?

But the nature of studies like this is that they deliver broad-brush assessments which must be challenged in the context of a specific organization before they make sense. Users in your company might, for instance, not be paid an average of $35/hour (and what defines a “business user” anyway?). Forrester makes the point that their study is “a framework to evaluate the potential financial impact of Microsoft Outlook for iOS and Android on their organizations.” In other words, don’t believe anything in the report until you’ve compared what it says against the reality of your company.

Let’s examine some of the key findings of the study.

Built-in Security

Forrester begins by saying that built-in security delivers big benefits. Hmmm… Instead of focusing on Outlook mobile by itself, their study assumes that the client is used in conjunction with Enterprise Mobility and Security (EM+S) and might replace a third-party mobile device management (MDM) tool. Although it’s true that EM+S is used extensively with Outlook mobile (the latest Microsoft results said that EM+S has 127 million subscribers while the most recent figure for Outlook mobile is 100 million), it’s reasonable to say that Outlook mobile is not the only reason why Office 365 tenants (in particular) use EM+S. Getting access to other features like Azure Active Directory conditional access policies might be more important.

Organizations decide on MDM strategy not based on a single application but rather on what the tool can do to manage a range of Microsoft and non-Microsoft applications. Email clients are important, but they’re not the single most important criterion for choosing an MDM application.

Forrester asserts that Outlook mobile reduced risk of a security breach by 10% because security teams did not have the desired level of control over settings of other email clients. Being able to wipe lost devices and insist on password policies (like requiring a PIN) isn’t unique to Outlook mobile and has been available for Exchange ActiveSync for years. Going forward, using Office 365 sensitivity labels (supported by Outlook mobile since October 2019) might be a more important factor in avoiding data leakage due to the way rights management restricts access to authorized recipients. This isn’t covered at all in the report.

Productivity Growth

Although it’s hard to quantify productivity growth, I think Forrester is on safer ground here. Compared to the other mobile email clients commonly used with Exchange Online, Outlook mobile is a far superior application. This isn’t the fault of the other email clients. Instead, it’s more to do with the now-old ActiveSync protocol which limits the features these applications can deliver. Allowing delegate access to user mailboxes is a recent example.

I especially find it easy to believe that people will be more productive working with their Exchange Online mailbox and calendar through Outlook mobile than with a client like the iOS native mail app.

Reducing the Cost of User Training

Forrester estimates that the transition to a single mobile email client reduces the cost of user training by 60%. I see two issues with this statement. First, Outlook mobile is a more functional client. It therefore surely requires more training to master, even taking account of the closer connection in interface between Outlook mobile and the other Outlook clients. The nature of modern apps is that few organizations deliver formal training because “they all work the same way.” That statement might be considered facetious, but I have heard it from senior leaders.

Forrester returns to more defendable ground when they state the obvious that having a single mobile email client simplifies technical support. I also accept that the better calendar implementation in Outlook mobile should reduce help desk tickets. However, I wonder if it is true that support engineers could “guarantee deep knowledge of the product” after adopting Outlook mobile. Microsoft updates Outlook mobile frequently to add new features (good) at the expense of creating a knowledge acquisition challenge for support engineers. Some new features, like including a better text editor, or easily assimilated by end users. Others, like delegate access, might not be.

It could be argued that third-party mobile email apps are easier and cheaper to support because the same degree of constant change doesn’t happen. Those apps are constrained because any feature that relies on access to Exchange content is limited by the boundaries of the ActiveSync protocol. In other words, once you know how to support the iOS or Samsung mail app, you don’t need to do much to keep updated.

Outlook Mobile Needs Management

The offset the expected savings, Forrester acknowledges that the deployment and ongoing management of Outlook mobile requires some investment. Ongoing administration and change management is calculated at twice the cost of the initial deployment (out to six months), which seems reasonable. The costs for an individual company will depend on the number of users, sites, and other factors such as the MDM used (if any), and other mobile device initiatives that you might have going such as encryption, custom apps, and so on.

Some Skepticism is Needed

Microsoft didn’t ask Forrester Research to undertake this study on a whim. It’s a competitive tool to convince customers of the merits of Outlook mobile and to provide champions inside organizations with some convincing arguments to deploy the client. When you read studies like this, always remember who paid for the work, ask why they invested the effort, and read the conclusions with some well-judged skepticism.

I was already convinced that Outlook mobile is the best mobile email client for Office 365 before reading the Forrester report. Studies like this don’t do anything to change that belief. The client’s functionality and usefulness deliver enough proof for me.