Office 365 Secure Score Analyzes Tenant Security

Secure Keyboard Hero

Scoring Office 365 Tenant Security

In August 2016, I wrote about the Office 365 Secure Score service, which was then in preview and noted that my tenant had scored 50 out of 243. Now, the service is in production and my score has advanced to 55 (Figure 1). Naturally, I am thrilled.

Secure Score Office 365 tenant
Figure 1: Viewing the Secure Score for an Office 365 tenant (image credit: Tony Redmond)

The idea behind Secure Score is simple. Microsoft acknowledges that it can be difficult for an administrator to understand how best to secure an Office 365 tenant. There are many places in administrative consoles where settings can be tweaked and much to monitor on an ongoing basis. It therefore makes sense to measure a tenant against a set of predetermined standards and score the tenant based on the actions taken to increase security. At the same time, outstanding actions can be flagged to the administrator, who then decides whether to implement the action and so increase the tenant score.

For example, if Rights Management is configured to allow tenant users to protect confidential content, it’s worth five points. Even better, if users store documents in OneDrive for Business, it’s worth ten points. Although you can argue that OneDrive for Business is a more secure location for documents than a local hard drive or a network file share, assigning ten points to this measurement seems like more of an encouragement to do better.

The points awarded for different aspects are combined into a tenant score. The maximum rating is 450 points. I have some work to do to increase my score from 55. On the upside, the dashboard says that the average score for an Office 365 tenant is 18, so most tenants have even more to do.

To assess a tenant, you log onto using a global administrator account (the plan is to include the service in the Security and Compliance Center). Global administrator access is required to measure all the areas that contribute to the security of a tenant. The first time you assess a tenant, you’ll be asked to grant access.

Assessment is not a one-time operation as a check is performed daily to determine an updated score, which is then published to the tenant dashboard.

Suggested Actions

The dashboard includes a useful list of suggested list of actions (Figure 2) that can be taken to improve the score. I noted some errors in the list such as the edict to enable mailbox auditing for all users, something that has been in place in my tenant for some time now. The report informed me that auditing was enabled or 343 mailboxes out of 385, which was an interesting observation considering that the tenant includes just 49 user, room, and discovery mailboxes. Another suggestion is to force password resets every 60 days, a technique that is not best practice when multi-factor authentication and strong passwords are used.

Office 365 Secure Score actions
Figure 2: Actions to improve a tenant’s Secure Score (image credit: Tony Redmond)

Some of the actions are noted as “Not Scored”. This indicates that addressing the action won’t influence the tenant score now – but it might in the future when Microsoft incorporates the action into the Secure Score assessment.

The Secure Score dashboard includes a Score Analyzer tab to allow administrators to:

  • Track progress of their score over time.
  • Understand the actions that contribute to the current tenant score.
  • Understand how they can improve their score by completing various actions. For example, a tenant score increases by 30 points if multi-factor authentication is enabled for all users whereas 15 points is added if the outbound spam policy notifies an administrator when a tenant user is blocked for suspicious activity.

Analysis tools like Secure Score are constantly reviewed to ensure accuracy and relevance. Some of the errors that I noted in August have been addressed by Microsoft and some new tests have been added. But that’s not the point. The reason why Secure Score exists is to drive awareness of the actions that administrators can take to increase the security of their tenant. You might not agree with Microsoft’s assessment of the importance of the various measurements but that’s just detail. The more important thing is to maintain awareness of security on an ongoing basis.

Pay Attention

More information on Secure Score can be gained by watching the Ignite 2016 session on the topic. You can help Microsoft develop Secure Score by noting any issues that occur in the Microsoft Technology Community. Overall, despite some minor glitches, Secure Score is a very worthwhile service that deserves your support – and your attention, especially if your tenant is one of those that scores below mine.

Follow Tony on Twitter @12Knocksinna.

Want to know more about how to manage Office 365? Find what you need to know in “Office 365 for IT Pros”, the most comprehensive eBook covering all aspects of Office 365. Available in PDF and EPUB formats (suitable for iBooks) or for Amazon Kindle.