No Way to Stop Gathering Data Used by Productivity Score

Microsoft Graph and Audit Log Too Important to Lose

Thinking about the fuss and bother which erupted over Microsoft Productivity Score, I concluded that the people concerned about management oversight of user activity within Microsoft 365 had very little knowledge about the topic. They looked at the pretty graphs and tables of user data and reached a conclusion that wasn’t grounded in reality. That’s sad, but when you shout about something you know little about, the output is seldom rational or useful. Usually, it’s just noise.

What’s sadder is that Microsoft gave in to the howling mob. This was the right PR response, but it left people in the position of not knowing what data is collected about Microsoft 365 and what that data can be used for.

No Way to Disable Data Collection

The expressed concern about user privacy, which is a good thing to focus on, provoked the question whether an Office 365 tenant can suppress the gathering of usage data. The answer is no. There’s no administrator-settable control in a tenant to stop Microsoft 365 apps faithfully and comprehensively gathering signals about user activity and stuffing the resulting data into the Microsoft Graph. You can anonymize the data presented in Graph-based usage reports, but the full underlying data remains available.

In fact, not only does Microsoft 365 collect signals about user activity, but often a user action results in an audit record. The net result is that a tenant can not only report numeric counts about what user activities, such as the number of documents created, or Teams meetings attended, but also the details of some events, such as the name of documents someone worked on. And compliance records are collected too so that communications compliance policies can monitor if someone uses inappropriate words in Teams or email conversations. You must be careful these days to not call someone an idiot in email!

High-Level Detail

Even so, the information available to tenants is all high-level stuff. For instance, I might be able to say that you created and uploaded six documents to SharePoint Online last week. I can’t tell what was in the documents, how valuable and correct the written content is, or even how long you spent working on each document. If a sensitivity label is applied, I might be able to deduct the degree of confidentiality of a document, and if I looked at all the audit records accrued for document updates, I might be able to form an idea of when you were active, but not to any degree of accuracy.

Any manager who tried to manage staff based on the information available in the Graph or audit log is being silly. That’s the least offensive word I can use. Maybe cretinous would be better. Seriously, there just isn’t enough data gathered to understand the import, accuracy, and impact of any user’s work. Turning on video might make someone’s picture available in a Teams meeting, but it doesn’t tell you about their spoken contributions. You could read the transcript of the meeting and listen to the recording to understand how acceptable a contribution was, but that’s at a level of detail (and effort) far beyond the review of information presented by Productivity Score. It’s the kind of thing that an eDiscovery investigator might do to hunt down evidence of malfeasance.

The Right Way to Use Graph and Audit Data

I don’t mean to say that the data gathered in the Graph and audit log is not valuable. It is. Period. It’s just not very interesting when it comes to assessing anyone’s productivity. An organization can use the Graph data to figure out if they are paying for expensive monthly Office 365 licenses for inactive accounts (like this PowerShell script). Audit data is terrifically useful for answering questions like who deleted a team or group, who shared a document, or what retention labels are being applied to documents, or who sent a message as someone else. But these aren’t run-of-the-mill usage statistics. They are much more precise queries about specific activities, the answering of which is the real value of the audit log.

Audit data is used in other ways. It flows into Microsoft Cloud Apps Security and it’s used for alert policies. You can turn off ingestion of events into the audit log by running the command:

Set-AdminAuditLogConfig -UnifiedAuditLogIngestEnabled $False

But disabling the audit log would be a huge mistake because the value of audit data to administrators is priceless.

I understand why the privacy advocates are worried when they see usage data being presented in a way that’s open to the worse possible interpretation. But that’s no reason to raise a hue and cry without understanding what the data is and how it can be used. I don’t think that happened around the Productivity Score feature.

Microsoft Folds

After receiving so much bad press, Microsoft decided to remove the user-level data from Productivity Score and rejig text to make the intent and purpose of the feature clearer. The change is already effective. I understand why Microsoft gave in on this matter but regret that they did. Giving in to the baying ignorance of uninformed opinion might have been smart in terms of calming everything down; it did nothing to reassure and inform organizations about what data is gathered about user activity inside Microsoft 365 and how that data can be used for positive effect to solve business problems.