Updated Compliance Assessment Report Released for Microsoft 365

Cohasset Assesstment


In late October, Microsoft released a compliance assessment report written by Cohasset Associates to compare Microsoft 365 services against the requirements of U.S. financial regulations. The report covers SharePoint Online, OneDrive for Business, Teams, Exchange Online, and Skype for Business Online, but excludes Stream, OneNote, and Planner because these apps do not support retention policies. Yammer is also excluded. In addition, the report focuses on email and files and excludes media like lists, whiteboards, and so on. The report is available to Microsoft 365 tenants on the Service Trust portal.

According to Microsoft, the report is: “an updated and expanded assessment of Microsoft 365 services against SEC Rule 17a-4(f) and FINRA Rule 4511(c), including SharePoint, OneDrive, Teams, Exchange, and Skype. This assessment examines new features that enable Broker-Dealers to retain their regulated books & records in select Microsoft 365 services in a compliant, secure, and immutable manner.”

The report is an update from a version released in January 2019. Like other reports of the type. Microsoft funded its production. This doesn’t make the report any better or worse, but it’s something to remember.

Dry but Valuable Information

Compliance reports can often be dry reading, and this is no exception. The intent is to assess how Microsoft 365 retention measures up against regulatory requirements to maintain books and records on electronic media, so it was never going to be a potboiler novel. Nevertheless, this is important information for compliance officers and others who manage records for financial institutions.

You always learn something from an in-depth review of an area, so the report is worth reading even if you’re not a compliance officer or the regulations don’t apply to your business. For example, you’ll learn about the steps Microsoft takes to protect customer data stored in Exchange Online and SharePoint Online.

In reading the document, it’s obvious that Cohasset Associates know their regulations backwards. However, they might not fully appreciate how different Office 365 components work. This is often the case when independent experts examine a particular aspect of a system. The experts know what they are looking for and what should be there, but they can rely on the information provided rather than what happens in real-life tenants.

Teams Reactions and the Audit Log

It’s important that statements made in reports like this are accurate and I found a couple of places where I disagree with the text. For example, the assertion on page 7 that: “reactions (e.g. likes, thumbs up) applied to Teams channel messages and Teams chats are tracked in the Microsoft 365 unified audit system” is surprising (the statement is repeated on pages 12, 22, 25, 29, and 50). At times, cut and paste repetition is obvious, probably because of similar requirements in different regulations.

The only unified audit system in Microsoft 365 is the unified audit log (the report covers the audit system on page 37). Currently, reactions to Teams messages are not stored in the audit log. In fact, reactions aren’t stored in the Teams compliance records captured by the substrate in Exchange Online mailboxes (referred to in the report as the “regulated record”).

It would be inappropriate if reactions were stored in the audit log. This data belongs in compliance records (see below) and not somewhere else. Keeping the two pieces of data apart makes the task of investigation much harder. The substrate captures compliance records for Teams messages. Creating additional records including reactions is eminently possible (as edits to Teams messages are captured) and it’s what should happen.

More Missing Data

Apart from reactions, other Teams message data not captured for compliance purposes include recordings of voice memos made on mobile devices and text inside code snippets. The report misses these facts completely even though they are obvious ways for those working in regulated industries to avoid oversight.

For example, Figure 1 shows a Teams chat message which includes some text, a code snippet, and a reaction.

Image 1 Expand
Teams compliance item code snippet chat
Figure 1: A Teams chat message with a code snippet (image credit: Tony Redmond)


Figure 2 shows the compliance record created in Exchange Online as viewed through the MFCMAPI utility. In this case, we’re looking at the PR_HTML property (formatted HTML version of what users see). The text is visible, but there’s no trace of the code snippet or the reaction.

Image 2 Expand
Teams compliance item code snippet
Figure 2: The compliance record in Exchange Online as viewed through MFCMAPI (image credit: Tony Redmond)


I’m puzzled why Teams private channel messages are excluded from the assessment given that these messages are captured in compliance records in personal mailboxes. Perhaps this is because a retention policy can’t yet be created to process just those records.

Teams and MAPI

Confusion continues across pages 29 and 32 when we’re told that “Teams channel messages and Teams chats are stored in MAPI format.” Once again, these are compliance records stored in Exchange. The substrate creates compliance records as mail items stored in mailboxes, which is why you can use MFCMAPI to examine their contents. Compliance records for Yammer conversations use the same approach.

The point is also made that Teams channel messages can be rethreaded as a download option. Chats can too, but conversation reassembly is only available in Advanced eDiscovery. Core eDiscovery content searches return individual messages.

Teams Retention Processing

In terms of Teams retention processing (page 20), it’s important to realize that the Exchange Managed Folder Assistant (MFA) removes Teams compliance records from mailboxes based on retention policy settings; processing then occurs to synchronize deletions from Exchange to the Teams chat service (Azure Cosmos DB). The report implies that the items in Exchange and Teams are equivalent: they are not. The version in Exchange is an imperfect copy created for retention and other compliance processing. The impression that Exchange stores Teams data is perpetuated in the table presented on page 49. This myth is widely held and it’s a pity that Microsoft did not emphasize that Teams channel messages and chats are stored in Azure Cosmos DB and nowhere else.

The statement on page 21 that a disposition review will “review mailboxes with at least 10 megabytes of data in the mailbox” is misplaced and has nothing to do with disposition reviews, which is where a manual review of expired items occur to decide upon their final fate. MFA does not apply retention policies to mailboxes unless they hold more than 10 MB, a state easily attained after a mailbox receives a small amount of traffic.


Encryption of records is covered on page 11 and says that this covers “records and associated properties and metadata attributes.” While not explicitly stated, I assumed this refers to sensitivity labels. Email and document metadata like subjects, author, and creation date are not encrypted when sensitivity labels are applied. But reading on further into the document, mentions on pages 24 and 30 clarify that this refers to using the Customer Key service to encrypt customer data at rest in Microsoft’s datacenter. Which then creates the question about what recommendation Cohasset Associates have for customers to deal with content encrypted by sensitivity labels?

Hidden Archives

You might also be puzzled to frequent references to “hidden archives.” This has nothing to do with archive mailboxes; it’s a common term applied to the recoverable items folder in Exchange Online mailboxes and the preservation hold library in SharePoint Online document libraries where items under hold are retained when deleted.

Hold Guests

Page 26 makes the intelligent recommendation to configure Teams meeting policies to force guests to wait in the lobby before they are admitted to meetings by an authenticated user. The recommendation is spoilt by referring to a retention policy, which isn’t involved with meetings.

Confusing Audit Events

It’s a pity that the table of audit events in 2.14.3 contains references to some operations to track. In passing I note that due to some unfathomable reason, several operations result in multiple audit records which seem similar. For example, the NewComplianceTag event is logged when someone creates a new retention label, but so is the New-ComplianceTag event. One event is created by the Data Governance workload, the other by the compliance center. The NewComplianceTag event is probably the one you want as it captures the important information about the new retention label.

$Records = Search-UnifiedAuditLog -UserIds [email protected] -StartDate "04-Nov-2020 19:00" -EndDate "05-Nov-2020 23:00" -ResultSize 1000 -Operations Newcompliancetag, New-Compliancetag

$Records | group operations | ft name, count

Name              Count
----              -----
NewComplianceTag      3
New-ComplianceTag     3

$Records | ft operations, creationdate, recordtype

Operations        CreationDate        RecordType
----------        ------------        ----------
NewComplianceTag  05/11/2020 11:50:24 DataGovernance
New-ComplianceTag 05/11/2020 11:50:23 SecurityComplianceCenterEOPCmdlet
New-ComplianceTag 05/11/2020 11:27:47 SecurityComplianceCenterEOPCmdlet
NewComplianceTag  05/11/2020 11:27:47 DataGovernance
NewComplianceTag  05/11/2020 10:57:14 DataGovernance
New-ComplianceTag 05/11/2020 10:57:12 SecurityComplianceCenterEOPCmdlet

No More TechNet

Page 42 tells us about the wonder of TechNet articles. Which is nice, except that TechNet is no more and its role has been taken by the Microsoft Technical Community. Maybe Cohasset Associates didn’t see Microsoft’s announcement.

The Need for E5

Cohasset Associates put great importance on regulatory record labels, which are a form of retention label which blocks changes to items to which they are applied. The problem here is that you can’t create regulatory record labels unless you have Office 365 E5 or Microsoft 365 E5 compliance licenses, which are necessary to access the Records Management section of the compliance center. The same is true if you want to retain audit data for more than 90 days, use the customer key service, or rethread Teams messages to form a conversation (with Advanced eDiscovery). Apart from recommending that you should have appropriate licenses and pay for “all appropriate services,” there’s no mention of license requirements in the document. It’s a small but important point, especially if you’re an organization without high-end licenses.

Good Compliance is Like Good Housekeeping

The report concludes that the assessed Microsoft 365 services meet the regulations “when compliance features are properly configured, carefully applied, and managed.” I have no qualms with this view. As we know, when retention goes wrong, it can have devastating effects.

This report includes some interesting and worthwhile information. It’s just a pity that some of its assertions are debatable and the terminology used is sometimes unclear. A careful readthrough by someone skilled in the art of Microsoft 365 compliance would have helped to refine and improve this document. Unfortunately, that doesn’t seem to have happened.