Office 365 Data Governance Framework Spans Multiple Workloads

Compliance and Regulations

Given the somewhat litigious nature of today’s business world, there is no surprise in the number of compliance features Microsoft builds into products like Office 365. In fact, the breadth and depth of those features is one reason why I think Office 365 is more popular with large enterprises than its major competitor, Google G Suite.

But good as the Office 365 compliance features are, gaps still exist. Yammer is an example of a product that has weak compliance functionality. Teams and Planner are others.

Keep What You Need and Get Rid of the Rest

Microsoft’s tag line for data governance is that “you keep what you need and get rid of what you don’t”. Last week, Microsoft made new functionality available through the Security and Compliance Center to help tenants keep content that they need and remove what they do not want to keep. The new functionality comes in the form of classification labels and retention policies, both of which combine to give tenants different options to control how long content exists in mailboxes, sites, and other Office 365 locations.

You create classification labels under the Classifications section of the Security and Compliance Center. When ready, you publish sets of labels in label policies, which then show up as retention policies under the Data Governance section. That seems a tad confusing, but it all comes together in the framework. Think of it this way: labels are the way to control content at a precise, item-specific level. Retention policies offer broad-brush coverage of content at volume. Together, the mixture of specific and general control affords tenants flexibility in how they build a data governance strategy for the organization.

Best of all, the new framework is designed to work across Office 365, including Office 365 Groups. It is a big step forward and is in line with other projects to offer cross-workload functionality in content searches and Data Loss Prevention (DLP).

Office 365 Retention Policies

Since their first appearance in Exchange 2010, retention policies have let administrators configure and apply policies to on-premises and cloud mailboxes to help users control items through a mixture of system-controlled tags and personal tags. Actions specified in the tags control how long items are kept in the mailbox and what happens once their retention period expires.

Retention policies work well for Exchange and Microsoft has gained a lot of experience in how customers use retention policies to manage content since 2010. All of which leads to the introduction of Office 365 retention policies to deal with Exchange (mailboxes and public folders), SharePoint, OneDrive for Business, Skype (IM conversations), and Office 365 Groups.

This is Microsoft’s second version of multi-workload retention as they launched preservation policies in 2015 to control content stored in Exchange mailboxes and SharePoint and OneDrive sites. Any preservation policies that exist in a tenant are automatically upgraded to become retention policies that keep but do not remove content after the retention period expires.

Expanding Retention Policies to deal with Multiple Office 365 Locations

To make retention policies available to other Office 365 workloads, Microsoft has evolved and expanded the core principles behind Exchange retention policies. In doing so, they have had to drop some Exchange-specific features, like the ability to move items to archive mailboxes.

Losing the ability to archive items automatically is regrettable (but only for Exchange). On the upside, retention policies incorporate the ability to set in-place holds so that users cannot permanently remove items if those items come within the scope of a policy. The simplest kind of policy puts every item in a mailbox or site on hold while more complex policies cover items that match queries or (for SharePoint and OneDrive for Business) or hold certain kind of sensitive data, like social security numbers or other “personally identifiable information” (PII). The same sensitive data types used in Data Loss Prevention policies are supported for retention.

It is all very flexible, and best of all, these policies implement the same processing across all the supported workloads. One policy to rule them all is so much better than having to configure multiple policies that work differently across different applications. The introduction of service-wide retention policies is yet another example of how Office 365 is fast leaving its roots of “cloudified” on-premises products behind.

Like Exchange Retention Policies but Better

Anyone who has ever worked with Exchange retention policies will find similarity with Office 365 retention policies. However, some significant differences exist:

Exchange Retention Policies Office 365 Retention Policies
Apply to Exchange mailboxes (including shared mailboxes) Exchange mailboxes

Office 365 Groups

SharePoint document libraries

OneDrive for Business sites

Skype for Business IM

Public folders

** Microsoft says that Yammer and Planner will be supported soon.

Assignment Assigned to mailboxes (the default policy is assigned to all Exchange Online mailboxes) Policies are assigned to mailboxes and other locations, but locations can also be excluded from policies.
Composed of Each retention policy consists of a set of folder tags for specific system folders (like the Inbox), personal tags, and default tags. Three default tags can exist in a policy (for deletion, archive, and voicemail). Policies function like default tags in that the policy applies to all items in a location that are not otherwise tagged (for instance, with an Exchange personal tag or an Office 365 classification label).
Actions Move to Deleted Items

Permanently Delete

Move to Archive

Keep and then remove content

Keep and do nothing

Remove old content

Enforced by Managed Folder Assistant Managed Folder Assistant (for Exchange and Office 365 group mailboxes); other background processes service the other locations.

Table 1: Comparing Exchange and Office 365 Retention Policies

Moving from Exchange Retention Policies

One inevitable question that arises is whether tenants should move from Exchange retention policies to Office 365 retention policies and classification labels. As obvious from Table 1, significant differences exist between the two types of retention policies, so the answer is unclear at this point.

Every tenant is different and although it might be easy for a cloud-only tenant with relatively simple retention needs to go ahead and embrace Office 365 retention policies, the situation is probably very different for large and complex tenants that already have a well-defined retention strategy in place. Things become even more complicated for hybrid tenants, who often want to use the same processes on-premises and in the cloud.

Experience and time will allow us to develop better answers. In the meantime, new tenants should start with Office 365 retention policies and classification labels while older tenants test, compare, and contemplate what is their best course of action. Microsoft says that they plan to keep the older workload-specific functionality available within Office 365 to allow organizations to make the transition. That is wise because the nature of retention is that items can be kept for a long time and no one wants to be forced into changing strategy in such a way that it might affect terabytes of retained content.

Classification Labels

Where retention policies handle the bulk-processing of content, Office 365 uses classification labels to mark content for specific treatment, like keeping certain documents for longer periods because they contain specially-valuable information.

Classification labels are published to the different applications to make them available to users, after which they can be applied to content. The way that classification labels appear in OWA and Outlook desktop is interesting because they are presented in the same way as personal retention tags. In other words, when Office 365 publishes classification labels to Exchange, clients pick up and use the labels like personal tags.

Figure 1 shows a mixture of classification labels and retention tags in the OWA UI. The retention tags appear because a retention policy applies to the mailbox, while the classification labels appear because a label policy includes the mailbox within its coverage. The user can select either a label or a tag to preserve an item, proving that Exchange retention policies co-exist peacefully alongside Office 365 classification labels.

OWA and Office 365 classification labels
Figure 1: OWA mixes classification labels with personal retention tags (image credit: Tony Redmond)

In Figure 2, we see how the same classification labels with the same retention settings are assigned to a document in a SharePoint library.

Office 365 Classification Labels - SharePoint
Figure 2: Classification Labels in use with SharePoint (image credit: Tony Redmond)

Only one label can exist on an item at any time. It is also possible to set a default label for a SharePoint site so that every item in the site inherits the classification.

Background processes make sure that the instructions contained in the label settings are respected. For instance, any item stamped with the “Archive Retention” label might be kept for 10 years and then removed from wherever it is stored.

Labels can also be used to mark items as “records.” This is a special status meaning that the item is needed for formal record-keeping and therefore cannot be changed or removed from Office 365 until its retention period expires.

Another interesting capability is when labels are auto-applied to content. To do this, you combine a label in a policy that is associated with a query or some sensitive data types. When the policy is published, Office 365 finds matching content and automatically applies the label. Users can overwrite an auto-applied label with a label of their choice.

Auto-apply label policies are a feature of the Office 365 E5 plan. It is a nice feature that helps tenants ensure that people do not make mistakes when they apply labels, but it is unlikely to be the stand-out reason why anyone upgrades to E5.

Rationalizing Labels within Office 365

“Label” is a generic term that is used elsewhere within Office 365 and Microsoft has some work to do to rationalize how they use the term. In the immediate future, we have to deal with three types:

  • Classifications are labels placed on Teams and Office 365 Groups give visual indicators to members about the sensitivity of the information belonging to the team or group but do not enforce any processing based on the classification.
  • Labels defined in Azure Information Protection can invoke processing to protect content stamped with the labels. For example, any content stamped with a label called “Most Sensitive” might lead to the automatic application of a rights management template to secure access to the content to users in a specific group.
  • Classification labels for Office 365 control the deletion and retention of content across multiple Office 365 locations, including the marking of items as formal records.

Creating a single uber-label that supports all the characteristics listed above will take some thought. A practical approach might be to have a single label whose capabilities are selectively enabled by licensing. The creation of such a label will need many contributions from different development groups. It will not happen overnight.

Groups and Compliance

Up to now, Office 365 Groups were a major problem area for compliance. Now, you can apply classification labels and retention policies to Office 365 Groups, but only those that use Exchange to store group conversations. Yammer-based groups are not yet supported.

Teams and Planner are still problematic, but at least we see progress. With that thought in mind, my next article will look at how to use retention policies with Office 365 Groups.

Follow Tony on Twitter @12Knocksinna.

Want to know more about how to manage Office 365? Find what you need to know in “Office 365 for IT Pros”, the most comprehensive eBook covering all aspects of Office 365. Available in PDF and EPUB formats (suitable for iBooks) or for Amazon Kindle