The Problem with Office 365 Backups
Lack of Microsoft APIs Creates Challenges for ISVs
As some of you might know, I’m not a great fan of the backup products available for Office 365 today. Sure, the likes of Spanning and Veeam do their best with the available tools, but some fundamental problems exist that only Microsoft can solve.
The first problem is that the APIs used by backup ISVs were never designed for cloud backup across the internet. For instance, it is by chance that Exchange Web Services (EWS) is available to move mailbox data from Exchange Online to ISV datacenters. Microsoft designed EWS to replace MAPI as the foundation for client-side applications, not heavy-duty shipping of terabytes of data across extended connections. We’re just fortunate that things work as well as they do. One hopes that the situation continues when Microsoft disables basic authentication for EWS on October 13, 2020.
The situation with SharePoint Online and OneDrive for Business isn’t much better. Many ISVs offer backup products to copy documents and other site elements, but once again the protocols they rely on are based on on-premises concepts and assumptions instead of the cloud.
No Backup for Cloud-Created Apps
But at least APIs and protocols exist for the two basic Office 365 workloads, in no small part because of their on-premises heritage. Things get more complicated for the new Office 365 apps that only exist in the cloud, like Teams, Planner, To-Do, Yammer, and Stream. Few options exist here because Microsoft hasn’t created backup APIs for Office 365, and ISVs are often limited to claiming that they can backup a workload because they have partial coverage. For instance, they might claim that Teams can be backed up because the SharePoint sites belonging to Teams are backed up. Or, even worse, coverage for chats and channel conversations is claimed because a vendor backs up the compliance records captured in Exchange Online mailboxes. In reality, Teams is the most difficult of all Office 365 applications to backup because it is so interconnected with different pieces of the Microsoft 365 ecosystem.
The lack of APIs is underlined by the hoops that ISVs go through to make tenant-to-tenant migrations possible. Although basic items like messages and documents can be moved between Office 365 tenants, significant and fundamental problems exist with apps like Teams because Microsoft has not delivered suitable APIs.
What Microsoft needs to do is to design and deliver backup APIs capable of moving data at cloud scale to backup locations with Azure or to external datacenters. The growing amount of data generated by Office 365 creates one challenge; the connectivity between different applications creates another. Teams and Planner, for instance, depend heavily on components from other parts of the Office 365 ecosystem. There’s no point in backing up raw data if you can’t reassemble it into fully-functional information when needed.
Things are going to become even more complex when Microsoft delivers applications based on the fluid framework. Long ago, Office documents became containers full of XML data; the fluid foundation introduces live updates of data between apps (OLE/DDE on steroids)., and that might create another complication for those who want to backup and restore information.
After delivering a reliable and performant set of APIs to backup vendors, Microsoft might then look at some of the tools included in Office 365 to make sure that they all work when stressed. Not everyone can afford to pay for a third-party backup service and rely on out-of-the-box like SharePoint’s Restore This Library, which sometimes doesn’t work so well. It would be nice if Microsoft offered basic backup and restore capabilities for all Office 365 workloads.
Encryption is a Growing Issue for Backups
Dealing with protected content is the last item on the agenda. Today, a very small percentage of Office 3655 data is encrypted, but that will change over the coming years because of Microsoft’s efforts to popularize rights management-based encryption through Office 365 sensitivity labels (recently enabled for SharePoint Online and Office Online).
Microsoft has previews running to show how Office 365 tenants can apply large-scale protection to at-rest data with background processes that can protect tens of thousands of documents or messages daily. When everything that Microsoft has in preview is generally available, the ways to protect Office 365 data will be many and varied. And that will mean that a far higher percentage of documents and messages will be protected over time.
Backup products, like many other add-on products for Office 365, operate based on unfettered access to data. How these products will work when faced with the need to process encrypted data remains to be seen. Streaming protected data is probably not a great issue; restoring that data might be a completely different matter. The Microsoft Information Protection team needs to step up and engage with ISVS to make sure that their products can deal with sensitivity labels.
ISVs Can Only Do So Much
Life was much easier on-premises. Applications are simpler. Data are simpler. The process to backup and restore data is simpler. The cloud has improved so many things in terms of functionality. It would be nice if backup APIs had kept up, even if Microsoft doesn’t really consider them necessary.