Microsoft’s New Copilot Benchmarks Tool Lets Businesses Track AI Adoption and ROI

Microsoft is giving organizations a new way to measure their AI progress with the Benchmarks feature in the Copilot Dashboard within Viva Insights. This tool lets leaders compare their Copilot adoption and engagement rates against anonymized data from other companies.

Microsoft notes that many organizations using Microsoft 365 Copilot struggle to assess how well they are adopting and utilizing the tool. It becomes challenging to identify gaps, measure success, or make informed decisions about enhancing AI integration without a way to compare their progress to similar companies.

How does the Benchmarks tool work?

The new Benchmarks feature allows organizations to compare Copilot adoption across different internal groups, like departments, regions, or job roles. It also provides key metrics such as the percentage of active Copilot users and returning user percentages to help leaders track engagement over time.

“The cohort benchmark is based on a comparison cohort of employees within your company who share similar job functions, regions, or manager roles. The calculation uses Job function, Region, and Manager attributes to determine expected values by role. The cohort result looks at the role composition of the selected group, and constructs a weighted average expected result based on matching roles across the tenant,” Microsoft explained.

The Benchmarks tool indicates a growing need among managers to measure how well Copilot is being adopted, especially to support ROI discussions with executive leadership. It offers data-driven insights that can help justify investments and guide future AI strategies.

This data could help organizations cut unnecessary license costs by identifying teams that rarely use Copilot. However, it also raises concerns about potentially pressuring employees to increase usage just to meet benchmarks.

Microsoft is not only enabling internal comparisons of Copilot usage within organizations but also offering external benchmarks to show how a company’s adoption compares against others. These comparisons can be filtered by industry, company size, or region.

Privacy and data protection measures

Microsoft emphasizes that all benchmark data is aggregated and anonymized to ensure privacy and compliance. “External benchmarks are calculated using randomized mathematical models to ensure privacy. Each benchmark group includes at least 20 companies and is derived from approximations to prevent the use of any single company’s actual data,” Microsoft added.

Microsoft notes that the Benchmarks feature is currently available in private preview within the Copilot Dashboard. The company plans to roll out this update gradually to all commercial customers later this month. Microsoft plans to introduce additional benchmarks based on customer feedback, though there is no ETA yet.