How to measure Copilot adoption beyond licence data
What the Copilot usage report and Copilot Dashboard show, what they miss, and how to measure AI adoption by use, proficiency and the reasons behind both.
Licence and usage data show who has Copilot and who has used it. They don’t show whether it helps with the work, who tried it once and gave up, or what holds people back. To measure AI adoption properly, track three things: how many people use it and how often, how well they use it on real work, and the reasons behind both. The first comes from Microsoft’s reports. The other two come from asking employees.
What does the Microsoft 365 Copilot usage report show?
The Copilot usage report in the Microsoft 365 admin center gives three headline numbers: enabled users (people with a licence), active users and the active users rate, which divides one by the other. You can view the last 7, 28, 90 or 180 days, see adoption by app (Teams, Outlook, Word, Excel and so on) and count prompts submitted.
The definition of “active” matters. A user counts as active when they take an intentional action with an AI feature, such as submitting a prompt. Opening the Copilot pane in Word doesn’t count. So one prompt in 28 days makes someone an active user for that period.
There is also a per-user table with each person’s last activity date in each app. By default, Microsoft hides user names in usage reports, and an admin has to change a setting to show them. Think carefully before you do. A list of named people who “don’t use Copilot” turns a learning question into a performance one, and people notice.
What does the Copilot Dashboard add?
The Microsoft Copilot Dashboard in Viva Insights covers four categories: readiness, adoption, impact and sentiment. It adds trends by group, usage levels from novice to power user, returning users, and filters by organisation or job function. Groups smaller than your minimum group size are hidden.
The impact page shows “Copilot assisted hours”, which Microsoft describes as an estimate: it multiplies the actions people take in Copilot by factors from Microsoft’s own research. “Copilot assisted value” then multiplies those hours by an hourly rate, set by default to $72. Treat both as modelled figures. They tell you the volume of activity, priced at an average rate. They don’t tell you whether a report written with Copilot was any good.
The sentiment section shows survey results from Viva Glint, Viva Pulse or a file you upload, alongside the usage data. That is a useful step, though what you get back depends entirely on the questions you ask.
What can’t usage data tell you?
Picture two teams with the same 60% active users rate. In the first, people use Copilot daily to draft client letters and trust what it produces. In the second, most people tried it once in a training session, got a weak answer, and haven’t been back; the rate is held up by a handful of enthusiasts. The dashboard looks the same for both.
Usage data also can’t see:
- work done in other tools, including personal ChatGPT accounts that never show up in your reports (see our post on shadow AI);
- whether the output was used, rewritten or thrown away;
- why a team stopped, which is the question leaders actually ask when the numbers drop.
How do I measure AI adoption across my company, beyond licence usage?
Change practitioners already have a useful frame. Prosci measures the people side of a change through speed of adoption (how quickly), ultimate utilisation (how many) and proficiency (how well). Applied to Copilot or any other AI tool, it looks like this.
Speed of adoption: how quickly
How many weeks from licence to first real use, by department. The usage report gives you this. A department that takes three months to reach the rate others hit in three weeks is worth a closer look.
Utilisation: how many, and how often
The active users rate, but also returning users and active days. One prompt a month and daily use both count as “active”, so look at frequency as well as reach. The Copilot Dashboard’s usage levels help here.
Proficiency: how well
This is where licence data runs out. Proficiency means people use the tool on tasks that matter, know when not to trust it, and get output they can use. You find it out by asking people about specific tasks: what they last used it for, how much they had to fix, and where it let them down. Peer review of real work, such as a sample of drafted documents, can add to this.
The reasons behind all three
Each number above tells you where to look. None tells you why. The reasons are usually practical, and they vary by team: no use case, no time, unclear rules, a poor first answer, a manager who never mentions it. We go through them in why employees don’t use AI tools.
How do you add the qualitative layer?
Run a short round of open questions next to the usage data, set up with the same departments so you can lay one over the other. Ask after the first few weeks, then each quarter. Use questions about real tasks; our list of AI adoption survey questions has 24 you can copy.
Two things make the answers useful. First, follow up on each answer, because “it’s fine” and “I don’t use it much” only become useful when someone asks what happened. Second, protect people. Report only at group level, with a minimum group size, and say so in the invitation.
Spradley does this with one-on-one AI interviews, by text or voice, sent by email or Slack. The interviewer asks a follow-up on each answer, and the report groups what people said into themes with the quotes behind them, so you can compare departments against your Copilot figures. Numbers and ratings are never shown for fewer than five people, and a privacy judge checks each written finding. Read more on our AI adoption page.
Frequently asked questions
What counts as an active Copilot user?
In Microsoft's usage report, someone who took an intentional action with a Copilot feature in the selected period, such as submitting a prompt. Opening the Copilot pane without using it doesn't count.
Is Copilot assisted hours a measure of time saved?
It is an estimate. Microsoft calculates it from the actions people take in Copilot, multiplied by factors from its own research, so treat it as a modelled figure rather than measured time.
Which metrics should we track for AI adoption?
Speed of adoption, utilisation and proficiency, following Prosci's frame. The first two come from usage reports. Proficiency and the reasons behind low use come from asking employees.
Should we look at named users who don't use Copilot?
Usage reports hide names by default. Look at groups instead, and ask people why. Singling out individuals tends to make people use the tool for show rather than for real work.
Key terms
- active user
- In Microsoft’s Copilot usage report, someone who took an intentional action with an AI feature in the period, such as submitting a prompt. Opening the Copilot pane alone does not count.
- Copilot assisted hours
- A Microsoft estimate that multiplies the actions people take in Copilot by factors from its own research. It is a modelled figure, not measured time.
- speed of adoption, utilisation and proficiency
- Prosci’s three measures of how a change is adopted: how quickly people take it up, how many use it, and how well they use it.
- shadow AI
- Employees using AI tools for work that their organisation has not approved, such as a personal chatbot account, a browser extension or an app on their own phone.
- minimum group size
- The smallest number of responses a group needs before its results are shown. Five is a common default; smaller groups are hidden or combined with a larger one.
