The enterprise software industry loves to buy seats. Execs sign a massive check for GitHub Copilot, announce an "AI transformation" at the all-hands, and pat themselves on the back. Six months later, finance asks for the ROI. Until recently, answering that question meant staring at useless vanity metrics. Daily Active Users (DAU) just tells you an engineer opened their IDE. "Lines of code added" is actively harmful—more code is a liability, not an asset. We needed behavioral data. GitHub finally shipped it. The Copilot usage metrics REST API now exposes AI adoption cohorts. Instead of treating all usage as equal, the API segments your developers into three distinct phases: code-first, agent-first, and multi-agent. This changes how engineering organizations measure AI adoption. You can finally stop guessing and start tracking exactly how your teams interact with the models. ## The Death of Vanity Metrics If you judge your engineering team by lines of code generated, you are doing it wrong. Copilot can spit out a thousand lines of repetitive boilerplate in seconds. That does not make your team faster; it makes your codebase a nightmare to maintain. Early Copilot metrics were rudimentary. They showed active users and a directional measure of code suggested versus accepted. It was enough to justify the initial pilot program, but useless for driving engineering culture. The new dashboard and API update shift the focus from output to behavior. By exposing "Requests per chat mode" and "Agent adoption," GitHub acknowledges that typing code is the least interesting part of a software engineer's job. The real value is in reasoning, context assembly, and orchestration. ## The Three Cohorts of AI Adoption The cohort API endpoint categorizes users based on their interaction patterns. This isn't just about who uses Copilot the most; it's about *how* they use it. ### Code-First: The Autocomplete Junkies This is the baseline. The code-first cohort treats Copilot as a glorified autocomplete engine. They type a comment, wait for the gray text to appear, and hit Tab. **Behavioral signatures:** * High volume of code completions. * Zero or near-zero usage of Copilot Chat. * No interaction with the `/fix` or `/explain` slash commands. * Primary value: saving keystrokes on boilerplate and standard library syntax. There is nothing wrong with code-first usage, but it represents the shallow end of the pool. If your entire enterprise is stuck in this cohort, you are overpaying for licenses. You don't need a massive language model just to remember how to reverse an array in JavaScript. ### Agent-First: The Context Whisperers This is where the ROI actually starts. The agent-first cohort treats the AI as a pairing partner. They don't just write code; they ask questions about the workspace, generate tests based on existing files, and use the model to refactor messy logic. **Behavioral signatures:** * Frequent use of Copilot Chat in the IDE. * High usage of workspace context features (`@workspace`). * Significant volume of requests sent to specific models (e.g., explicitly selecting Claude 3.5 Sonnet or GPT-4o for complex reasoning). * Lower volume of raw code completion, higher ratio of code *modification* (lines deleted or refactored). These engineers understand that the hardest part of software engineering is state management and system architecture, not syntax. They use the agent to map the codebase before they write a single line. ### Multi-Agent: The Orchestrators This is the bleeding edge. The multi-agent cohort doesn't just talk to one AI; they build workflows where multiple specialized agents interact. **Behavioral signatures:** * Heavy utilization of custom Copilot Extensions. * Integration with internal CLI tools, CI/CD pipelines, and automated PR reviews. * High API polling rates as they pull metrics and telemetry back into their local context. * Tasks are delegated to background agents while the engineer acts as the reviewer. Engineers in this cohort are no longer writing features. They are managing the automated systems that write the features. ## Interrogating the API You don't need the shiny web dashboard to see this data. Real engineers use the REST API. The endpoints allow enterprise and organization owners to pull granular JSON telemetry. Here is how you extract the cohort breakdown for your organization. You need a classic personal access token (PAT) with the `manage_billing:copilot` or `read:enterprise` scope. ```bash #!/bin/bash # Fetch Copilot usage metrics with cohort breakdowns ORG_NAME="your-engineering-org" GITHUB_TOKEN="ghp_your_secret_token" curl -s -H "Accept: application/vnd.github+json" \ -H "Authorization: Bearer $GITHUB_TOKEN" \ -H "X-GitHub-Api-Version: 2022-11-28" \ "https://api.github.com/orgs/$ORG_NAME/copilot/metrics?include=cohorts" | jq . ``` The response payload is dense, but the new `cohorts` array is where the signal lives. ```json { "date": "2026-06-01", "total_active_users": 1450, "cohorts": { "code_first": { "user_count": 850, "percentage": 58.6, "avg_completions_accepted": 45 }, "agent_first": { "user_count": 450, "percentage": 31.0, "avg_chat_requests": 18, "top_models": ["gpt-4o", "claude-3.5-sonnet"] }, "multi_agent": { "user_count": 150, "percentage": 10.4, "extension_invocations": 1240 } } } ``` This JSON tells a specific story. Over half the company is still just hitting Tab. Only 10% have figured out how to chain agents together. ## The Team Filter: Finding the Laggards Aggregate data is great for the board of directors, but it is useless for engineering managers. You need to know *which* teams are adopting the tooling and which are stuck in 2023. The recent update allows you to combine the `cohorts` endpoint with the `teams` filter. This gives you exact granularity. ```bash # Fetch cohort data specifically for the backend-platform team curl -s -H "Accept: application/vnd.github+json" \ -H "Authorization: Bearer $GITHUB_TOKEN" \ "https://api.github.com/orgs/$ORG_NAME/teams/backend-platform/copilot/metrics?include=cohorts" ``` If the `backend-platform` team has 40 engineers and 38 of them are in the `code_first` cohort, you have a tooling problem. They are likely writing Go or Rust, and perhaps your internal documentation on how to use `@workspace` for those languages is terrible. Conversely, if your `frontend-infrastructure` team is 90% `agent_first`, you know who to tap for the next internal tech talk. ## Processing the Telemetry with Python Piping JSON through `jq` is fine for a quick gut check, but tracking cohort migration over time requires a script. You want to see the flow of users graduating from code-first to agent-first week over week. Here is a realistic Python script using `requests` and `pandas` to aggregate this data and calculate the transition velocity. ```python import os import requests import pandas as pd from datetime import datetime, timedelta GITHUB_TOKEN = os.getenv("GITHUB_TOKEN") ORG = "your-engineering-org" HEADERS = { "Authorization": f"Bearer {GITHUB_TOKEN}", "Accept": "application/vnd.github+json", "X-GitHub-Api-Version": "2022-11-28" } def fetch_metrics(date_str): url = f"https://api.github.com/orgs/{ORG}/copilot/metrics" # In reality, you paginate or filter by date via the API params # Assuming standard REST filtering for brevity response = requests.get(url, headers=HEADERS, params={"since": date_str}) response.raise_for_status() return response.json() def analyze_cohort_trends(days=30): start_date = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d") raw_data = fetch_metrics(start_date) records = [] for day_data in raw_data: date = day_data.get("date") cohorts = day_data.get("cohorts", {}) records.append({ "date": date, "code_first": cohorts.get("code_first", {}).get("user_count", 0), "agent_first": cohorts.get("agent_first", {}).get("user_count", 0), "multi_agent": cohorts.get("multi_agent", {}).get("user_count", 0), }) df = pd.DataFrame(records) df['date'] = pd.to_datetime(df['date']) df = df.sort_values('date') # Calculate day-over-day shifts df['agent_growth'] = df['agent_first'].diff() df['multi_agent_growth'] = df['multi_agent'].diff() print("\n--- Cohort Migration Report ---") print(df.tail(7).to_string(index=False)) total_graduated = df['multi_agent_growth'].sum() print(f"\nNet new multi-agent users (30d): {total_graduated}") if __name__ == "__main__": analyze_cohort_trends() ``` Run this in a cron job, dump the output to a slack channel, and watch the engineering managers scramble to improve their team's metrics. ## Model Usage and Chat Modes The API doesn't just track *if* they use chat; it tracks *what* they chat with. The "Model usage per day" and "Model usage per chat mode" fields are vital for understanding technical maturity. Developers who stick to default models are usually solving simpler problems. Developers who actively toggle between models understand the limitations of LLMs. They know that Claude 3.5 Sonnet is often vastly superior for dense React refactoring, while a specific GPT model might be better for generating boilerplate Python scripts. When you see a high variance in model selection within the `agent_first` cohort, it signals a healthy, experimental engineering culture. They are treating the AI as a tool with specific tolerances, not a magic 8-ball. ## The Cohort Comparison Understanding the difference between these usage patterns is the key to building an effective enablement strategy. | Metric / Behavior | Code-First | Agent-First | Multi-Agent | | :--- | :--- | :--- | :--- | | **Primary Interaction** | Inline autocomplete (Tab) | Chat sidebar, Slash commands | CLI, Custom Extensions, API | | **Context Scope** | Current file, open tabs | Entire workspace (`@workspace`) | External databases, CI/CD logs | | **API Data Signature** | High completions accepted | High chat request volume | High extension invocation | | **Code Impact** | Adds boilerplate | Refactors, documents, tests | Orchestrates entire PRs | | **Training Focus** | IDE shortcuts, basic prompting | Context assembly, prompt engineering | Custom agent development | ## Practical Takeaways Having this API data is worthless if you don't act on it. Do not just build a Grafana dashboard and call it a day. ### 1. Target Your Training Stop sending generic "How to use Copilot" emails to the entire company. Use the API to identify the code-first users and mandate a targeted session on workspace context. Show them how to use `@workspace /explain` on a legacy module. ### 2. Identify Internal Champions Find the 5% of your engineers in the multi-agent cohort. Look at the API logs to see which custom extensions they are triggering. Buy them a coffee, figure out what internal tools they have hooked into Copilot, and productize those workflows for the rest of the company. ### 3. Audit Your Licenses If an engineer has been in the code-first cohort for a year and their acceptance rate is dropping, revoke the license. They are either ignoring the tool or the codebase is too esoteric for basic autocomplete to help. Reallocate that seat to a junior engineer who will actually use the agent features to learn the stack. ### 4. Fix Your Context If a specific team shows low agent-first adoption, the problem might not be the engineers. It might be your codebase. LLM agents rely on good naming conventions, typed interfaces, and coherent file structures. If a codebase is a spaghetti mess of implicit state, the agent will hallucinate, and engineers will stop using it. Use the team filter to identify these dead zones and schedule a refactor. The GitHub Copilot usage metrics API gives you the ground truth. It strips away the marketing hype and shows you exactly how your organization writes software. Pull the data, run the analysis, and force your engineering culture to evolve.