9 min read - Copilot BYOK Metrics: Reconcile the Gaps Before You Govern

GitHub added bring-your-own-key (BYOK) and enterprise custom-model data to the Copilot usage API on 3 November 2026. An engineering leader can now see, separately from Copilot-hosted usage, personal-key and local-model requests made in Copilot CLI and the Copilot app, and enterprise custom-model requests from every client, including IDEs. Existing totals are unchanged and still include that usage.

The new fields solve an attribution problem, but they do not create a complete AI usage ledger. GitHub says local BYOK figures rely on client telemetry, can undercount when telemetry is disabled and may miss tokens when a local provider does not report them. A report can therefore be accurate about what it observed while remaining incomplete about what happened.

Security and finance teams need to treat the API as one record in a reconciliation, not as the sole inventory or bill.

What GitHub added, and where the boundary sits

GitHub's 3 November release note separates two paths.

Local BYOK covers a developer's provider key or a model served through tools such as Ollama, LM Studio, Foundry Local or llama-server. The new reports can indicate that local BYOK was used and break requests, sessions and reported tokens down by BYOK kind and by Copilot CLI or Copilot app; IDEs are not covered yet. They do not identify the local model by name.

Enterprise custom models take a different route. Administrators configure them in GitHub settings, and requests pass through GitHub to the provider. GitHub marks these model entries as custom and observes their request and token counts on the server, including requests from IDEs. The same model name can appear once as Copilot-hosted and again as an enterprise custom model.

Those paths have different evidence quality. GitHub describes local BYOK counts as a lower bound because client telemetry may be off and some local providers omit token usage. Enterprise custom-model counts do not have that client-telemetry limitation. The custom-model flag is not backfilled: report days before the release do not include is_custom_enterprise_model. Unrecognised custom model names can appear as unknown.

The broader Copilot metrics documentation adds further limits. Most detailed metrics depend on client telemetry. Network rules, proxies and client settings can stop those events, while server-side signals may still count a person as active. In that case, feature, language and lines-of-code breakdowns can remain empty, so an empty breakdown can mean “not observed” rather than “not used.” Some surfaces, such as Copilot Chat on GitHub.com and GitHub Mobile, are excluded from the data, and individual agent-app metrics appear only in the APIs and NDJSON exports, not in the usage metrics dashboards.

Keep four records instead of one dashboard

A defensible view of developer AI needs four records that answer different questions.

Do not force these records into false agreement. A developer can hold a Copilot seat and never use it. A local model can run without a provider invoice. A provider bill can include requests made outside Copilot. A high request count can still produce no accepted change.

The inventory should preserve those differences. NIST's voluntary Generative AI Profile suggests (action GV-1.6-003) that inventory entries consider, among other items, underlying foundation models, their versions and access modes, human oversight roles and responsibilities, and special rights covering intellectual property, licensed works and personal, privileged, proprietary or sensitive data. It does not say that a vendor telemetry feed is the inventory. The Copilot usage API now reports all three paths, but its local BYOK counts cover only Copilot CLI and the Copilot app and remain a lower bound, so it cannot serve as the inventory on its own.

Define a small event contract

Reconciliation fails when every source names the same activity differently. Agree on a minimal event contract before building a dashboard:

• time window and a pseudonymous actor or team identifier;
• client surface, provider, requested model and returned model when available;
• access path: Copilot-hosted, enterprise custom, personal key or local endpoint;
• request status, input and output tokens when the source supplies them;
• repository or work-class label, policy decision and a correlation identifier;
• source system, collection method and a field that marks estimated or missing data.

Keep prompt bodies, generated code and tool arguments out of the default usage record. They can contain source code, personal data, credentials or client information, and most governance questions do not require them. Store only the minimum identifiers and counts needed for the stated purpose, then set access and retention accordingly.

The OpenTelemetry generative-AI attribute registry offers common names for provider, operation, requested model, response model and token usage. The registry now lists these attributes as moved to a dedicated GenAI semantic-conventions repository and still marks well-known operation and provider values as Development, so pin the convention version or commit you adopt rather than assuming every library emits identical fields. The registry also warns that message and tool-call content can be sensitive. Standard names help with joins; they do not justify collecting the content itself.

Reconcile weekly, with explicit uncertainty

Start with one engineering group and a fixed seven-day window. Export Copilot usage, seat and policy records. Pull provider usage and billing data for managed keys, and collect aggregate counters from any approved local gateway. Then link the records by time, team and access path, keeping the original source values.

Classify every mismatch instead of silently filling it:

Label the reconciliation coverage. For example, a team can report that enterprise custom-model requests were observed server-side, local BYOK activity is a lower bound, and provider costs cover only managed keys. That statement is less tidy than a single total, but it tells a decision-maker what the number can support.

Set a review trigger for unexplained variance. The threshold should reflect the decision: finance may care about material unallocated spend, while security may investigate any unapproved provider or access path. This article does not supply a universal percentage because the sources do not establish one.

Do not confuse consumption with value

Requests and tokens describe consumption. They do not show whether code passed tests, survived review or reduced delivery time. GitHub's documentation separates usage metrics from impact and code-generation views, and even those measures require careful interpretation.

Connect the reconciled usage record to a small set of work outcomes: accepted pull requests, reviewer rework, escaped defects, elapsed time and successful rollback when needed. Compare similar task classes rather than ranking individual developers by tokens. Providers can count tokens differently, some local providers report none, and a longer session may reflect a harder task rather than waste.

Our cost-per-accepted-task guide explains how to include review and rework instead of stopping at API price. The agent inventory guide covers ownership and access classification.

A two-week implementation sequence

During the first three days, list Copilot seats, approved BYOK paths, enterprise custom models, provider accounts and local endpoints. Record the owner of each key or model server without copying the secret.

For the rest of week one, export all four records for a pilot team. Test one known request through each approved path and verify where it appears. Disable client telemetry in a test profile, where policy permits, to confirm how missing detail is represented. Do not use a production developer's settings for that experiment.

In week two, implement the event contract and mismatch table. Review unexplained provider use, unknown model names, inactive seats and records with no work outcome. Finish with a signed decision: expand collection, fix a gap, narrow the allowed paths or keep the pilot running.

The deliverable is a record of which activity was observed, which costs were allocated, which uses remain outside coverage and who owns the next check.

Explore Exceev's AI and engineering services if your team needs to map developer-AI access paths, instrument a pilot or connect usage to reviewed delivery outcomes.

Sources and limitations

We opened and reviewed every source below on 5 or 6 November 2026.

• GitHub's BYOK and enterprise custom-model release note is the primary source for the 3 November change, field coverage, the custom-model flag's missing backfill, model-name handling and stated telemetry limits.
• GitHub's Copilot usage metrics concept guide describes included surfaces, client and server telemetry, missing-detail behaviour and the separation between dashboards and exports.
• GitHub's usage-data reference defines current dashboard, API and export fields. Product coverage and schemas can change.
• NIST's Generative AI Profile supplies independent, voluntary guidance on AI inventories. It does not evaluate GitHub Copilot or require this reconciliation design.
• The OpenTelemetry GenAI registry supplies sensitivity warnings and points to the GenAI semantic-conventions repository, which now holds the field definitions. The conventions are still changing, so implementers should verify and pin the version they adopt.

We did not access a Copilot enterprise tenant, provider invoice or local-model gateway, and we did not test the new fields. The four-record model, mismatch checks and two-week sequence are Exceev's analysis, not GitHub, NIST or OpenTelemetry requirements. This article does not make a security, privacy, compliance, cost-saving or productivity guarantee.