10 min read - GitHub AI Scan: widen PR coverage without lowering the bar

GitHub changed where its AI Scan can run on 16 September 2026. The pull-request scanner no longer depends on CodeQL default setup, so teams can apply it to eligible repositories where CodeQL does not cover the language or framework. That removes a setup dependency. It does not turn an AI finding into proof of a vulnerability or a safe merge.

GitHub's own documentation draws the boundary clearly. AI Scan is a public preview, its findings are advisory, and they cannot be required by a ruleset. It scans qualifying pull-request changes rather than the full repository, excludes pull requests from forks and Dependabot, and may report false positives. A sensible adoption plan therefore asks a more useful question than "should we switch it on?" The question is which coverage gap it can test, who will validate the results, and which existing control remains authoritative.

This article proposes a pilot for technology leaders and engineering teams. It is not a report of tests on GitHub's service or an Exceev client environment. Product eligibility, billing and behaviour may change during the preview.

Start with the coverage gap

The 16 September GitHub announcement says AI Scan can now run when CodeQL default setup is absent. Code scanning and AI Scan must still be enabled at the appropriate repository, organization or enterprise level. The release applies to eligible organization-owned and personal repositories on GitHub.com for GitHub Advanced Security customers; GitHub Enterprise Server is outside its stated scope.

The detailed AI Scan documentation adds important conditions. During the preview, the feature requires both GitHub Advanced Security and GitHub Copilot licences, and usage consumes AI credits. It targets gaps in CodeQL coverage, including PHP, Shell or Bash, Terraform configuration, Dockerfiles, JSP and Blazor. The supported set may change.

Those facts point to a narrow first use. Inventory repositories by language, framework, exposure and current security checks. Then identify pull-request code that receives little or no static security analysis today. A mixed application with Terraform, deployment scripts and a PHP service may have a clear gap. A repository already covered by tuned CodeQL queries, another SAST tool and specialist review needs a different justification.

Do not count repositories alone. Record the files and changes that the scanner can examine. AI Scan does not process a full repository scan, fork pull requests or Dependabot pull requests. Those exclusions may leave an important contribution or dependency path untouched even when the feature is enabled.

Keep detection separate from enforcement

GitHub says AI Scan starts on pull-request creation and after new commits when the repository and effective setting are eligible. It works directly from the pull-request code, can search the repository for context, and does not rely on custom instruction files. Its findings appear in the pull request with an AI label and may include a suggested fix.

That workflow is useful for discovery. It is not a merge policy. The documentation says findings are advisory and cannot currently act as required ruleset checks. They also do not appear as backlog alerts in the repository security view. Trying to simulate a hard gate around a preview result can create a brittle process without solving the missing backlog or coverage problem.

NIST's Secure Software Development Framework supports a risk-based choice of practices rather than a universal checklist. It asks teams to align security work with business needs, risk tolerance, cost, feasibility and dependencies between practices. NIST's separate developer verification guidance lists static scanning alongside threat modelling, automated testing, hardcoded-secret checks, structural tests, fuzzing and other techniques. The document explicitly says its recommendations do not cover all software verification.

Use that combination as the operating model. AI Scan can add a signal on a changed Dockerfile or Terraform module. It does not confirm the application's authorization design, test a deployed service or inspect every dependency update. Keep the controls that cover those jobs.

Run a bounded pilot

Choose a small set of repositories with a known gap and reviewers who can read the affected code. Avoid an organization-wide rollout until the team understands which pull requests trigger scans, how findings arrive, how much review they require and what the credit consumption looks like.

A useful pilot record fits on one page per repository:

• repository, language and framework;
• security-sensitive paths and the current review owner;
• existing scanners and their scope;
• AI Scan enablement level and date;
• excluded PR paths, including forks and Dependabot;
• measured usage, triage effort and disposition of each finding.

Start with ordinary work rather than a synthetic vulnerability contest. Select repositories that receive representative pull requests and have no production incident under way. Keep the same tests and review rules in place, then observe AI Scan for a fixed number of eligible pull requests or a fixed review period. The team should agree on that boundary before enabling the feature.

For each finding, capture the affected commit, file and line, the claimed weakness, the evidence used to validate it, the reviewer, the final disposition and any fix. Useful dispositions include confirmed, plausible but unproven, false positive, duplicate, and out of scope. Do not dismiss a finding solely because the proposed patch fails or accept it solely because the explanation sounds specific.

The OWASP Secure Code Review Cheat Sheet explains why this validation matters. OWASP treats manual review as a complement to automated testing, with people focusing on business logic, complex security controls and contextual flaws. It also recommends human filtering of false positives and using tool output to guide deeper review.

Validate the claim, then the fix

A scanner finding is a hypothesis about a code path. Reconstruct the path before editing code. Find the input, the trust boundary, any validation or encoding step, the sensitive operation and the conditions required to reach it. For an authorization issue, check enforcement in middleware, framework policy and downstream services rather than reading the changed function in isolation.

Next, write the smallest test that would fail if the weakness were real. That may be a unit test for an unsafe parser, an integration test for an access-control path, or a policy test for Terraform. Run the existing deterministic scanners as well. A disagreement between tools is a reason to inspect the assumptions, not a vote where the majority wins.

Review any suggested remediation as new, untrusted code. GitHub notes that most findings include a suggestion, not that every suggestion is correct. Check whether the change closes the reported path, creates a compatibility problem, weakens error handling or moves the risk elsewhere. Re-run the targeted test and the repository's normal suite.

Sensitive findings need an incident decision too. If a pull request contains an exposed credential, removing the string from the diff may not be enough; the owner must determine whether to revoke or rotate it based on its actual exposure. If the finding suggests an exploitable flaw in deployed code, route it through the incident and vulnerability process rather than leaving the discussion in a pull-request thread.

Measure signal and operating cost

The first pilot should answer whether AI Scan adds useful, reviewable coverage. Track eligible pull requests, scans observed, findings, confirmed findings, false positives, duplicates and findings that existing controls already caught. Record reviewer time and AI credit use from the available billing records.

Avoid a target such as "zero findings". A silent scanner may have found nothing, may not have run, or may not cover the code that changed. Check execution evidence first. Likewise, a large finding count can mean useful discovery or noisy rules. The disposition record separates those cases.

Look for repeated weakness types. If several confirmed findings involve the same unsafe shell construction, fix the shared helper or coding standard and add a deterministic check where possible. The long-term gain comes from preventing recurrence, not asking reviewers to rediscover the same pattern in every pull request.

Expansion should require evidence that the intended pull requests triggered scans, reviewers could validate the results within the team's normal flow, useful findings justified the licence and credit cost, and exclusions have another control. Pause or narrow the pilot if ownership is unclear, the queue grows without triage, or teams start treating an absent finding as approval.

Make the decision reversible

Document the enterprise, organization and repository settings that produce the effective state. GitHub's permission hierarchy still applies, and the feature is not allowed at enterprise level by default. Name who may enable it, which repositories may opt out and who reviews usage. Keep a link to the current documentation beside the decision because preview terms and supported languages can move.

Set a review date and an exit condition. A repository may leave the pilot when the feature becomes too noisy, the language gains stronger deterministic coverage, costs exceed the value observed, or the product's data handling no longer fits the organization's requirements. Before enabling a cloud analysis service on sensitive source code, the accountable owner should also review the current product terms, privacy documentation and contractual permissions. This article does not establish that approval.

Teams that need a broader view of engineering controls can connect this pilot to their AI-assisted development guardrails and GitHub Actions trust-boundary review. Product, delivery and governance questions outside the repository belong in a separate services discussion.

Sources and limitations

We opened and reviewed every source below on 17 September 2026.

• GitHub's 16 September announcement establishes the removal of the CodeQL default-setup dependency, the public-preview scope and the GitHub Enterprise Server exclusion.
• GitHub's AI Scan documentation defines eligibility, operation, licences, billing, supported examples, findings and current limitations.
• NIST's Secure Software Development Framework supplies independent guidance for choosing and combining secure-development practices according to risk and business needs.
• NIST IR 8397 describes static scanning as one of several developer-verification techniques.
• The OWASP Secure Code Review Cheat Sheet explains how manual review and false-positive validation complement automated tools.

GitHub is the primary source for its preview service. NIST and OWASP do not certify GitHub AI Scan or this proposed pilot. We have not enabled the feature, measured its detection quality, audited its models or assessed a customer's licence, privacy or contractual position. Verify current product documentation and observed repository behaviour before changing a security process.