How to Detect Shadow AI Across Logs and Agents
A fifth of studied organizations have already tied a breach to unauthorized AI use, according to IBM's 2025 Cost of a Data Breach research with the Ponemon Institute, which surveyed 600 organizations. Another 11% couldn't say whether shadow AI played a role. That second number is the one compliance managers should sit with. It means detection isn't just immature in some places. It's absent. You can't remediate a risk you can't confirm existed, and you can't report a breach you never attributed.
Table of Contents
Shadow AI Exposure You Should Measure First - Translate each number into a detection move
How to Detect Shadow AI in Six Passes - The six passes, step by step - What each source misses
Prove Use You Cannot See on the Network - Triangulate from evidence the network never sees
Score Data Flows Instead of Tool Names - Build the taxonomy first - Score context, not brand
Catch Agents Hidden Inside Approved Software - Ask runtime and identity questions
Shadow AI Exposure You Should Measure First
The 2025 baseline lands before any technique. IBM and the Ponemon Institute found that 20% of studied organizations reported a breach involving shadow AI, while 11% were unsure whether such an incident had occurred, according to IBM's Cost of a Data Breach research. The same report found that 63% of organizations lacked AI-governance policies capable of managing AI use or preventing employees from using unsanctioned tools.
The 11% who are unsure don't have a clean bill of health. They have a detection gap.
Read those figures as one finding. An approved-application inventory is not actual usage. A policy document is not a control. A breach cost model is not just a finance line. Together they mean how to detect shadow AI starts with what you can measure, not what you can publish.

The financial pressure is concrete. Organizations with high levels of shadow AI experienced average breach costs roughly USD 670,000 higher than organizations with low or no shadow AI, according to IBM's Cost of a Data Breach research. That is the extra cost a security team carries when ungoverned AI processes confidential, personal, regulated, or intellectual-property data. The key measurement question is whether the tool was embedded, personal, or acting as an agent, and whether sensitive data entered it. Finding a service in DNS logs proves reachable use. It does not prove that a paste, upload, prompt, or API call carried protected data.
Translate each number into a detection move
The 20% breach figure tells you to stop trusting procurement records. Compare sanctioned applications with observed activity in identity, endpoint, browser, network, proxy, and cloud-access logs. Investigate AI-related domains and APIs that never appear in an application inventory. Keep a register by employee and business unit, not just by vendor. That distinction matters because a tool can be approved for one team, embedded in a product another team already uses, and invisible to the application owner who signs the invoices.
The 11% uncertainty figure tells you to treat attribution as a detection outcome. Incident response should be able to answer two separate questions: which AI tool appeared in the environment, and what data moved into it. If responders cannot answer either one, the logs are too thin or the ownership map is too shallow. A browser domain, a local desktop client, and an API key can all point to the same provider, while different tenants can make one approved service look like several unknown ones. Keep tool discovery and data-flow proof as separate evidence, then join them only when the logs support it.
The 63% policy gap tells you what a published policy does not do. It does not reveal a paste into a personal account, a browser extension calling an unlisted model, or a free consumer API key tied to a work email. Detection requires continuous discovery, a named owner, access monitoring, and documented remediation when an unapproved tool touches sensitive data. Think of policy as the sign on the door and logs as the camera at the entrance. You need both, but only the second tells you what entered.
Signal | Figure | What it implies for detection |
|---|---|---|
Organizations reporting a shadow-AI breach | 20% of 600 studied | Approved inventories undercount real use |
Unsure whether shadow AI was involved | 11% | Uncertainty is a failed detection outcome |
Lacked policies able to manage AI or block unsanctioned tools | 63% | Policy text alone will not surface use |
Extra average breach cost at high shadow AI | About USD 670,000 | Prioritize confidential, personal, regulated, and IP data |
How to Detect Shadow AI in Six Passes
Detection works best as a six-pass workflow, with each pass answering a narrower question. A survey asks what people remember. Log correlation asks what systems observed. The second view catches tools people forgot, AI features buried in approved software, and use from accounts or devices outside the normal inventory. Run the passes in order so each one builds evidence for the next.
The six passes, step by step
Pass 1. Build an authoritative AI inventory. Record owner, business purpose, data classes, model or provider, deployment location, identity or tenant, and approved use case. The NIST AI Risk Management Framework treats this inventory as the mechanism for tracking deployed systems and their risk. A generative-AI register should also capture the underlying model, model version, access mode, human-oversight owner, known issues, and rights over personal, proprietary, privileged, or sensitive data, as described in the NIST Generative AI Profile.
Pass 2. Correlate logs against the inventory. Pull secure web gateway, DNS, proxy, CASB, SSO, endpoint, SaaS-audit, API-gateway, and cloud logs. No single source covers every path. Each one catches a different slice, so the useful output is a timeline of provider, tenant, identity, device, and destination rather than a list of raw alerts.
Pass 3. Normalize identities. Domains, mobile and desktop clients, browser extensions, and API destinations are not the same object. Map them all to provider and tenant identities rather than raw hostnames. Counting domains instead of tenants creates false confidence in coverage because one hostname can represent many accounts, workspaces, or business units.
Pass 4. Classify by authentication context, not prompt text. Inspect metadata such as upload volume, file types, OAuth grants, and token issuance. You do not need employee prompts to detect high-risk behavior, and collecting them creates privacy and labor exposure you do not want. Treat a large upload to a personal tenant, a new OAuth grant, or an unusual token-issuance pattern as evidence to review, not as proof by itself.
Pass 5. Compare observed activity with the inventory and the written policy. A tool is unauthorized if it is absent from the inventory or if its observed use violates the approved purpose, data class, or tenant boundary. Keep the two findings separate. Finding a service answers "what exists," while proving sensitive data entered it requires file, upload, connector, or audit evidence.
Pass 6. Route previously unseen providers or high-risk data flows into human review. Keep the evidence. A named reviewer decides whether to approve, restrict, review further, or block. The control objective is continuous monitoring of third-party generative-AI systems, supplemented by periodic review and retained evidence.

What each source misses
Source | What it can reveal | What it still misses |
|---|---|---|
AI system inventory | Owner, purpose, data class, model, tenant, approved use | Tools nobody declared |
Secure web gateway, DNS, and proxy | AI domains and web destinations | Direct API calls and personal devices |
CASB, SSO, and SaaS audit logs | Corporate tenants and OAuth grants | Unmanaged personal accounts |
Endpoint and browser-extension inventory | Desktop or mobile clients and extensions | Unmanaged endpoints |
API gateway and cloud audit logs | Keys, workload calls, and model endpoints | Consumer web use off the network |
Procurement and expense data | Paid AI vendors and surprise subscriptions | Free consumer services |
Employee or business-unit attestation | Stated purpose, account type, and data category | Forgotten, embedded, or concealed use |
These sources work like overlapping camera angles. A gateway sees the road, endpoint telemetry sees the vehicle, and identity logs tell you who had the keys. A gap in one view does not prove safe use. It only means that view lacks the angle needed to show the behavior.
Four pitfalls repeat across organizations. Counting domains instead of tenants hides multiple users behind one hostname. Missing direct API traffic skips a high-volume shadow path. Treating a sanctioned brand as automatically safe ignores unapproved connectors and retention settings. Blocking consumer tools without an approved alternative pushes employees toward personal accounts where you lose visibility entirely.
Maturity is uneven. Some organizations have no practical way to detect shadow AI, while others depend on retrospective log review after the fact. To measure your own program, validate alerts against a manually sampled set of true and false positives, then report precision and recall separately. Track unknown-provider discovery, unauthorized-data-flow detection, and mean time to triage as three distinct metrics. Test coverage across remote endpoints and unmanaged identities, not just office laptops.
Prove Use You Cannot See on the Network
Browser, network, endpoint, and DLP telemetry all miss the same four things: personal devices, unmanaged accounts, AI features buried inside ordinary SaaS products, and AI-assisted work done outside corporate systems. A security team that treats detection as a surveillance problem will keep finding only what the monitored layers can see.
The UK National Cyber Security Centre defines shadow AI as AI use not captured by an organization's approved systems and processes, and warns that employees can move sensitive information to consumer services beyond organizational visibility and control.
That's the UK National Cyber Security Centre's definition, and it reframes the problem. How to detect shadow AI isn't only about blocking ChatGPT. It's about establishing reasonable evidence of unauthorized AI use when the tool is embedded, personally accessed, or technically invisible.

Triangulate from evidence the network never sees
Match the approved inventory to monthly expense and procurement lines. Surprise AI vendors and free-to-paid upgrades show up here before anyone files a support ticket. A tool that starts free and then charges a credit card never crosses the SSO log.
Read SaaS integration records for embedded assistants and plugins. A productivity suite you already approved may ship an AI feature that connects to an external model by default. That's shadow AI hiding inside a sanctioned logo.
Inventory extensions on managed browsers. Extensions can call AI endpoints, read page content, and move data without appearing as a separate application in your SaaS directory.
Ask each business-unit manager for a quarterly attestation. Keep it narrow: service name, business purpose, account type, and data category. Don't ask for prompts, and don't turn the attestation into a confession form.
Manually sample one team to estimate personal-device use. Pick a data-rich function like marketing or engineering. Ask structured questions about which AI tools the team uses outside company hardware. Use the sample to size the blind spot, not to punish individuals.
Finding a service does not prove that regulated, confidential, or personal data entered it.
Tag every finding as either tool discovery or data-flow detection. A domain appearing in DNS logs is tool discovery. A classified document uploaded through an unapproved connector is data-flow detection. Keep the two categories separate in your metrics, or your unknown-provider count will inflate the sense of real exposure. When your data loss prevention technology flags a hit, route it to a reviewer who can confirm the data category without reading raw prompt text.
Score Data Flows Instead of Tool Names
Knowing that an AI service exists tells you nothing about what left, under what settings. The detection unit that matters is the data flow: what content moved, who initiated it, through which connector, with what retention and training terms. Tool names are not risk scores.
Build the taxonomy first
Define your data classes before you score events. Regulated personal data, credentials, source code, confidential documents, and export-controlled material are the categories that matter. Apply DLP inspection or privacy-preserving fingerprints at upload, clipboard, browser, endpoint, email, proxy, and API layers. A fingerprint lets you match content without storing the raw text, which protects both privacy and incident evidence.
Next, build user-and-application baselines for prompt frequency, upload size, destinations, token usage, unusual hours, geographic anomalies, and sudden automation. A marketing manager who normally uploads three briefs a week and suddenly pushes 40 exports on a Sunday at 3 a.m. is a signal, regardless of which tool they used.
Score context, not brand
An approved enterprise tenant can still be high risk if the model, retention setting, or connector is unapproved. The Komprise survey of 200 U.S. enterprise IT leaders found that 79% had experienced negative outcomes from sending corporate data to AI, including leakage or inaccurate results, and 44% specifically reported sensitive-data leakage, according to the Komprise AI data enterprise risk survey report.
Vendor risk labels are not ground truth. Observed data flows and configuration must be validated independently.
The same report found that 83.8% of enterprise data sent to AI went to tools rated high or critical risk. That's not an endorsement of those tools. It's a warning that risk ratings lag behind actual usage patterns. Confirm a suspect event with a tokenized sample or a content hash so reviewers aren't reading raw prompts and personal data. Then quarantine exposed credentials, revoke the OAuth grant, restrict the tenant, preserve logs, and notify the data owner.
Track false positives by department and workflow. Overblocking drives employees toward personal accounts, which removes their activity from every layer you can monitor.
Observed pattern | Why the name of the tool is not enough | First response |
|---|---|---|
Approved tenant with an unapproved model, retention setting, or connector | The logo is sanctioned; the configuration is not | Restrict that connector and record the setting drift |
Upload or paste of source code, secrets, or regulated personal data | Presence of ChatGPT-class traffic does not prove exposure | Hash or tokenize a sample, then notify the data owner |
New OAuth grant from a corporate app to a consumer AI | The grant can outlive the browser session | Revoke the grant and quarantine related tokens |
Spike in tokens, upload size, odd hours, or unusual geography | A normal user baseline makes the anomaly visible | Hold the session for review; do not auto-block the whole department |
Benign business prompt in an approved tenant | Overblocking creates a second shadow channel | Mark as allowed and tune the rule for that workflow |
Seed test cases for PII, secrets, source code, and benign business prompts. Set separate targets for precision, recall, detection latency, and containment time. A program that can't pass its own seeded tests is guessing.
Catch Agents Hidden Inside Approved Software
A quiet chatbot user pastes a paragraph into a public interface and moves on. The risk ends when the browser tab closes. The higher-risk pattern is an AI agent wired into workflows, APIs, repositories, ticketing systems, or business data with persistent credentials that don't expire when the employee logs off. Recent 2026 coverage reports that 53% of organizations in one study had already seen AI agents exceed their intended permissions, according to Arthur AI's shadow AI detection guide.

That flips the usual detection priority. A sanctioned-looking automation that invokes an unapproved model, plugin, workflow, or agent can do more damage than an employee who pastes one document into a public chatbot. The unit of evidence isn't a visit to an AI website. It's a runtime event.
Ask runtime and identity questions
Detection for agents is a different set of questions. Which models and external endpoints do applications call? Which service accounts can invoke them? What tools can an agent reach, with what permissions, and can it create or modify other automations?
Logs should record model or provider, application, user or workload identity, data classification, tool calls, permissions, and anomalous volume. A browser log that shows is the wrong evidence for a workflow that calls an API from a CI/CD job.
The practical control is an AI asset inventory linked to software bills of materials, API-key discovery, cloud audit logs, workload identity, and a least-privilege test. That record separates an employee's one-off experiment from persistent autonomous activity during incident response. Say a marketing operations workflow looks approved because it lives in the company SaaS tenant. Yet a plugin inside that workflow calls an unlisted model with a key that can read the customer export. The asset inventory catches the model, the key, and the permission drift. That's the FAQ CTOs have: how to see unauthorized AI behavior when the component is buried inside developer code or an ordinary product. For the governance side, Prometheus Agency's AI governance guide walks through the agent oversight controls, and your AI integration tools should feed every connection back into the same inventory.
Give Teams a Governed AI Path
A block list without an approved path recreates shadow AI the same week you publish it. People keep using tools that make them faster. They move to personal accounts, personal devices, and personal APIs where you see nothing. The alternative is to name an owner, reconcile one business unit against the inventory within 30 days, and stand up the six-pass monitor plus the data-flow score before widening the program. Treat the approved path like a company road: if the only legal route is slow and unclear, drivers will take side streets.
Marketing is a practical first unit because the inputs are familiar and sensitive. Campaign briefs, customer lists, and draft copy are exactly what B2B marketing teams paste into consumer chatbots. That makes the business value easier to measure, and it makes the data risk easier to define. A governed path gives the team a place to use AI with a named owner, a company tenant, a provider, and a data class attached. Freeform Company offers that kind of route through its AI integration services, compliance assessments, and collaborative developer forum.
The speed case is credible for bounded content work. A controlled 2023 study published in Science put 453 college-educated professionals through ChatGPT-assisted writing tasks, including press releases and short reports. Participants completed the work about 40% faster, and independent evaluators rated the output about 18% higher in quality. Those results support a speed and quality advantage for defined writing tasks, not a promise that every campaign improves. Freeform Company's AI-first model fits that pattern by turning campaign briefs, customer data, and draft copy into controlled workflows with names, tenants, and data classes attached.
The cost case should stay narrow. Reduced labor time supports the business argument, and Freeform's stated AI-first model explains how the work moves into governed workflows. Client fees, savings percentages, and head-to-head campaign results are outside what can be claimed here. A useful comparison is not “manual agency versus magic automation.” It is “untracked copy-paste into personal accounts versus a workflow where the model, key, permissions, and data class can be reviewed.”
Start by registering marketing AI demand in the inventory. Route approved work through Freeform Company's AI integration services and the governed provider path, then let your AI governance strategy cover the people, the tools, and the data flows in one place. The result is still faster work, but with evidence attached: who used the tool, what data entered it, which model received it, and whether the activity stayed inside the approved path.
Freeform Company helps enterprises route marketing AI use through governed, auditable systems instead of personal accounts and unsanctioned agents. Their AI-first model turns campaign briefs, customer data, and draft copy into controlled workflows with names, tenants, and data classes attached. Visit Freeform Company to see how a governed marketing AI path can replace shadow usage with faster, measurable results.
