Docs As Code
FeaturesLong read

How to Tell Agent Visits From Human Visits in Docs Analytics

Layer headers, behavior, networks, and logs to spot agents.

· 9 min read
Cover illustration for “How to Tell Agent Visits From Human Visits in Docs Analytics”
Features · November 13, 2026 · 9 min read · 1,933 words

Telling agent visits from human visits in documentation analytics takes more than a bot filter. It requires layered evidence from headers, behavior, networks, and server logs, with room for uncertainty. The aim is not simply to remove automated traffic from human metrics, but to understand which readers and machines use the docs, and what a successful interaction means for each.

Why documentation analytics are misreading traffic in the agent era

On Monday morning, a documentation dashboard can present a puzzling combination of flat organic traffic, short visits, and missing referrals, even while the content appears in AI-generated answers. Has readership stalled, or has the route between reading and referral changed? The dashboard cannot reliably answer that question alone. Its measures were built around human visits, so they can record machine activity without explaining what it accomplished.

Consider an agent that lands on a reference page, extracts the information it needs, and leaves in two seconds. Conventional analytics can register a bounce, although the interaction completed its objective. Volume creates a second distortion. ClaudeBot crawls approximately 23,951 pages for every single referral it returns, illustrating how far consumption can diverge from attributed visits. Even if human use stays the same, the machines reading the docs can change how the traffic mix looks.

These distortions affect more than traffic reporting. If brief visits mean failure in one segment and successful extraction in another, their combined average has no stable interpretation. A team could think useful documentation was failing to help its readers. Before deciding whether a page performs well, the analysis needs to establish who (or what) used it.

Why agents and humans leave structurally different traces

An agent is not merely a faster human. Human documentation visits are exploratory and iterative, shaped by what a reader already knows and what they discover along the way. An agent acts autonomously after receiving an instruction, with success defined by completing that task. Its visit may therefore finish in a single traversal rather than a cycle of reading, reconsidering, and returning.

That difference changes the baseline for interpreting activity. Agents can operate continuously rather than intermittently, so tools calibrated against human activity may miss their patterns or flag them as anomalies at scale. Unusual no longer reliably means unwanted or unsuccessful. A visitor’s activity should be assessed against the kind of actor it is.

What user-agent strings tell you and where they stop being reliable

User-agent strings offer the most direct starting point because some agents announce their identity. Teams can capture those declarations and compare them with published agent registries. Published IP ranges and reverse-DNS verification let teams check the claim in the string against other evidence. The distinction is between a claim and corroboration. An identity presented in a header is useful evidence, but it is not automatically proof.

Spoofing a user-agent string is straightforward, and an agent can present as a standard browser while leaving other fingerprints intact. That creates a contradiction to investigate. FP-Agent research from UC Davis found that browser fingerprints had limited discriminative power when multiple AI browsing agents shared them.

Teams should build a user-agent allowlist from published registries and use verified matches to identify known agents. Unmatched traffic still needs further scoring. Otherwise, the system quietly turns “not recognized” into “human,” contaminating the very segment it was meant to clean.

Behavioral signals that separate agent sessions from human ones

Behavior provides the strongest discriminating layer in the supplied research. FP-Agent used an instrumented honeypot site to compare seven AI browsing agents with humans who carried out flight booking, shopping, and forum tasks. Its multi-class classifier detected all seven agents, while Cloudflare’s bot detection caught only one. The controlled setting is important to the interpretation. It supports using behavioral fingerprints, but it does not establish that every documentation session can be classified with the same certainty.

For documentation teams, the practical application is observable proxies. Session evolution is one such proxy. The speed at which humans navigate and their reading patterns change gradually, while agent signals can stay the same or change mechanically from one action to the next. Teams can also compare mouse movement, because synthetic cursor paths differ from human paths in ways they can measure when agents click elements placed at random. These patterns provide evidence even when browser identities match.

Reading rhythm makes the distinction more concrete. Humans pause, scroll back, and reread, while agents may traverse a page directionally and completely or jump to a needed anchor. Time on page gives that behavior context. A two-second visit to a lengthy reference page is implausible as a human reading session, but normal as an agent extraction. It is a clue about the visit that requires corroboration to establish the visitor’s identity.

An agent may follow the documentation hierarchy from API reference to endpoint to parameter, while a human takes a messier route shaped by search terms and prior knowledge. How consistent is that path across the session? A hierarchical route becomes more informative when paired with mechanical timing and directional scrolling. Teams can operationalize these observations in their existing analytics or log pipeline without treating any one behavior as decisive.

Network-level and upstream-ratio signals that complement behavioral data

Network evidence provides an independent cross-check when behavioral instrumentation is sparse. A headless-browser user-agent string originating from a published cloud provider’s egress range is a strong agent signal. The same string from a residential IP is more consistent with a scraper or a human using unusual tooling. Neither origin settles the classification on its own. Its value comes from how it strengthens or contradicts the other evidence.

Request cadence can expose a different mismatch with human reading. Evenly spaced requests across many pages, without pauses longer than a few hundred milliseconds, differ structurally from intermittent human activity. Upstream-to-downstream ratios offer a further cross-check because agents can produce large upstream payloads tied to task execution. Those ratios help classify traffic, but there is no single cutoff that works for every case.

HUMAN Security’s trust classification model combines network properties, behavior, and cryptographic verification. When HTTP Message Signatures and a public key registry are available, they can be used to verify high-trust agents on each request. For traffic without such verification, user-agent, behavioral, and network evidence should produce a confidence score.

The access paths that bypass page-view analytics entirely

Even layered session scoring has a blind spot. What happens when documentation is read without a browser visit at all? MCP servers and direct file fetches are access paths that most documentation teams have not yet instrumented. These interactions can bypass conventional page-view analytics entirely, leaving no session for a behavioral scoring system to examine.

Direct file access includes llms-full.txt. It embeds the content of every linked page in one file, allowing an agent to ingest an entire documentation corpus with a single request. Server logs may show one file fetch for what would be dozens of page views. Comparing that count directly with human page views would introduce another misleading equivalence.

Cash App developer documentation exposes /llms.txt and supports Markdown versions of pages, making direct machine access intentional and loggable. JavaScript-instrumented analytics alone cannot show when those files are read. Teams therefore need separate visibility into these endpoints. Server-log analysis is not an optional supplement if the goal is complete segmentation, because some documentation use never enters the browser-based measurement system.

Why agent traffic is not a single segment and collapsing it creates new errors

What is the agent trying to accomplish? An IDE coding agent reading API reference to write integration code represents a task-oriented use of the documentation. A training crawler that collects content raises governance questions without helping a user right away. An answer-engine agent may bring the content to more people without ever sending a human visitor to the docs.

Those differences cannot reliably be inferred from user-agent identity alone. Knowing what an agent is trying to do also cannot tell teams how much to trust its identity. HUMAN’s model distinguishes high-trust agents with cryptographic verification and no abuse signals, medium-trust agents with more easily spoofed identity signals, and unknown agents requiring ongoing evaluation. Purpose and trust answer different analytical questions. A useful segmentation preserves both.

The distinction also has a governance consequence. AI systems do not understand which content is sensitive unless restrictions are enforced, so putting all agent access in one group can hide cases where regulated, strategic, or access-controlled material is exposed. The practical model has three buckets, which are confirmed human, classified agent by type, and unresolved, with trust distinctions retained within the agent view.

Building a working segmentation from layered signals

Diagram: Four Layers of Agent-Traffic Segmentation. Visualizes: Show a stepped four-layer flow illustrating how a documentation visit is classified from raw signal to final bucket.

The first layer captures user-agent and request headers, checks published agent registries, and compares IP origins with published ranges. Agents cryptographically verified using HTTP Message Signatures can go straight into the confirmed-agent bucket. Unmatched sessions remain candidates for further examination.

The second layer scores how a session behaves in light of the documentation it accesses. Relevant inputs include time on page relative to content length, navigation path shape, scrolling, mouse patterns, and request cadence. The third layer uses network evidence, upstream-to-downstream ratios, and request timing to cross-check sessions that are hard to classify. Contradictions should remain visible in the resulting score. Traffic with a browser-like identity and mechanical behavior should not be counted as human just because its header looked familiar.

The fourth layer extends the inventory beyond browser sessions. Teams should instrument /llms.txt, /llms-full.txt, and raw Markdown endpoint fetches separately in server logs, so those reads appear in the agent analysis. The resulting classification needs an explicit confidence threshold. Sessions below it go in the unresolved bucket, where they stay out of both human and agent reporting.

The dashboard can then expose three parallel views. A primary human-traffic KPI view excludes confirmed and high-confidence agent sessions, while an agent view separates traffic by type and a third tracks unresolved sessions over time. HUMAN’s Sightline provides a reference implementation through its AI Agents Monitoring Dashboard, which tracks volume, top targeted routes, trust classification, and visitor sessions. The segmentation still needs revision as implementations evolve. New agents may publish identity signals, or leave teams to interpret behavior and network evidence instead.

What readable agent traffic makes possible for documentation teams

Separating traffic creates two feedback loops with different definitions of success. In the human loop, time on page relative to content length, return visits, and search queries become interpretable without dilution from machine extraction. The agent loop asks a different set of questions. Which reference pages receive visits, which endpoints follow them, and which machine-readable files are fetched? Those patterns show which parts of the documentation are used in AI-driven workflows.

This changes how documentation defects should be understood. Heavy agent traffic to an API reference page can signal use by coding agents, making gaps or outdated examples both a risk to generated integration code and a frustrating reading experience. A stale sample can propagate errors into AI-assisted integrations built against it. The most useful measurement is not simply that machines arrived, but what they consumed and where that content could fail them.

For documentation teams, the corresponding action is to support both reliable access and reliable content. Structured headings, machine-readable formats, exposed llms.txt endpoints, and CI/CD-integrated update workflows help agents find, parse, and use documentation that stays in sync with product changes. These changes help with both agent-readability and SEO. Once machine consumption is visible, a short visit stops being an automatic failure, and a page without referrals stops being an automatic dead end. Teams still need to assess whether the documentation helped its particular visitor complete the task correctly.

Sources

  1. FP-Agent: Fingerprinting AI Browsing Agents
  2. AI Traffic Has Outgrown Network Security Telemetry

More in Features