> ## Documentation Index
> Fetch the complete documentation index at: https://agent-traffic.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Dataset

> The JSON files behind the classifier.

The dataset is plain JSON, so you can use it from any language or SQL engine. Import it from the package or read it from the repository.

```js theme={null}
import { agents, httpClients, headers } from '@mintlify/agent-traffic'
import agentsJson from '@mintlify/agent-traffic/data/agents.json' with { type: 'json' }
```

| File | Contents |
| - | - |
| [`agents.json`](https://github.com/mintlify/agent-traffic/blob/main/data/agents.json) | Known agents, AI crawlers, search crawlers and named bots |
| [`http-clients.json`](https://github.com/mintlify/agent-traffic/blob/main/data/http-clients.json) | Generic HTTP libraries, native-app HTTP stacks and headless browsers |
| [`headers.json`](https://github.com/mintlify/agent-traffic/blob/main/data/headers.json) | What each request header tells you and who controls it |

## Agent entries

```json theme={null}
{
  "id": "gptbot",
  "name": "GPTBot",
  "operator": "OpenAI",
  "category": "ai-crawler",
  "kind": "training",
  "runsOn": "operator",
  "userAgents": ["\\bGPTBot/"],
  "verify": { "ipRanges": ["https://openai.com/gptbot.json"] },
  "docs": "https://platform.openai.com/docs/bots",
  "examples": ["Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.4; +https://openai.com/gptbot)"],
  "lastVerified": "2026-10-23"
}
```

**id** `string` *required*

Stable identifier, returned as `id` by `classify`.

**category** `string` *required*

`agent`, `ai-crawler`, `crawler` or `bot`.

**kind** `string` *required*

Detail within the category.

**runsOn** `'operator' | 'user'` *required*

Where requests come from. `user` entries are never IP-checked.

**userAgents** `string[]` *required*

Case-insensitive regular expressions anchored on the product token. A `(?<version>…)` group fills `version`.

**accept** `string[]`

Exact `Accept` strings the product sends.

**signatureAgent** `string[]`

Hosts named in the product's `Signature-Agent` header.

**verify** `object`

Operator-published sources: `ipRanges` URLs, `reverseDns` domains and a `webBotAuth` key directory.

**examples** `string[]` *required*

Real user agents. The tests require each to classify as its own entry and match no other.

**notes** `string`

Caveats, such as shared identities.

**lastVerified** `string` *required*

The date the entry was last confirmed.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.