Traffic log import

30 days of HTTP request logs from your CDN or origin, one row per request. Each row becomes a classified crawler visit with verification status, content context, and policy outcome.

Where do I find this file?

Use this section when asking a data, analytics, CMS, or engineering teammate for the right export.

Common places to get it

Cloudflare HTTP request log export (transform to the columns below)
Akamai, Fastly, or origin/nginx/apache access log export
Any CDN or server log you can map to the generic CSV format

Do not upload your article catalog here (that is step 1).

Required columns: timestamp, userAgent, method, host, path, statusCode

Optional columns: ip, asn, reverseDns, bytesSent, referer

Download test CSV: traffic-logs-example.csv

Expected CSV format

Download sample CSV

timestamp,ip,asn,reverseDns,userAgent,method,host,path,statusCode,bytesSent,referer
2026-06-16T10:04:22Z,40.83.2.71,8075,crawler.openai.com,"Mozilla/5.0; compatible; GPTBot/1.3; +https://openai.com/gptbot",GET,publisher.com,/premium/banking-analysis,200,182001,

Required columns: timestamp, userAgent, method, host, path, statusCode. Raw IP addresses are hashed during import and are not stored.