Analytics
Crawler analytics
Connect Cloudflare to see which AI bots crawl your site, how often, and what they fetched. The data lands in Bourd alongside your citations.
Citations tell you which sources AI models cite in answers. Crawler analytics tells you which pages AI models actually fetch. Bourd now surfaces both, by pulling crawler events from your Cloudflare account into a new workspace dashboard.
What’s new
- Cloudflare integration: Connect a workspace with a scoped API token. The first sync covers the last 7 days; updates hourly thereafter.
- Three-category breakdown: Traffic is split into user-triggered fetches (chatgpt-user, claude-user, perplexity-user…), search index builders (oai-searchbot, claude-searchbot…), and training crawlers (gptbot, claudebot, ccbot, applebot, and others).
- Per-bot time series: Daily request counts for every detected bot, on a shared axis.
- Period-over-period comparison: Total volume against the previous window of equal length, with presets for 7, 14, and 30 days plus arbitrary ranges.
Why this matters
Bourd already shows you which sources AI models cite in answers. Crawler analytics adds the upstream signal: which pages AI models pull from your site to generate those answers.
The two streams together tell you whether your content is being absorbed into training, surfaced through search and retrieval, or clicked through from live answers. Where the streams disagree is usually where the work lives.
How to use
Open Settings → Integrations in any workspace, click Connect Cloudflare, and paste a scoped API token. The first sync covers the last 7 days; the page populates within the hour.
Cloudflare’s AI Crawl Control is on Pro, Business, and Enterprise plans. Free plan retention is too short for this integration.
See the Crawler analytics guide for the full bot list, token scopes, category interpretation, and troubleshooting.