Measuring AI Traffic: Which AI Bots Visit Your Store?

Why Measure AI Traffic?

When ChatGPT, Claude or Perplexity recommend your store, you won't see it in Google Analytics. AI assistants visit your website through dedicated bots – and those visits never appear as "traffic" in traditional analytics tools. Yet they're crucial: every bot visit signals that an AI platform is actively indexing your content.

Measuring AI traffic answers three core questions:

1. Am I visible? – If no AI bot crawls your site, your store doesn't exist for AI assistants.
2. Which platforms index me? – GPTBot (ChatGPT), ClaudeBot, PerplexityBot and others have different reach.
3. Are my GEO efforts working? – After creating an llms.txt, bot visits should increase.

Key AI Bots at a Glance

At least 13 AI bots regularly crawl the web today. The most important ones for store owners:

Especially relevant for e-commerce: GPTBot, ChatGPT-User, PerplexityBot and ClaudeBot – these platforms are actively used for product recommendations.

Identifying AI Bots in Your Logs

AI bots identify themselves via their User-Agent string in server logs. Every legitimate AI bot sends a unique name:

The simplest way to find AI bot visits is to search your Apache/Nginx access logs:

The problem: on shared hosting you often don't have access to raw logs. And even with access, manual analysis is tedious. That's where better methods come in.

3 Methods for Measuring AI Traffic

Method 1: Server Log Analysis (manual)

Suitable for developers with SSH access. You search access logs for bot user agents and aggregate the data. Pros: maximum control, no dependencies. Cons: time-consuming, no automatic monitoring, requires technical expertise.

Method 2: JavaScript Tracking Snippet

A small script on your website detects bot visits server-side and logs them. Most AI bots don't execute JavaScript, so detection must happen at the server level – via the User-Agent in the HTTP request. This method works on shared hosting too.

Method 3: Dedicated Bot Tracking Tool

Tools like our AI Bot Statistics dashboard collect, aggregate and visualise bot visits automatically. You see at a glance: which bots visit, how often, which pages get crawled, and how frequency trends over time.

Which Metrics Matter?

Not every bot visit is equally valuable. Focus on these key metrics:

Crawl frequency describes how often a bot visits your pages. Daily visits from GPTBot are a strong signal – your content matters to OpenAI. Weekly visits are normal. No visits at all means you're invisible to that platform.

Page coverage shows which URLs get crawled. If only your homepage is visited, the bot is missing your product pages. Ideally, the bot crawls product and category pages equally.

Response codes reveal technical issues. 200 codes are good. 403/429 codes mean your server or CDN is blocking the bot. 404 codes point to dead links in your sitemap or llms.txt.

Bot diversity measures how many different AI platforms crawl you. The more different bots visit, the broader your AI visibility. A store visited only by GPTBot is only visible to ChatGPT – not to Claude, Perplexity or Gemini.

Interpreting Your Results

Scenario 1: No AI bot visits at all

More common than you'd think. Possible causes: your robots.txt blocks AI crawlers, your website is too new or has too few external links, or your CDN/WAF blocks bots at the server level. First step: check your robots.txt.

Scenario 2: Only 1–2 bots visit sporadically

You're partially visible but not yet on every platform's radar. An llms.txt can be the breakthrough: it gives bots a clear structure of what to index. After creating an llms.txt, crawl frequency typically increases within 1–4 weeks.

Scenario 3: Multiple bots crawl regularly

Your GEO strategy is working. Now it's about optimising quality: Are the right pages linked? Are product details captured correctly? Check your Schema.org quality and make sure your llms.txt is up to date.

Measure AI Traffic Now

Instead of parsing server logs manually, use our free bot tracking:

From Measuring to Optimising

Measuring AI traffic is the first step. Full AI visibility comes from the complete GEO stack:

1. Measure – set up bot statistics, establish a baseline (→ Bot Tracking)
2. Audit – check robots.txt, Schema.org and existing llms.txt (→ AI Audit)
3. Optimise – generate llms.txt, complete Schema markup (→ Generator)
4. Distribute – actively submit to AI platforms (→ AI Push)
5. Monitor – check bot frequency and page coverage weekly