Do AI Crawlers Visit Evenly Every Day? 1,052 Requests, Then Another 1,101

This article was first published in Korean by Intrix Lab on August 31, 2026.

AI crawler A did not visit evenly every day. In intrix.kr server logs, the first large collection totaled 1,052 requests and the second totaled 1,101, with a gap of 18–21 days between them. These findings cover about 3 months through 2026-08-31. Summarizing this activity with a single average request count hides the actual collection pattern: intermittent bursts.

How did the two bursts unfold?

We counted all requests from crawler A, operated by an external platform, from 2026-06-02 through 2026-08-31. We also checked whether requests impersonating the same bot name were mixed in. Total requests during this period numbered 2,205.

The first burst totaled 1,052 requests from 2026-08-04 through 08-09. Daily requests rose through counts of 1, 9, 44, 180, 586, and 232. No large collection appeared for the following 18–21 days.

The second burst totaled 1,101 requests from 2026-08-27 through 08-31. Daily counts were 1, 16, 30, 747, and 307. The final 307 requests were a running total for 2026-08-31, so we did not interpret that figure in the same way as a finalized full-day total.

What supports treating these as successive visits by the same crawler?

Requests on 2026-08-30 through 08-31 were distributed across 88–98 IPv6 addresses. Each IP requested 1–3 pages. Rather than one address crawling the site sequentially, multiple addresses each retrieved a small number of pages.

We also broke down requests by page type. Over the two days, there were 710 requests for other pages, 105 for hubs, 102 for company information, 39 for the home page, and 6 for blog pages. Repeated requests for a single article did not inflate the total.

We did not establish authenticity from the bot name alone. There were 10 requests classified as impersonation, or 0.5% of all requests.

Why does an average hide this pattern?

Dividing the total for about 3 months by the number of days makes it look as though collection happened a little every day. The actual logs showed a long gap after 2026-08-09, followed by 747 requests concentrated on 2026-08-30. The same total can mean different things depending on when collection occurs.

This difference matters particularly in before-and-after comparisons. Even if requests increase immediately after a site change, without a comparison period we cannot separate an effect of the change from a burst that would have happened anyway. These findings support only the measurement rule that crawl requests should be recorded as a daily time series.

What does 0 requests for robots.txt mean?

Of the 2,205 requests sent by crawler A, 0 were for robots.txt. That figure alone, however, does not establish that the crawler ignored the rules. Server logs alone cannot rule out the possibility that it cached previously retrieved rules or checked them through another route.

This article therefore does not claim a violation. It records only that no repeat requests for robots.txt were found among requests under this name that reached the origin server over about 3 months. Testing compliance would require a separate intervention design with a disallowed path.

How far do these findings apply?

The population consists of a single site, intrix.kr, and one crawler, crawler A. The period covers about 3 months, from 2026-06-02 through 2026-08-31, with only two bursts recorded. There is no guarantee that other sites or crawlers will show the same intervals and request volumes.

In particular, the 18–21 days between the two bursts cannot be called a cycle. There were only two occurrences, and counting was still underway on the final day of the second burst. These data support only the conclusion that an intermittent large-scale collection pattern appeared again on the same site.

Requests that did not reach the server are absent from the logs. The interpretation that distributed IPs divided up the same task is also based on the pattern of request times and paths. Intrix does not guarantee rankings, citations, or visibility.

How can we check for the same pattern again?

Total request counts alone hide bursts. Breaking requests down into a daily timeline helps distinguish intermittent large-scale collection. Intrix's free SEO and GEO audit checks a site's crawlability and response status through actual requests. Crawl counts and AI answer performance are different metrics and are measured separately.

Frequently asked questions (FAQ)

Q. Do 1,101 requests mean AI citations increased?
No. Server logs show only that a crawler requested pages. Source selection and mentions in answer text must be measured using separate response data.

Q. Does the same burst repeat every 18–21 days?
We cannot say that. With only two recorded bursts, the sample is too small to estimate a cycle. More daily logs are needed, without assuming when the next collection will occur.

Q. If it never requested robots.txt, did it break the rules?
Zero requests alone do not establish a violation. We could not rule out cached rules or another verification route, and we did not conduct an intervention testing access to a disallowed path.