Question Sets Shape AI Visibility Rankings: Leaders Hold, but 4th Place and Below Shift
Published
This article was first published in Korean by Intrix Lab on September 21, 2026.
Rankings of who gets cited most often in AI answers depend on the question set. As of 2026-09-14, across 6 Intrix CT studies (36 questions, 540–1,080 responses), we randomly selected half the questions and recalculated rankings 500 times. The leading domain held its position in 360 to 500 of the 500 runs (from seven in ten runs to every run), and top-10 overlap (Jaccard) was 0.72–0.84, but domains ranked 4th and below moved between high positions and outside the rankings in all six studies. A paper featured on GEO Watch described visibility scores as an "answer market" defined by the prompt corpus used for evaluation, rather than actual demand. In our data, that statement held more for the tail than for the head.
Why test this without submitting new questions?
The paper's claim is that "changing the question set changes the score." Selecting subsets of questions within existing studies is enough to measure a lower bound on that effect. If rankings shift even when using only 18 of 36 questions on the same topic, they will shift more when the topics themselves change. The cost was 0.
How did we measure it?
For each study, we compared the top-10 domain ranking based on all questions with rankings recalculated using only half the questions, repeating this 500 times. We examined how often the leading domain retained its position, overlap among the top 3 and top 10, and the range of ranking movement for each domain.
| Study (industry) | Leader retained | Top-3 overlap | Top-10 overlap | Resampled ranks of the overall 4th-place domain |
|---|---|---|---|---|
| Divorce law | 475/500 | 0.94 | 0.84 | 2nd to outside the rankings |
| Personal rehabilitation law | 460/500 | 0.97 | 0.74 | 2nd to 10th |
| Sexual offense law | 500/500 | 0.60 | 0.79 | 2nd to outside the rankings |
| Branding agencies | 365/500 | 0.91 | 0.72 | 3rd to 9th |
| Children's clothing stores | 360/500 | 0.58 | 0.77 | 2nd to outside the rankings |
| GEO agencies | 500/500 | 0.88 | 0.77 | 3rd to outside the rankings |
"Outside the rankings" means falling outside the top 10. In all six studies, the overall 4th-place domain ranked 2nd–3rd in some resamples and outside the rankings in others.
Why do the leaders hold up?
In the legal studies, the leading site was a legislation site, while courts and the Korea Legal Aid Corporation ranked 2nd–3rd. These sources are cited regardless of which questions are selected, making them less sensitive to the question set. Hosts ranked 4th and below, by contrast, are cited for only a few specific questions, so they disappear from the rankings when those questions are omitted from the sample. In the commerce and branding studies, even the leading domain retained its position only 360–365 times out of 500, because citations are widely distributed in these industries and even the leaders have a thinner base.
How should we read rankings differently?
The statement "our brand ranks 7th in AI visibility" is a function of the question set. Top-3 classifications generally hold up even when half the set changes, but rankings below that have no meaning unless you also examine which questions were included and how many. This is why Intrix includes tables showing the composition and coverage of the question set in every study. When a report says rankings have improved, the first question to ask is whether the question set stayed the same.
Limitations
Resampling uses subsets within the same question pool, so it produces less variation than switching to a question pool covering different topics. The paper also examined changes to scoring rules and weights, but we looked only at question composition. The 6 studies were Korean-language studies conducted by Intrix and are not a sample of the entire industry. These findings are not a basis for guaranteeing specific rankings.
Frequently asked questions (FAQ)
Q. Does that make ranking metrics useless?
No. The leading group, such as the top 3, holds up. Below that, question-level metrics such as "for how many questions was the brand a candidate?" are more stable than rankings.
Q. Does adding more questions solve the problem?
Within the same topic, larger samples reduce variation, but rankings shift again when the topic mix changes. Keeping the composition fixed comes before increasing the question count.
Q. Can we check our own study this way?
You need citation records at the question level to check this. Intrix provides the composition and coverage of the question set with every study.