US market research

Does Rephrasing the Same Question Change AI Recommendations? Results from 30 Phrasings

Published

This article was first published in Korean by Intrix Lab on September 25, 2026.

Yes. As of 2026-09-25, when we expressed the same intent using 10 actual search phrasings and asked two AI models, the recommended companies varied much more than when we repeated the same sentence. All 18 companies recommended most often within each intent were absent from all 5 responses for at least one phrasing. If you measure AI visibility with a single sentence, the sentence you choose determines the result.

Why ask about the same intent in multiple ways?

Most tools that measure AI visibility represent each intent with a single question. They reduce variability by asking the same question multiple times. But actual customers ask in different ways even when looking for the same thing.

Search research is also addressing this issue. The SELF-INDEX study (arXiv 2609.19656), released in September 2026, argued that even documents with relevant content may not be retrieved when the query's wording does not match the wording in the index. We therefore started by checking whether one sentence could represent an intent.

How did we choose the phrasings?

We examined three intents: finding a GEO agency, finding a hospital marketing agency, and finding an advertising agency in Busan. We selected 10 phrasings for each intent.

We took the phrasings from actual search demand data rather than writing them ourselves. Some were single keywords, such as "hospital marketing," while others were full questions. We submitted each phrasing to two AI models 5 times each and set the evaluation rules before collecting the data.

How much did repetition and rephrasing differ?

We compared the overlap in recommended companies between answers to repeated versions of the same sentence (Jaccard) with the overlap between answers to different phrasings. The larger the difference between the two values, the greater the effect of phrasing on the results.

Model · IntentSame sentence repeatedDifferent phrasingsPhrasing effect [95% interval]
Model A · GEO agency0.3820.2120.170 [0.072, 0.284]
Model A · Hospital marketing0.2260.0860.139 [0.057, 0.259]
Model A · Busan advertising agency0.4080.1620.246 [0.114, 0.367]
Model B · GEO agency0.1880.0850.102 [0.036, 0.191]
Model B · Hospital marketing0.0630.0370.026 [0.000, 0.057]
Model B · Busan advertising agency0.1490.0210.127 [0.061, 0.221]

Model A exceeded the predefined threshold for all three intents and was classified as showing a "phrasing effect." Model B exceeded the threshold for only one intent, so we withheld a judgment. For Model B's hospital marketing questions, overlap was only 0.063 even when the same sentence was repeated, leaving little room to distinguish a separate phrasing effect.

How much did the same company's results vary by phrasing?

For each intent, we selected the 3 most frequently recommended companies from each model, for a total of 18. All 18 appeared 0 times out of 5 for at least one phrasing. All 9 companies from Model A appeared in all 5 responses for at least one other phrasing.

This means that measuring a company's appearance rate with a single sentence could produce either 0% or 100%. Increasing the number of repetitions does not reduce this difference. Repetition only reduces variability within the same sentence.

Doesn't AI account for phrasing differences when it searches?

AI does not search using the user's question verbatim. It rewrites the question into search queries. If AI converted different phrasings into similar search queries, the difference should shrink. That generally did not happen in these results.

For overlap in the search queries AI actually used, the phrasing effect was 0.08–0.20 for Model B and 0.05–0.18 for Model A. Different phrasings produced different search queries. Model B searched for only 72% of keyword-style questions, compared with 100% of sentence-style questions. Its answers to keyword-style questions were also about half as long: 992 characters versus 1,812 characters for sentence-style questions.

How should measurement change?

Representing an intent with one sentence ties the results to that sentence. For the same budget, these results support using more phrasings rather than repeating one sentence many times.

To find out whom AI recommends in your industry and which phrasings leave your company out, we can work with you on question design through our Initial Assessment. To check whether AI can read your site first, start with a free website audit.

How far do these results apply?

The study covered 3 intents involving company recommendations, 30 Korean phrasings, and 2 models. The results were collected at a single point in time, 2026-09-25. Because the answers came from API calls, they cannot be assumed to match what appears in consumer apps. We do not generalize that effects of the same size will occur in other industries or for other intents.

Because we selected phrasings only from actual demand data, the GEO agency intent was unbalanced, with 7 keyword-style phrasings and 3 sentence-style phrasings. The conclusion remained the same in an analysis that excluded questions specifying tasks such as blogging, SEO, and viral marketing. These results also cannot support a guarantee that a particular phrasing will always produce a recommendation for a specific company.

Frequently asked questions (FAQ)

Q. Wouldn't repeating the same question more often solve this?
No. Repetition reduces variability within the same sentence, but it does not change whether that sentence represents the intent. Even the 18 most frequently recommended companies in this study ranged from 0 to 5 appearances depending on the sentence.

Q. Is asking with keywords different from asking in full sentences?
It was in this study. Model B searched for only 72% of keyword-style questions, while it searched for all sentence-style questions. Measuring visibility using only keyword-style questions may understate that model's results.

Q. If the companies change with the phrasing, does that mean AI is wrong?
It is more a result of looking for different things than being wrong. When the phrasing changed, the search queries AI used changed, which in turn changed its supporting sources and recommended companies.

← Back to insights