Piloting a check of where AI recommendations change by customer type

Intrix has begun a pilot of AI conversation checks by customer profile. It uses the details AI asks for to compare customer types and see where recommendations diverge for the same question. Intrix is reviewing results from selected measurement rounds. The check is not yet part of customer dashboards or measurement reports, and customers do not need to apply or enter anything.

Compare a customer who fits with one who does not

This follows our previous update on clarification. Before choosing a company, AI may ask about location, industry or business size. Those details are the criteria it uses to distinguish recommendations for that question. The new check creates two customer profiles from those details: one that fits the service area and target customers stated on the company website, and one that does not. Using only a favorable profile could make the results look consistently positive.

In a fictional example, someone asks, “Please recommend an accounting firm to handle bookkeeping for a corporation.” AI asks about location, industry and revenue. A profile that fits the website’s stated customers could be an IT startup in Gangnam, Seoul, with 12 employees. The contrasting profile could be a cafe run by one person in Jeju.

Separate prior knowledge from details supplied in conversation

Intrix asks the same question again for each customer, in two situations. In one, AI receives the customer information beforehand, as with an AI app’s memory feature. In the other, the customer answers AI’s questions one at a time during the conversation. If AI asks again, the customer answers again, continuing until it recommends companies.

In an initial test using three questions about Intrix itself, one AI service included remembered user information in its search terms in six of nine cases. Without memory, it did so in zero of nine cases. Cited sources also changed substantially for questions about choosing a company. Of the 54 domains cited without memory, only 24 and 16 remained for the two customer types, respectively. This test covered three questions in one industry, so these proportions should not be generalized.

See where your site drops out and which sources take its place

The check compares each customer type with the question asked without a profile. It looks at whether the stage where your site drops out of recommendations changes, distinguishing a page AI never read from one it read before choosing another company. It also shows the sources cited when your site was left out. These offer a starting point for finding information your page may be missing for that customer.

Intrix records the conversation turn when company recommendations first appear and the turn when your site’s status changes. This keeps the recommendation alongside the conversation that led to it.

Pilot results will guide a decision on inclusion

Intrix will review the pilot results before deciding whether to include the check in the regular service. Timing and format have not been decided. Read the previous clarification update for the starting point, or contact Intrix with questions about the check.