A growing number of people ask an AI assistant for recommendations before they ever open a search engine. They ask which provider to hire, what to look for when buying something, or how a certain process works. The answer they get might name your business, name a competitor, describe you inaccurately, or ignore the whole category. Most owners have no idea which of those is happening, because there is no simple report that shows it.
The good news is that you can build a basic tracking habit without expensive software. This guide shows you how to decide what to track, build a fixed set of test questions, record results in a way that is fair and repeatable, check accuracy as well as presence, and turn the findings into actions. You will also learn the limits of this kind of measurement, so you do not read too much into any single answer.
Key takeaways
- Tracking mentions means testing a fixed set of realistic questions in the tools your audience uses, and recording what comes back.
- Answers vary between runs and between tools, so repeat each question and look at patterns over time, not single results.
- Check accuracy as well as presence: being described wrongly can be worse than not being mentioned.
- Record which sources the tool cites, because they show where your reputation is being built or misunderstood.
- Use what you find to fix your own pages and listings, not to chase tricks.
Why track AI mentions at all
For years, businesses have tracked rankings, traffic and reviews. AI answers add another place where your reputation is formed, and it is one you cannot see by looking at a rankings report. If a tool describes your service wrongly, buyers may form a mistaken first impression before they reach you.
What tracking can tell you
- Whether your brand appears for the questions your customers ask.
- How you are described: correctly, vaguely or wrongly.
- Which competitors are named alongside or instead of you.
- Which websites the tools seem to rely on when talking about your field.
- Whether changes you make to your content or listings are followed by changes in answers.
What it cannot tell you
It cannot tell you exactly how many people saw an answer, or how many acted on it. Tools rarely share that data. Your sample of tests is a small window, so treat the results as signals and not as precise measurements.
What counts as a mention
Before you record anything, agree on definitions. Otherwise two people will score the same answer differently, and trends will be meaningless.
| Category | Definition | Why it matters |
|---|---|---|
| Direct mention | Your brand name appears in the answer text | The clearest form of visibility |
| Citation or link | Your website or page is shown as a source | Shows the tool used your content and offers a click |
| Description | The answer says what you do or who you serve | Shapes how people perceive you |
| Recommendation | The answer suggests choosing you, or lists you among options | Closest to a buying influence |
| Absence | The question is relevant but you are not named | Highlights a gap |
| Misstatement | The answer gets facts about you wrong | A priority to correct |
Building your question set
The quality of your tracking depends on the questions. They should reflect how real people ask, not how marketers write. Gather them from customer emails, sales calls, support requests, reviews and your own search data.
Types of questions to include
- Category questions: what is the best way to solve a problem, or what should I look for in a provider.
- Comparison questions: how do two approaches or providers differ.
- Brand questions: who is your business, what do you do, is it any good.
- Location or audience questions: options for a specific place, size or type of customer.
- Problem questions: how to fix a particular issue your business solves.
How many to track
Start with twenty to thirty questions. That is enough to see patterns and small enough to run regularly. Group them by theme so you can see which areas are strong or weak. You can ask an AI tool to suggest additional questions, but check them against what customers really say. For related thinking, see our article on AI keyword research without trusting it blindly.
Write each question exactly once
Keep the wording of each tracked question fixed. Changing phrasing changes the answer, and you will not know whether a difference came from the wording or from real movement. If you want to test variations, add them as separate questions.
Choosing where to test
Different tools serve different audiences and behave differently. Test where your customers are likely to ask.
- General purpose chat assistants.
- AI summaries that appear within search results.
- Answer engines that show sources by default.
- Assistants built into browsers, devices or business software your customers use.
Ask your customers which tools they use. A short question in an onboarding form or a conversation can give you a better guide than guessing. If you are unsure how to pick tools for your own work, our guide on how to choose an AI tool gives a framework.
Account and setting differences
Results can depend on location, language, whether you are logged in, previous conversations and settings such as web access. To keep tests comparable, use a consistent setup: the same tool, a fresh conversation for each question, and the same region and language. Note the setup in your sheet.
A repeatable testing method
Consistency is more important than sophistication. Follow the same steps each time, and keep notes.
- Open a fresh conversation in the tool.
- Enter the exact question from your list.
- Save the full answer, with any shown sources. A screenshot or pasted text is fine.
- Repeat the same question two more times in fresh conversations, since answers can vary.
- Score the answers using your agreed categories.
- Record the date, tool, setup and who ran the test.
How often to test
Monthly is a sensible start for most small businesses. If you are making major changes to your content, you might test more often for a short time. Avoid testing daily, since the noise will exceed the signal and you will spend time without learning anything.
Protecting privacy
Do not enter customer names, confidential plans or internal data in your test questions. Keep questions generic and public. Our guide to protecting company data when using AI tools offers more detail.
Setting up your tracking sheet
A spreadsheet is enough. Use one row per question per tool per run, with consistent columns.
| Column | Example entry type |
|---|---|
| Date | The day of the test |
| Tool and setup | Which assistant, region, web access on or off |
| Question ID and text | A short code plus the exact wording |
| Run number | One, two or three |
| Brand mentioned | Yes or no |
| Position or role | First named, one of several, passing reference |
| Cited as a source | Yes or no, with the page |
| Description accurate | Accurate, partly accurate, wrong |
| Competitors named | A list |
| Sources shown | Websites listed |
| Notes | Anything unusual |
Summarising the data
Once you have a few rounds, create a summary tab. For each question group, count in how many runs you were mentioned, how many descriptions were accurate and which sources appeared most. Present these as simple fractions, such as mentioned in four of nine runs, rather than percentages that suggest more precision than you have.
Checking accuracy and sentiment
Being mentioned is not enough. How you are described matters as much. Read each answer that names you and compare it with the facts.
Accuracy checklist
- Is the description of what you do correct?
- Are the places you serve and the people you serve correct?
- Are any claims about quality, features or history supported by your own records?
- Is anything out of date, such as a service you no longer offer?
- Is your business confused with another one with a similar name?
Models can state errors confidently. This is the same problem described in our article on AI hallucinations. When you find a mistake, record the exact wording, the tool and the date, then look for where it might have come from: an old page, a directory listing, a review or a similar company.
Tone and framing
Note whether the answer is positive, neutral or cautious, and whether it frames you as a leading option, a niche choice or an afterthought. Keep this judgement simple, and have two people score a sample so you can check that you are consistent.
Reading the sources behind answers
Some tools show where information came from. These sources are a goldmine, because they reveal which websites help shape answers in your field.
| Source type | What it suggests | Possible response |
|---|---|---|
| Your own pages | Your content is being found and used | Keep it accurate and current |
| Review and directory sites | Third party profiles influence your description | Check and update your profiles and respond to reviews honestly |
| Industry publications | Trade coverage shapes perceptions | Contribute genuine expertise where appropriate |
| Competitor pages | Competitors are defining the category | Publish clearer, more useful explanations of your own |
| Outdated pages | Old information is circulating | Ask for corrections or publish updated pages |
Do not assume that a tool will cite the same sources next month. Use the list as a map of influences, not a fixed recipe. For a wider view of how answer focused content is structured, read our guide to answer engine optimisation.
Connecting mentions to real results
A mention is interesting, but a business needs outcomes. Look for indirect signs that AI visibility is leading people to you.
- Referral traffic from AI tools in your analytics, where the tool passes referral data.
- Branded searches rising after periods of visibility.
- Direct visits to pages you have recently improved.
- Self reported source on enquiry forms, with an option such as an AI assistant.
- Sales conversations in which people say they found you through a recommendation from a tool.
Add a simple question to your enquiry form asking how people heard about you, with an option for AI tools. Self reported answers are imperfect, but they give you a signal that analytics alone may miss. Be honest in reports: correlation is not proof, and many factors influence enquiries.
Turning findings into actions
Tracking is only worthwhile if it leads to improvements. Use your results to guide a short list of actions each quarter.
| Finding | Likely cause | Action |
|---|---|---|
| Not mentioned for category questions | No clear page answers the question, or low visibility | Create or improve a page that answers it plainly |
| Mentioned but described wrongly | Inconsistent or outdated information | Align your site, profiles and listings, and update old pages |
| Competitor named, you are not | They have clearer or more widely referenced content | Study what they explain well and publish something more useful |
| Confused with another business | Similar names or thin identity signals | Strengthen your about page, location details and consistent naming |
| Answers vary widely | Topic is ambiguous or sources conflict | Clarify the topic on your pages with definitions and examples |
If you use AI to help draft improved pages, check the output carefully with the process in our guide on checking AI output for mistakes.
Limits and pitfalls
- Small samples. Twenty questions run a few times is not a statistical study.
- Variation. Answers change between runs, tools and over time.
- Personalisation. Your own history may influence what you see, so results may differ for others.
- Over fitting. Rewriting pages to match the exact wording of a test question can harm natural writing.
- False certainty. A single good answer does not prove success, and a single bad one does not prove failure.
- Paid tools. Some services offer automated monitoring. Evaluate them with scepticism, ask how they sample and how they handle variation, and compare with your own manual checks before paying.
Monthly routine in one list
- Run your question set in the chosen tools, three runs each.
- Record mentions, citations, accuracy and sources.
- Summarise by question group.
- Identify the top three problems and top three wins.
- Assign actions with owners and dates.
- Review last month’s actions and note whether answers changed.
A sample scoring guide
To keep scoring consistent between people, write the rules down. The table below is a simple example you can adapt. The point is to make each score depend on what is in the answer, not on how the reviewer feels that day.
| Score | Presence | Accuracy |
|---|---|---|
| 0 | Not named and not cited | Not applicable |
| 1 | Named in passing, among many options | Contains a clear error about what you do |
| 2 | Named with a short description | Mostly right with minor gaps or out of date detail |
| 3 | Named, described and cited or recommended | Fully consistent with your own facts |
Have two people score the same ten answers separately, then compare. If scores differ a lot, tighten the wording of the rules. This small calibration step makes your month to month comparisons far more trustworthy, and it takes less than an hour.
Frequently asked questions
Can I automate this tracking?
Partly. Some tools and scripts can run questions on a schedule and store the output. Automation helps with volume, but you still need to review accuracy and context yourself. Check the terms of any tool you use before automating requests.
How many times should I repeat each question?
Three runs is a practical minimum, since answers vary. For a few critical questions, you could run more to understand the range of responses.
What if an AI tool says something wrong about my business?
Record it, check your own pages and listings for anything unclear or outdated, and fix those first. Some tools offer feedback options, which you can use to report errors. You cannot guarantee a correction, so keep information clear and consistent everywhere.
Does being cited mean more traffic?
Not necessarily. A citation gives people a chance to click, but many will be satisfied by the answer itself. Measure visits and enquiries rather than assuming.
Is this worth doing for a very small business?
A light version is. Ten questions, once a month, takes under an hour and may reveal errors or gaps worth fixing. Scale up only if it proves useful.
Should I track competitors too?
Yes, as context. Note who is named for the questions you care about and what sources support them. Use this to understand the landscape, not to copy.
Conclusion
Tracking your brand in AI answers does not need special software. It needs a fixed set of realistic questions, consistent testing, honest scoring and a willingness to act on what you learn. Record presence, accuracy and sources, repeat the tests, and look for patterns over time rather than celebrating or panicking over single answers.
Start this week with ten questions and one tool. Build your sheet, run the first round and note what surprises you. Then fix the clearest problems on your own pages and listings. If you want to deepen your understanding of the underlying technology, read our explainer on what a large language model is, and for structuring pages so they are easier to quote, see our AEO guide.
