AI Search Visibility
Answer Engine Optimization Agency: Questions Before Hiring
The questions to ask an answer engine optimization agency about reporting, tooling access and proof before you sign a retainer you cannot audit.
Sheetal Shakshi4 Nov 202616 min read
To choose an answer engine optimization agency, ask for a written reporting spec, read access to the tracking data, and proof from a baseline plus a held-out prompt set. An AEO agency should report which AI engines and prompts it checks, how often your brand is named, and which pages get cited. Noma by Nurdd lets you run your own tracking across 13 AI engines whichever agency you pick.
To choose an answer engine optimization agency, ask for three things before you sign: a reporting spec that names the engines, prompts and metrics; read access to the tracking data; and proof from a baseline and a held-out prompt set. An agency that cannot show all three is asking you to buy a black box.
This guide gives you the questions, the answers that should satisfy you, and the answers that should worry you. It also includes a reporting template, a test you can run on the agency's claims, a worked pricing example and a contract checklist. Use it on a sales call, or hand it to whoever owns the budget.
Key takeaways
- Ask for a written reporting spec before the contract: engines, prompt list, run frequency, country, and the exact metrics reported each month.
- Insist on read access to raw tracking data, or run your own tracking, so a retainer can be judged without relying on the agency's slides.
- Ask for proof as a baseline plus a before-and-after on a fixed prompt set, not as traffic charts or a single screenshot of a ChatGPT answer.
- Run a held-out test: the agency leaves a set of prompts untouched so you can see whether the lift is theirs or the market's.
- Write exit terms, data ownership and a "no guaranteed placement" clause into the contract.
- Keep an independent tracker such as Noma running so the measurement outlives the agency relationship.
Why is hiring an answer engine optimization agency harder than hiring an SEO agency?
With classic SEO, you can check rankings in Search Console, Google Analytics and a dozen tools, and the agency cannot easily hide a drop. With AEO, the answers change between runs, differ by engine, and often differ by country and by whether web search is switched on. An agency can therefore show you a screenshot that is true and still unrepresentative.
The stakes are rising because the surfaces are crowding the click. A Semrush, October 2026 study of a fixed panel of 400 commercial keywords found AI Overviews on 88% of them in U.S. desktop data, up from 46% at the start of its September 2025 to August 2026 window. A Pew Research Center, July 2025 analysis of March 2025 browsing found that users who saw an AI summary clicked a traditional result link in 8% of visits. If buyers are getting answers without visiting you, being named inside the answer is the thing you are paying an agency to change.
For the background terms, see our explainer on generative engine optimization, GEO versus AEO versus SEO and the AI search glossary for marketers. In this guide, "AEO" means getting your brand named and cited in AI answers, and "GEO" is treated as the same job from the content side.
What do the current guides to hiring an AEO agency cover, and what do they leave out?
Before writing this, I reviewed an agency research page from Onely, November 2026 and a buyer guide for SaaS teams from Aspiration Marketing, August 2026. Between them they cover agency types, evaluation questions and the need for baseline data.
What they do not give you is a reporting template you can paste into a statement of work, a test that separates the agency's effect from the market's, a table of who should own which piece of tooling, or contract language. Those are the four additions in this guide.
How do you choose an answer engine optimization agency? Seven steps
Work through these in order. Each step is an action, the result it gives you, and why it matters.
Step 1: Agree the prompt set and the outcome before you discuss price
Action: Write a list of 30 to 100 prompts your buyers would plausibly ask, grouped by funnel stage: category discovery, comparison, and brand-specific. Add the countries and languages that matter. Share it with every agency you talk to.
Result: Every agency quotes against the same yardstick, and you can see how they react to it. An agency that rewrites your list without explanation, or says it will "build the list later", is telling you something.
Why: Without a fixed prompt set, any visibility improvement can be real, or the result of changing what is being measured. Our guide on which prompts to track for AI search visibility explains how to build the list.
Step 2: Ask for a written reporting spec
Action: Before any proposal is signed, ask the agency to send a sample monthly report and a one-page spec for what goes in it. Compare it to the template below.
Result: You learn what "reporting" means in their contract, instead of discovering it in month two.
Why: Reporting is where black boxes hide. A report that shows only traffic, impressions or a handful of hand-picked screenshots cannot tell you whether you were named more often in the answers you care about.
Use this as the minimum reporting spec:
| Report element | What it must state | Why you need it |
|---|---|---|
| Engines checked | Named list, with the ones you can add | Visibility differs by engine, so an average hides gaps |
| Prompt set | Full list, version-dated, with changes logged | Stops the yardstick moving under you |
| Run frequency | How often each prompt is re-run | Answers vary between runs; one run proves little |
| Conditions | Country, language, web search on or off | Results change with each condition |
| Mention rate | Share of runs in which your brand is named | The headline outcome |
| Citation sources | Which URLs are cited, yours and others | Shows what the engines are actually reading |
| Competitor comparison | Same prompts, same runs, named competitors | Gives the score a context |
| Sentiment and accuracy | Whether the description of your brand is correct | Being named wrongly is a cost, not a win |
| Actions taken | Pages changed, content published, sources pursued | Links effort to movement |
| Next month's plan | Specific prompts and pages targeted | Makes the retainer forward-looking |
The engine, frequency and conditions rows matter most, because an answer can differ with each of them. Our own guide to AI search share of voice explains how to turn mention counts into a comparable score.
Step 3: Ask who owns the tooling and the data
Action: Ask four questions: Which tracking tool do you use? Will I get a login? Can I export raw run data? What happens to the history if we part ways?
Result: You know whether you could audit the work or move to another provider without losing your baseline.
Why: Tracking history is the only way to prove an AEO programme worked. If it lives in a login you cannot access, you cannot judge the programme without trusting the people being judged.
| Tooling question | Answer that satisfies | Answer that should worry you |
|---|---|---|
| Which tool tracks visibility? | Named tool, or your own tool is accepted | "Proprietary methodology" with no detail |
| Do I get access? | Read access to the dashboard from week one | PDF reports only |
| Can I export raw runs? | CSV or equivalent, including the answer text and cited URLs | Summary scores only |
| Who pays for the tool? | Stated clearly, either bundled or passed through at cost | Hidden in a flat fee that rises |
| Can I run my own tracker alongside? | Yes, and they will reconcile differences | Discouraged or ignored |
| What happens at exit? | Full history handed over | History stays with the agency |
The simplest fix is to keep your own tracking, independent of the agency. That is the design behind Noma, covered below, and it is the position we take in answer engine optimization agency versus in-house tooling.
Step 4: Ask for proof, then test it
Action: Ask for two things: a baseline audit from a previous engagement, and the before-and-after on a fixed prompt set. Then apply the held-out test described below to your own engagement.
Result: You separate agency-driven change from market drift, and you stop accepting single screenshots as evidence.
Why: A guide for SaaS buyers from Aspiration Marketing, August 2026 tells buyers to demand baseline audits and before-and-after citation data for specific prompts rather than vague traffic metrics. That is the right standard, and it works for any category.
Here is the held-out test, which neither page reviewed describes:
- Split your prompt set into two lists of similar difficulty and funnel stage: list A (agency works on these) and list B (held out).
- Record a baseline for both lists across the same engines, using your own tracking, over at least two weeks of repeated runs.
- Tell the agency to optimise for list A only. Agree in writing that list B is not to be targeted directly.
- After 90 days, compare the change in mention rate on A against the change on B.
- If both rise by a similar amount, the movement is probably the market or an engine update, not the agency. If A rises and B stays flat, the work is likely doing something.
It is not a controlled experiment, because content improvements on list A topics can spill into list B. It is still a more honest read than a before-and-after on a single list.
Step 5: Ask what changes on your site and what happens off it
Action: Ask for a list of the changes they expect to make on your site, the content they will produce, and the third-party sources they will try to earn mentions from.
Result: You see where the work is concentrated, and you can check that the technical basics are not blocking you.
Why: Engines draw on both your own pages and pages about you elsewhere. Our guide to why a competitor is cited in AI answers and you are not covers the off-site side.
Ask the agency to verify, in writing, the crawl and control settings that decide whether you are eligible at all:
- Google: Google's guide to optimising for generative AI features, July 2026 says content must be crawlable, and that a site must be included in Search generative AI features in Search Console to be eligible for display. It also says there is no ideal page length.
- Google controls: Google Search Console Help describes a Search generative AI control with include, exclude and inherit options, so ask the agency to confirm yours is set to include.
- ChatGPT: OpenAI's crawler documentation, October 2026 says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, and that each robots.txt setting is independent of the others. Ask which of OpenAI's bots your robots.txt allows.
If an agency sells you llms.txt files or schema stacks as the main lever, ask for the evidence that the engines you care about use them. Google's guide, for example, frames the work as making crawlable pages for your audience, not as a special file.
Step 6: Test the pricing and the term with real arithmetic
Action: Convert the retainer into a total for the full term, subtract any tooling you could run yourself, and compare it with what a result would be worth.
Result: You see the commitment as a total, not a monthly figure.
Why: Agencies price by monthly retainer, and the contract term hides the real exposure.
Worked example, with hypothetical numbers for illustration only, not a market price. Suppose a quote is $5,000 per month with a six-month minimum and a 30-day notice period after that. The minimum commitment is $5,000 × 6 = $30,000, and a first-year total is $5,000 × 12 = $60,000. Before you compare agencies, confirm whether each quote is monthly or per project, and what tracking it includes.
Now ask what portion of that spend is reporting and measurement. If the agency bundles tracking into the retainer, a separate tracker you run yourself makes that line item checkable. If it does not, the retainer should be justified by the work done, which brings you back to the reporting spec in step 2.
Step 7: Put the answers into the contract
Action: Convert what you learned into clauses before signing.
Result: A contract that can be audited and exited.
Why: Verbal assurances on a sales call do not survive staff turnover.
Use this contract checklist, in this order:
- Attach the reporting spec from step 2 as a schedule, including the prompt set and the engines.
- State that you receive read access to tracking dashboards and raw exports throughout the term.
- State that tracking history and all content produced belong to you at termination.
- Include a baseline period of at least two weeks before the work is measured.
- Include the held-out prompt list and the rule that it is not directly targeted.
- Add a clause that no placement in any AI answer is guaranteed, and treat any guarantee in the pitch as a red flag.
- List the on-site changes, who approves them, and who has access to the CMS.
- Set a notice period you can live with, ideally 30 days after an initial 90-day period.
- Name how off-site work will be evidenced, such as the URL of each placement.
- Agree a monthly review call with the numbers shared in advance.
Which answers should satisfy you, and which should worry you?
Use this scorecard during calls. A row marked as a worry is not an automatic rejection, but it needs an explanation you find convincing.
| Question | Satisfying answer | Worrying answer |
|---|---|---|
| How do you measure visibility? | A fixed prompt set, repeated runs, named engines and conditions | Manual spot checks in one assistant |
| Which engines do you track? | A named list that includes the engines your buyers use | "All of them" with no list |
| What will you report? | A sample report matching the spec table above | Traffic and rankings only |
| Can I see raw data? | Yes, from week one | Only in monthly slides |
| Show me a baseline from a past client | A redacted baseline and later comparison on fixed prompts | Case study with no prompt list |
| What do you guarantee? | Process and reporting, not placement | Guaranteed mentions or rankings in AI answers |
| Who does the work? | Named people and a review process for facts | Mass AI-generated pages |
| How do you handle wrong answers about my brand? | A process to fix sources that carry the error | No process, or "we cannot influence that" |
| What happens if we leave? | History and content handed over | Data stays in their tool |
On content volume, the Onely, November 2026 page reports practitioners seeing mass AI-written content lift traffic briefly and then decay, a good reason to ask who writes and checks the content.
What are the common mistakes when hiring an AEO agency?
- Hiring from agency list articles alone. Onely's page says answer engines draw their agency recommendations mostly from pages agencies publish about themselves, including lists and homepages. A list is a starting point for a shortlist, not evidence of results. Our own comparison of AEO and GEO agencies should be read in the same spirit.
- Accepting a screenshot as proof. One answer from one engine on one day is an anecdote.
- Letting the agency pick the prompts after signing. The yardstick should be fixed first.
- Measuring only Google. AI Overviews matter, and Google now offers a Generative AI performance report in Search Console, but ChatGPT, Perplexity, Claude and Gemini are separate surfaces.
- Skipping the baseline. Without a pre-work measurement, every improvement is unprovable.
- Paying for tracking you cannot see. If the tool is yours to log into, you can verify every number in the report.
- Confusing activity with outcome. Pages published and schema added are inputs. Mention rate, citation sources and accuracy are outputs.
What can Google's own reports tell you, and what can't they?
Google launched Search Generative AI performance reports in Search Console. According to Google Search Central, June 2026, the reports show impressions, pages, countries, devices and dates for how URLs from your site appeared in generative AI features, including AI Overviews and AI Mode, with a separate report for Discover. Ask your agency to include this report in its monthly pack, because it is free and comes from the platform.
It covers only Google. It reports your own URLs appearing, not which competitors are named in the same answer, and not what ChatGPT, Perplexity, Claude or Gemini say. That gap is why independent tracking across engines still matters.
How does Noma help you keep your own AEO tracking, whatever agency you hire?
Noma by Nurdd is a web dashboard that tracks whether a brand appears in answers from 13 AI engines, including ChatGPT, Gemini, Perplexity and Claude, polled daily. It shows citation frequency by category, competitor comparison, and recommendations for improving AI visibility, and it records whether your brand is named and which page each citation comes from. Noma lets you keep your own tracking record, so you can check an agency's numbers against it.
Book a demo at https://www.nurdd.club/demo, or start the 3-day Noma trial at nomahq.app.
In practice, Noma fits the steps above in four places. It gives you the baseline in step 4, because history starts when you start tracking. It supplies the independent tracker in step 3. It gives you competitor comparison on the same prompts for the report spec in step 2. And it keeps your data if the agency relationship ends. Noma has a 3-day free trial; a card is taken at signup and billing starts on day 3, with no charge if you cancel before then. You can see how it compares with other trackers in Noma vs Profound, Noma vs Peec AI and Noma vs AthenaHQ, and in our AI visibility tracking buyer checklist.
If you want a managed service rather than doing it in-house, Nurdd also runs answer engine optimization and generative engine optimization services, with results measured in Noma. The same questions in this guide apply to us. Ask for the reporting spec and the tracking access, and expect to see them. The How Nurdd works page describes the capability, and each engagement is scoped per brand.
How should you prepare before you speak to any agency?
Spend an hour on this checklist first. It makes every call shorter and makes comparison fairer.
- Run the GEO readiness scorecard to see which basics are missing on your site.
- Write your prompt list and group it by funnel stage, following the AEO audit checklist.
- Check robots.txt for OAI-SearchBot and Google's generative AI control in Search Console.
- Start tracking and record a two-week baseline, using the method in how to track your AI search visibility score week over week.
- Note your Google AI Overviews exposure using how to track Google AI Overviews for your brand.
- Decide who inside the company approves content changes and who owns the CMS login.
- Check any statistic an agency quotes against our list of AI search statistics you can safely cite.
What should you do next?
Pick the agencies you are comparing, send them the same prompt list and the same reporting spec, and ask for a redacted baseline from a past engagement. Start your own tracking in parallel, so the first month of the programme is already measured. Then decide on the answers, not the pitch.
To have your own tracking in place before you brief anyone, book a demo or start the 3-day Noma trial at nomahq.app.
How we researched this
This guide draws on the pages cited in it: two agency pages (Onely and Aspiration Marketing), platform documentation from Google Search Central, Google Search Console Help and OpenAI, a Pew Research Center analysis and a Semrush study. Numbers are quoted as those sources state them. We did not run hands-on tests of any agency, and we have no customer data or interviews to report. Nurdd sells Noma and AEO services, so treat this guide with that interest in mind and check each claim against its linked source.
Frequently asked questions
How do I choose an answer engine optimization agency?
Fix a prompt list first, then ask each agency for a written reporting spec, dashboard access and a redacted baseline from a past engagement. Compare their answers using the scorecard in this guide. Prefer the agency that lets you run your own tracking alongside theirs.
What should an AEO agency report to me?
At minimum: the engines and prompts checked, how often they are re-run, the conditions used, your mention rate, the URLs cited, a competitor comparison, accuracy of how your brand is described, and the actions taken. Traffic and rankings alone are not enough, because answers can name you without producing a click.
Should an AEO agency guarantee a mention in ChatGPT?
No. Answers vary between runs and engines, and no agency controls them. Treat a guarantee of placement as a red flag, and ask for guarantees about process, reporting and access instead.
How much does an answer engine optimization agency cost?
It varies by agency and scope, and published figures differ widely, so ask each agency for a written quote. Confirm whether the figure is monthly or per project, what the minimum term is and what tracking it includes. Use the arithmetic in step 6 to compare total commitments.
Do I need an agency, or can I do AEO in-house?
It depends on content capacity and how much off-site work you need. Many teams keep tracking in-house and buy specialist help for content and source earning. Our guide to agency versus in-house tooling sets out the trade-offs.
How long before an AEO agency shows results?
There is no reliable published timeline, so be suspicious of precise promises. Ask for a baseline period and a 90-day checkpoint on a fixed prompt set, with a held-out list to separate agency effect from market movement.
Can I track AI visibility myself if the agency already reports it?
Yes, and it is a sensible check. Independent tracking lets you compare the agency's numbers with your own and keeps your history if you change provider. Noma tracks 13 AI engines, including ChatGPT, Gemini, Perplexity and Claude, and has a 3-day free trial with a card taken at signup.
Does Google Search Console show my AI visibility?
For Google only. Google's Search Generative AI performance reports show impressions, pages, countries, devices and dates for AI Overviews, AI Mode and Discover, according to Google Search Central in June 2026. They do not cover ChatGPT, Perplexity, Claude or Gemini, so you need separate tracking for those.