Your director asks why the brand appeared less often in AI answers this month. First check whether the sample changed. Then show which findings need an owner and which need more evidence.
To keep the mechanics concrete, we will reference Kestrel Cloud, a hypothetical B2B expense-management software provider, using synthetic monthly reporting numbers throughout.
1. Choosing Customer, Product, and Buying-Stage Questions
Choose prompts that represent buying decisions the team can act on.
Aim for a stable baseline—for example, 40 core prompts reviewed monthly. Group them into three commercial tiers:
- Problem and Category Discovery (Early Stage): Unbranded questions from buyers framing a business need. Hypothetical Kestrel Cloud prompt: "How do mid-market finance teams automate multi-currency expense approvals?"
- Product and Shortlist Evaluation (Mid Stage): Category comparisons, integration requirements, and use-case fit. Hypothetical prompt: "Best expense management software with native NetSuite integration for UK companies."
- Brand Validation and Commercial Terms (Late Stage): Direct inquiries about your brand’s pricing, security, compliance, or implementation. Hypothetical prompt: "Does Kestrel Cloud support SOC 2 Type II compliance and SSO on its standard tier?"
Keep unbranded discovery prompts separate from branded validation prompts during analysis. Expecting high brand mention rates on broad, early-stage prompts skews expectations, whereas factual errors on late-stage branded prompts require immediate correction.
2. The AI Visibility Data Dictionary
| Metric | Operational Definition | Unit | Eligible Denominator |
|---|---|---|---|
| Brand Mention Rate | Responses where the target brand is explicitly named in the generated answer body. | Percentage (%) & Count | Total successful runs across eligible prompts (e.g., 196 successful runs). |
| Owned Citation Rate | Responses that include a clickable link or explicit source attribution to a domain controlled by the brand. | Percentage (%) & Count | Total successful runs (or tracked separately as a share of runs with a brand mention—state which). |
| Fact Accuracy Rate | Evaluated factual claims about the brand (pricing, features, compliance, integrations) that match current approved product documentation. | Percentage (%) & Count | Total verifiable brand claims extracted from runs containing a brand mention (distinct from run count). |
| Competitor Gap | Percentage-point difference between the target brand's Mention Rate and a named primary competitor's Mention Rate on non-branded prompts. | Percentage points (pp) | Total successful runs across non-branded category and evaluation prompts only. |
Notice the distinct denominator for Fact Accuracy Rate. If Kestrel Cloud is mentioned in 84 successful runs, those 84 answers might contain 150 discrete, checkable claims about pricing, integrations, or compliance. Fact accuracy must be calculated against those 150 verifiable claims—not total runs—so a single answer containing two true statements and one hallucinated pricing tier is scored with precision.
3. Structuring a Comparable Monthly Report
Sampled AI answers vary across geographies, interfaces, and time. For a monthly leadership report to support trend analysis, every scorecard must lock and disclose its collection metadata.
Include a header block in your monthly tracker, informed by the monitoring scope,covering these eight parameters:
- Market & Geography: Country or regional setting simulated during collection (e.g., United Kingdom).
- Language: Prompt and expected response language (e.g.,
en-GB). - Platform & Model: Specific AI provider and underlying model family where identifiable.
- Surface: Whether responses were collected via an end-user consumer web UI, a search-integrated AI overview, or a developer API.
- Prompt Set Version: Version control tag (e.g.,
v1.2 - Q2 Core 40) to flag wording edits. - Collection Window: Exact date range and time-of-day distribution (e.g., 3–5 May, sampled across three intervals).
- Total Attempts: Scheduled query executions (e.g., 40 prompts × 5 platforms = 200 attempts).
- Successful Runs: Completed responses excluding timeouts, network errors, or safety refusals (e.g., 196 successful runs; 4 failed attempts excluded from metric denominators).
Report referral visits, form events and CRM leads alongside the AI sample. Give each its own source and denominator.
4. Interpreting Shifts When Coverage or Systems Change
When a rate moves, first check whether the test conditions or sample size changed.
- Retrieval changes. If source links appear or disappear across many unrelated prompts, record the change. Review comparable answers before drawing a conclusion about the brand.
- Model Upgrades: Providers routinely swap underlying models or adjust system instructions, changing how concise answers are or how many vendors get listed in a shortlist.
- API and consumer interface. These surfaces can use different retrieval, location and personalization settings. Keep their baselines separate.
- Prompt Version Drift: Changing "best expense software" to "top enterprise expense platforms" alters the competitive set. If you update prompt phrasing, reset the baseline or report the new prompt cohort in a separate appendix for one cycle.
5. Decision Rules, Owners, and Approval Workflows
- Wrong Facts (Low Fact Accuracy Rate):
- Trigger: Any hallucinated pricing, deprecated feature claim, or false compliance statement on high-intent prompts.
- Action: Audit owned product pages, pricing tables, and public help docs for ambiguous phrasing or outdated PDFs still indexed on your domain. Publish clear, machine-readable HTML clarifications.
- Owner: Product Marketing Lead.
- Approval: Legal / Compliance (for regulatory or security claims) and Product Management.
- Missing Brand Mention (Low Brand Mention Rate / Negative Competitor Gap):
- Trigger: Zero or low presence on mid-stage category evaluation prompts where competitors consistently appear.
- Action: Assess third-party footprint—software review directories, partner integration listings, and independent industry comparisons—alongside missing use-case landing pages on your own site.
- Owner: Content & Digital PR Lead.
- Approval: Brand Marketing Director.
- Mentioned Without Owned Link (Gap Between Mention Rate and Owned Citation Rate):
- Trigger: The AI accurately names your brand in shortlists but cites third-party aggregators or outdated forums instead of your domain.
- Action: Check
robots.txtrules for AI search crawlers, consolidate duplicate URLs, add structured reference pages (e.g., integration specs, comparison matrices), and ensure canonical pages load cleanly without heavy client-side rendering barriers. - Owner: Technical SEO / Web Operations Lead.
- Approval: Head of Engineering or Web Governance Lead.
6. One-Page Executive Scorecard (Synthetic Example)
Below is a synthetic one-page monthly brief for our hypothetical company, Kestrel Cloud, demonstrating internally consistent denominators and clear separation between sampled AI output, web user behaviour, and direct business records.
Monthly AI Visibility Scorecard: Kestrel Cloud (Synthetic Data)
Report Metadata (Sampled AI Output)
- Market / Language: United Kingdom (
en-GB) | Prompt Version:v1.0(40 Core Prompts: 30 Non-Branded, 10 Branded) - Platforms & Surface: 5 major AI search/assistant web UIs | Window: 1–4 June
- Execution Summary: 200 Total Attempts | 196 Successful Runs (4 timeouts excluded; 148 non-branded runs, 48 branded runs)
Core Visibility & Accuracy Summary (Sampled AI Output)
- Brand Mention Rate: 42.9% (84 / 196 total successful runs) — Non-branded subset: 27.0% (40 / 148 runs); Branded subset: 91.7% (44 / 48 runs).
- Owned Citation Rate: 16.8% (33 / 196 total successful runs) — Note: Kestrel Cloud’s domain was cited in 39.3% (33 / 84) of runs where the brand was mentioned.
- Fact Accuracy Rate: 88.0% (132 accurate claims / 150 verifiable brand claims extracted across the 84 runs mentioning Kestrel Cloud).
- Competitor Gap (vs. Hypothetical Rival "FinTrack"): -11.5 pp on non-branded prompts (Kestrel Cloud: 27.0% [40 / 148] vs. FinTrack: 38.5% [57 / 148]).
Adjacent Commercial Context (Separate Systems — Not Sampled AI Output)
- User Behaviour (Web Analytics): 310 sessions from identifiable AI assistant referrers; 14 website demo-request form conversions.
- Direct Business Records (CRM): 9 marketing-qualified leads (MQLs) citing AI tools in self-reported attribution fields (tracked separately from web referrer sessions).
Leadership Action Register (Synthetic Scenarios)
| Observation | Likely Business Impact | Next Step | Owner (Approval) | Confidence |
|---|---|---|---|---|
| 18 inaccurate claims (12.0% of 150 claims) say Kestrel Cloud lacks NetSuite sync. Several answers link to an archived help article still live on a subdomain. | Mid-market prospects using NetSuite may disqualify Kestrel Cloud before visiting the website. | Redirect legacy help URL to current NetSuite integration spec page; update comparison table HTML. | Product Marketing (Product Lead) | High (Repeated across 3 platforms citing the legacy URL) |
| -11.5 pp Competitor Gap against FinTrack on UK multi-currency expense prompts (40 vs. 57 mentions out of 148 non-branded runs). | Fewer mentions in this simulated set of UK discovery questions. Buyer behaviour remains unknown. | Publish dedicated UK multi-currency compliance guide; update G2/directory integration profiles. | Content & PR Lead (Marketing Director) | Medium (Consistent over 2 months; 1 platform updated UI mid-window) |
| 51 mentioned runs lacked an owned link (84 mentions vs. 33 owned citations); third-party review directories captured the citation slot. | Buyers following those citations would land on third-party pages. The sample contains no click data. | Audit crawler access on /security and /pricing; add concise HTML summary blocks at top of core pages. | Technical SEO Lead (Web Governance) | High (Verified across all 5 sampled web surfaces) |
If your team wants to establish a structured baseline without maintaining manual spreadsheets and ad-hoc screenshots, explore FoundUp’s AI Visibility Monitoring service to set up consistent prompt tracking, claim verification, and leadership-ready reporting tailored to your market.