How many prompts do you need to measure AI visibility?

A market is a set of buyer questions about one product category. A brand-market series is one brand's visibility on one engine in one market. Its visibility rate is the share of AI answers naming the brand, averaged over the market's prompts. A tracking design is k prompts asked r times each.

Reruns of the same prompt are correlated, because the prompt fixes much of the answer. Measured across 12,053 series, ten reruns of one prompt carried the information of 2.02 prompts asked once, and three reruns carried 1.60 prompts' worth. Two independent computations agree: 2.02 from the measured variance components and 2.02 from the observed disagreement between random samples. On the 72 market-engine pairs with at least 20 prompts the figure was 2.10.

The practical form: three different prompts asked once had a typical error of 19.97 points, and one prompt asked ten times had 24.28 points. Typical error is the root-mean-square gap, in percentage points, between a design's reading and the brand's full-window visibility rate.

Ten answers can be one prompt asked ten times or ten prompts asked once. The ten-prompt design had a typical error of 11.28 points; the one-prompt design had 24.28, 2.2 times larger. Thirty answers as 10 prompts × 3 runs gave 8.77 points; as 3 prompts × 10 runs, 14.07, 1.6 times larger. Three answers as 3 prompts × 1 run gave 19.97; as 1 prompt × 3 runs, 27.30.

The 10-prompt designs are measured on the 72 market-engine pairs with at least 20 prompts, so that two teams can hold disjoint 10-prompt sets. On that same panel, one prompt asked ten times gave 24.47 points and 3 × 10 gave 14.16, so the comparison holds on matched markets.

The table gives the typical error and the 95% band for every measured design. The 95% band is ±1.96 times the typical error: the range a single reading falls in 19 times out of 20. The run-only error is the disagreement between two teams that track the same prompts on different runs; it isolates rerun noise.

Ten prompts asked once (10 answers) had a 95% band of ±22.1 points. Ten prompts asked ten times (100 answers) narrowed it to ±15.1. One prompt asked ten times stayed at ±47.6.

Split each series' answer-level variation into a between-prompt part and a run-to-run part. The between-prompt part was 43.8% of the total when pooled over all 12,053 series (median series 40.2%). This is the prompt share of variance: how much of whether a brand appears in an answer is settled before the engine runs.

Reruns average out only the run-to-run part. For five-prompt designs, prompt choice accounted for 43.7% of the disagreement between two teams at one run, 70.0% at three runs, and 88.6% at ten runs. The more you rerun, the more of what remains is the prompt set.

Vendor guides compute a margin as 1.96 × √(p(1−p)/n), with n the number of answers. That formula assumes every answer is an independent draw. Across the measured designs the observed 95% band was 1.06 times the formula at one run per prompt, 1.44 to 1.45 times at three runs, and 2.28 to 2.36 times at ten runs.

Example: 5 prompts × 10 runs is 50 answers. The formula promises ±9.1 points; the observed band was ±21.2. With one run per prompt the formula is close. Each added run widens the gap, because the added answers repeat the same prompts.

Using the measured variance components, the table computes the 95% band for setups practitioners recommend. 15 prompts × 5 runs, a common persona setup, gives ±12.9 points from 75 answers. 40 prompts asked once gives ±10.7 from 40 answers. The rule that about 300 answers resolve a rate to ±5 points holds when those are 300 different prompts asked once (±3.9). As 30 prompts × 10 runs, the same 300 answers give ±8.7; as 100 prompts × 3 runs, ±5.3.

The binomial column shows what the independent-answers formula promises for the same design. The two columns agree only when each prompt is asked once.

For each series we computed the prompt count at which the 95% band reaches ±5 points. The median series needed 144.5 prompts asked once, 92.3 prompts asked three times (277 answers), 71.4 asked ten times (714 answers), or 65.0 asked thirty times (1,950 answers). For ±10 points the median needs were 36.1, 23.1, 17.9, and 16.2 prompts.

A quarter of series needed more than 247.9 prompts asked once to reach ±5 points. These figures extend the measured components beyond ten prompts; the components reproduced the observed error within 0.1 points at all twelve measured designs.

With unlimited reruns, rerun noise vanishes and only the between-prompt part remains. That floor is ±44.7 points for one prompt, ±14.1 for 10 prompts, ±10.0 for 20, ±6.3 for 50, and ±4.5 for 100. Ten runs get most of the way to the floor: at 10 prompts the typical error is 10.90 points at one run, 8.62 at three, 7.66 at ten, and 7.21 with unlimited runs.

The curves below give the typical error by prompt count at one, three, ten, and unlimited runs, computed from the measured components.

Error in percentage points is largest when a brand appears in some answers and not others. Series with 30–60% visibility needed a median of 382.4 prompts asked once to reach ±5 points; 15–30% needed 257.1; brands above 60% needed 330.2; brands at 5–15% needed 112.4. At 5 prompts × 3 runs, the typical error ran from 9.85 points in the 5–15% band to 17.82 in the 30–60% band.

The prompt share of variance was similar in every band, 42.4% to 46.4%, so the extra prompts are a matter of scale, not a different mechanism.

Median prompts asked once for ±5 points: 143.9 on ChatGPT Search and 145.0 on Google AI Mode. The prompt share of variance was 46.0% on ChatGPT Search and 41.7% on Google AI Mode, so reruns average out slightly more on Google AI Mode: at ten runs the medians were 74.8 and 68.8 prompts. Typical error at 5 prompts × 3 runs was 12.41 and 11.97 points.

The two engines are scored on the same markets and prompts, so the near-identical figures are not a coverage artifact.

A market leader is the brand with the highest visibility rate in a team's reading; a leader is clear when no other brand ties it. With 10 prompts × 10 runs, both teams had a clear leader in 90.0% of 1,440 market samples, and when both did they named the same brand 53.6% of the time. At 5 prompts × 3 runs (15 answers), both were clear in 57.3% of 11,080 samples and agreed 45.3%; at 3 prompts × 1 run, both were clear in 12.8% and agreed 37.1%.

Leader identity depends on the gap between the top brands. In many markets several brands sit within a few points of each other, and a 10-prompt reading cannot separate them.

The table lists the 14 active markets with the most prompts in the panel. For each, it shows the leader and its visibility rate on each engine over the full window, and the share of 40 paired samples (20 per engine) at 5 prompts × 3 runs in which both teams had a clear leader and it was the same brand.

Agreement ran from 60.0% in online therapy platforms, where Talkspace led at 68.9% on ChatGPT Search and 69.7% on Google AI Mode, to 7.5% in pet insurance, where Pets Best led at 50.9% and 45.6% with several brands close behind. A dominant brand does not guarantee agreement: Pipedrive led CRM and sales pipeline software at 92.4% on ChatGPT Search, yet the 15-answer readings agreed only 30.0% of the time, because other brands tied it at the top of small samples.

The panel keeps markets with at least 10 prompts that have 20 or more answers per engine in the window: 277 of 1,698 markets and 4,048 of 17,970 organic prompts. Series need at least 5% visibility. Raising that threshold to 10% leaves 6,113 series with a prompt share of 43.5% and a 5 × 3 typical error of 14.52 points, so the headline does not depend on the threshold.

Two disjoint samples from the same market give an unbiased error at any prompt count, because the finite-population correction cancels in their difference. The variance components predicted the observed typical error within 0.1 points at every design (5 × 3: 12.19 predicted, 12.19 observed; 1 × 10: 24.23 vs 24.28). Runs were drawn at random across 13 weeks, so run-to-run error includes drift inside the window. Brands resolve to their owning company, which is why one market whose leaders resolved to Alphabet and Meta was left out of the named table. The visibility rate is prompt-weighted, matching designs with equal runs per prompt.

Size the prompt set first. Forty prompts asked once (±10.7 points) beat 15 prompts asked five times (±12.9) with about half the answers. Add runs only after the prompt set is as large as the budget allows; three runs capture most of the rerun benefit, and ten runs reach within a point of the floor.

Report every visibility rate with its 95% band from the prompt count, not the answer count, and treat a change smaller than the band as no change. Two teams with 10 prompts × 10 runs agreed on the market leader about half the time, so a leader claim needs a margin over the next brand, not just a rank.