What a 30,000-Product Catalog Taught Us About How AI Reads Your Store

A store with 30,000 SKUs installed FoundGPT and broke our generator five times. Fixing it taught us more about AI-readiness for large catalogs than any best-practice guide — because the constraints we hit are the same ones every AI system hits reading your store. War story + the big-catalog playbook.

Most Shopify stores have a few hundred products. The AI-visibility advice ecosystem is written for them: "enrich every description!", "add schema to everything!", "make every page citable!"

Then an industrial-supplies retailer with roughly 30,000 products installed our app, and their onboarding crashed our catalog processor. Five times. Out-of-memory, every time, while trying to build their llms.txt and readiness score.

We fixed it (attempt six: 59 seconds, clean). But the reasons it kept breaking are worth your time even if you'll never sell 30,000 SKUs — because every constraint our pipeline hit is a constraint ChatGPT, Gemini, and every shopping agent also hits when they try to read a store. We just hit them first, loudly.

Lesson 1: Nothing reads everything

Our first naive approach — load the whole catalog, then process — is also the naive mental model merchants have of AI: "the AI will read my whole store." It won't. It can't. At 30,000 products, "everything" is hundreds of megabytes of HTML; no crawler budget, context window, or attention span covers it.

What we shipped instead is what every AI system does: stream, sample, and cap. Our generator now processes page-by-page and caps the catalog file at 5,000 products, noting the truncation honestly in the file itself.

The merchant takeaway: for a large catalog, the question is never "is everything covered?" — it's "do the things that matter make the cut?" Curation beats completeness, because something is doing the cutting whether you choose it or not. Better it's you.

Lesson 2: Description bloat starves everything else

The killer wasn't product count — it was that a few thousand products carried enormous HTML descriptions (spec tables, legacy formatting, pasted PDFs). A handful of whale pages consumed more budget than thousands of normal ones.

We now truncate descriptions to their first 2,000 characters at the source. And here's the thing: almost nothing of value lived past character 2,000. What lived up front — when the page was written well — was the product's actual identity: what it is, who it's for, the key specs.

The merchant takeaway: front-load. Your first two sentences should carry the product's identity and its top facts, because extraction systems weight beginnings, truncate ends, and skim middles. A 4,000-word description whose only good paragraph is at the bottom is, to a machine, a page with no good paragraph.

Lesson 3: Structure survives where prose doesn't

When you must compress 30,000 products into what a machine will actually consume, the content that survives compression is the structured kind: name, type, price, availability, category, key attributes. Prose gets truncated; JSON-LD fields get kept. This is exactly why structured data punches above its weight for AI visibility — it's the part of your page that's already in the machine's preferred shape (and why we obsess over stores shipping duplicate or incomplete schema).

The merchant takeaway: on a big catalog, complete structured data on all products is worth more than beautiful prose on some. Do schema first, at 100% coverage; polish descriptions by priority tier after.

Lesson 4: Big catalogs need a priority order, so here's ours

When only ~5,000 of 30,000 products can be first-class citizens, ranking is the strategy. The order we use, and recommend:

1. Revenue earners — your bestsellers, always in.
2. Question magnets — products in categories buyers actually ask AI about (comparison-heavy, research-heavy purchases).
3. Differentiators — products only you (or few) sell; the long tail where an independent store wins AI answers outright.
4. Everything else — covered by clean collection pages and category-level content rather than per-product enrichment.

Collections deserve special mention: our citation corpus records 3,679 citations to /collections/ URLs. For the tail of a big catalog, one excellent collection page ("Industrial Fasteners — grades, materials, how to choose") can represent a thousand SKUs no engine would read individually.

The postscript

That retailer's score went from 0 to 71 once processing survived, and their team spent a quarter of an hour inside the resulting report on first open. Large catalogs aren't disadvantaged in AI search — they're disadvantaged by undifferentiated treatment of 30,000 pages. Ranked, structured, and front-loaded, a big catalog is 30,000 lottery tickets.

Big catalog? The free scanner will tell you what machines can currently extract from it: foundgpt.app/scan · FoundGPT for Shopify

The merchant in this story is anonymized. Numbers (catalog size, caps, truncation thresholds, processing outcomes) are from our production systems, August 2026.

About the author

Rahul — Founder, FoundGPT

Rahul built FoundGPT and ran the analysis behind this piece. Across 111 Shopify stores he has run 23,000+ real AI-search checks on ChatGPT, Gemini, Google AI Mode and Perplexity, and applied ~99,000 structured-data fixes.

→ About FoundGPT