Case Study · Ecommerce: baking, brewing, and restaurant ingredients
OliveNation: Taming 630,000 Junk URLs on BigCommerce Enterprise
5,223 redirect rules added in three weeks
OliveNation, a wholesale supplier of baking, brewing, and restaurant ingredients, came to me with Google reporting roughly 630,000 crawled but unindexed URLs against a catalog of about 950 products, nearly 11,000 legacy 404s, and a robots.txt that Googlebot was silently ignoring. Starting in April 2026, I closed the robots.txt loophole, mapped and imported more than 5,200 redirects in three weeks, delivered a full technical audit and a competitor-driven keyword gap analysis, and engineered around BigCommerce's hard platform limits. The engagement is ongoing, with implementation complete on the highest-priority crawl fixes.
Results SnapshotThe Situation
The Challenge
OliveNation sells roughly 950 products across 186 categories. Google, however, was wrestling with a site hundreds of times that size. As of late April 2026, Search Console reported about 630,000 URLs sitting in "Crawled, currently not indexed," almost all of them faceted filter combinations that Googlebot kept recrawling. That works out to roughly 660 junk URLs for every real product in the catalog.
The causes compounded each other. The robots.txt file contained a Googlebot block with an empty Disallow line, which meant Googlebot ignored all 25 rules that every other crawler obeyed and was free to crawl cart, login, search, and filter URLs. A platform migration years earlier had left 10,987 dead URLs returning 404s as of April 28, 2026, much of it legacy .aspx addresses. A May 2026 audit crawl found 82.1 percent of pages carrying unused CSS from the shared theme and 67.9 percent rated poor for JavaScript execution. And the platform itself fought back: BigCommerce, even at the Enterprise tier, offers no wildcard redirects, no 410 status codes, and no server-side noindex.
The Work
What Was Done
Closed the robots.txt loophole without breaking what worked
I copied 24 of the 25 disallow rules into the Googlebot block and deliberately left out the blanket query-string rule, because paginated category URLs were earning real traffic: a single page-two category URL had drawn steady clicks and impressions in the 90 days before the change. I validated the update against Google Merchant Center, Shopping, and AdsBot behavior and live-checked legacy endpoints before recommending it.
Cleared the 404 backlog in two controlled batches
Batch one mapped 1,834 301 redirects with confidence scoring, imported at the end of April; when 249 rows failed on import, I diagnosed and fixed them the same day. I then built a client review workbook that triaged the remaining 3,388 dead URLs into confidence tiers with a simple approve, edit, or skip workflow, and batch two added 3,368 more redirects on May 21. Just as important is what I refused to redirect: 1,173 SKU-parameter URLs were left untouched to protect Merchant Center feeds and paid campaigns.
Gave the client an honest strategy for the 630,000-URL backlog
I wrote a costed decision paper laying out three paths, from doing nothing extra all the way to a customer-owned Cloudflare Worker returning true 410 responses on spam-pattern URLs, a workaround for BigCommerce's missing wildcard and 410 support. My recommendation was the zero-cost option first: let the completed fixes work before buying infrastructure.
Audited the whole site and reduced it to six work streams
A full crawl of roughly 1,163 pages surfaced 19 distinct issue types, which I grouped into six prioritized work streams with clear ownership split between my lane and the client's developer. The key reframe: since 82 percent of pages shared the same unused CSS pattern, this was one root cause in the global theme, not 954 separate page fixes.
Wrote the schema by hand
I authored Organization and WebSite JSON-LD with site search markup for the homepage, then diagnosed product pages emitting nine structured data blocks with duplicated return-policy markup and queued the cleanup alongside the script work.
Stabilized layout shift through Script Manager
I built font fallback metrics and image aspect-ratio reservations to cut cumulative layout shift on product and category pages. When the first deployment broke the homepage hero, I rolled it back in the same session and rebuilt a safe version that deliberately excludes hero rules until the developer confirms the theme's actual dimensions.
Found the demand the catalog was missing
A keyword gap analysis built from 19 competitor domains, 90 days of Search Console data, and the full product and category sitemaps produced a universe of roughly 8,300 keywords. From it I recommended 10 new category pages, each validated against competitor rankings, search volume, and an inventory floor. Themes with strong demand but thin stock, like one high-volume ingredient category backed by only 4 SKUs, were routed to a sourcing conversation rather than padded into the list. A quick-wins tab flagged 65 striking-distance terms, including a high-demand category query where the client sat at position 14.7 and the best-placed competitor held position 4.
The Payoff
Results
The engagement was still in flight at the time of writing, so the figures below document verified implementation from platform exports rather than final traffic outcomes. Exact traffic figures and keywords are confidential to the client.
| Metric | Before | After |
|---|---|---|
| Redirect rules live in BigCommerce Redirect Manager | 3,478 (export dated April 30, 2026) | 8,701 (export dated May 21, 2026) |
That is 5,223 rules added in three weeks, a 2.5x increase, resolving the bulk of a 10,987-URL 404 backlog measured on April 28, 2026, while deliberately protecting the feed and pagination URLs that were already earning traffic. Post-implementation indexation and traffic measurement is in progress. Performance data available on request.
Tools used: Google Search Console, SEMrush, GA4, Lighthouse and PageSpeed Insights, desktop crawl audits with GSC and GA4 API integration, BigCommerce Script Manager and Redirect Manager, and hand-written JSON-LD.
Want results like this for your site?
Get a free SEO audit and a plan built on what actually works.