A design retailer, crawled properly

1. What the business is

A designer home-goods brand in Maine, founded in the late nineties and still run by its founders. Rugs, textiles and home accessories, every product individually named, with prices running from under a hundred dollars to several thousand. A physical store, a Shopify storefront, and a wholesale side.

This is a brand with genuine design credibility and a loyal following. Nothing about the assessment was a rescue. It was the ordinary case — a good business whose website nobody had examined closely in some time.

2. The first crawl was wrong, and catching that mattered more than the findings

A full technical crawl of the site returned 444 responses. One hundred and fifty-four of them came back as 429 Too Many Requests. On the face of it that looks like a site failing under load, which would have been a serious and alarming finding.

It was not. It was our own crawler. Hosted storefronts rate-limit automated traffic, and at default crawl speed we had triggered that limit ourselves. Every one of those 154 errors was manufactured by the tool, not by the site. There were zero genuine 404s.

We re-ran the crawl at one thread and half a URL per second. The errors disappeared. Had we shipped the first report, the client would have spent money chasing a problem that did not exist — and would have been right to distrust everything else in the document.

3. What the clean crawl actually found

Page titles and meta descriptions measured against what mobile search results actually display, rather than against a desktop assumption. Heading structure. Canonical tags and redirect chains. Structured data. Security headers. Image weight, page by page. Orphan pages that nothing on the site links to.

The report ran to sixty pages, organized by what each finding costs rather than by technical category — because a client does not need a list of everything that could theoretically be improved. They need to know what to do first.

4. What we would not claim

A crawl is a snapshot of what is technically there. It does not tell you whether the photography is working, whether the product naming helps or hinders search, or whether the wholesale side is being served by the same site as the retail one. Those are separate questions and we said so.

We also quoted the marketplace build separately rather than folding it into the assessment, because bundling a build into a diagnosis is how assessments stop being trustworthy.