METHODS

How we work out what is actually happening

Most applied analytics fails at choosing the method, not at running it. This page is about that choice.

THE STARTING POINT

People do not always do what they say they do

That sentence sounds obvious and it is the reason most marketing research is weaker than it looks. A survey records what somebody was willing to tell you, some time after the fact, in the language your question gave them. Behavior records what actually happened and never explains itself.

Two kinds of evidence, and why neither is enough aloneWhat people say— Surveys and polls— Interviews and focus groups— Reviews and ratings— Social listeningExplains why. Recalled, and edited in the telling.What people do— Clicks, sessions, dwell time— Cart and checkout behavior— Search terms actually typed— Purchase and reorder recordsRecords what. Never says why.The gap between the two is the finding. We look for where they disagree.
We look for where the two disagree. That gap is usually where the useful finding is.

So we use both, deliberately, and we say which is which. When a client hears “customers want X,” the first question is always whether that came from something they said or something they did — because the two carry very different weight.

THE DECISION PATH

Most of a purchase happens where nobody is looking

The buying decision runs through recognizable stages, and the amount of evidence available at each one is wildly uneven. Analytics sees the transaction clearly and the three stages before it barely at all — which is exactly backward from where the decision was actually made.

Where the evidence exists at each stage — and where it does notNeedsomething changesSearchengines, marketplaces, sComparereviews, price, proofBuyor abandonAfterreview, return, repeatinvisiblepartly visiblemostly invisiblefully visiblepartly visibleMost analytics only sees the fourth box. Most of the decision happened in the first three.That is why we pair behavioral data with the kind you have to go and ask for.
The stage where the choice gets made is usually the stage with the least data behind it.

This is why an assessment that only reads the analytics account will tell you what happened and almost nothing about why. Search terms, review language, competitor comparison and social conversation fill in the earlier stages, and they have to be gone and got.

CHOOSING
Dyed eggs laid out in a continuous gradient from pink through orange, yellow, green, blue and violet.
A gradient looks like one smooth thing from above. Up close, every step was somebody’s decision about where the next band starts.

The method is the decision that matters

Nearly every serious mistake we see in applied analytics is a method-selection error rather than an execution error — a predictive model answering a causal question, a significance test run on data that breaks its assumptions, a segment built from a variable that does not drive anything.

Five questions asked before any method is chosenIs this a why question or a how-many question?Why → qualitative first. How many → a sample that can carry the claim.Does the answer need to generalize?If yes, the sample decides everything. If no, depth beats breadth.Can this be observed instead of asked?Observed data avoids the gap between what people say and do.What else changed in the same period?Without this, any result is a guess wearing a number.What would prove me wrong?If nothing would, it is not a finding. It is an opinion.
Choosing correctly matters more than being able to run any of them.

Being able to run a technique is not the hard part and has not been for years. Knowing which one the question calls for, and being willing to say the data cannot support the conclusion somebody wants, is the part we take most seriously.

THE TOOLKIT

What we actually use

Behavioral

Analytics accounts, event data, search console, server logs, session and heatmap tools, marketplace search data, and the client’s own order history — usually the most underused dataset in the building.

Attitudinal

Structured surveys, on-site and exit-intent polls, interviews, review and ratings analysis, and social listening for the things nobody was prompted to say.

Technical

Full-site crawls, metadata and structured data audits, page speed measurement, accessibility checks, and retrievability testing — including whether AI assistants can reach the pages at all.

Comparative

The same instrument run across a population, so a result means something against a comparable set rather than against an opinion. Public advertising archives. Competitor structure read from what is publicly visible.

Experimental

Split tests where volume supports them, holdouts and geo tests where it does not, and an honest answer about when a business is simply too small to test its way to certainty.

Contextual

A confounder log kept from day one — what else changed, whose change it was, and whether attribution survives it. This is the sheet that separates a study from an anecdote.

THE HARD PART

What we will not claim

We will say

  • This is what the data shows, and here is the size of it
  • This is observed behavior; that is stated preference
  • These two things moved together, and here is what else was happening
  • We do not have enough here to answer that yet, and this is what it would take
  • Our earlier recommendation was wrong, and here is the number that changed it

We will not say

  • That correlation is causation because the client would prefer it
  • That a test reached significance when the sample cannot carry it
  • That a rise was ours when three other things changed the same month
  • That a method is appropriate because it is the one we happen to own
  • That a finding is settled when it is a hypothesis worth testing

Findings do not get edited on the way to the client. Independence is the whole reason an outside assessment is worth anything, and if a finding gets softened it stops being useful to anybody.

WHERE IT COMES FROM

Twenty-five years, then the method

Experience without method produces confident guesses. Method without experience produces a framework applied by somebody who has never had to make payroll from a website. We think you want both, and we are unusual in having taken them in that order.

The operating years came first — a catalog business run through marketplaces, wholesale and direct, with the analytics read every morning because the rent depended on it. The formal training came afterward, deliberately: statistics, data science, research design, consumer behavior. Not to learn marketing, but to find out which of the things twenty-five years had taught would survive contact with a method.

Most of them did. A few did not — and those are the ones worth talking about.

START HERE

Send us a web address.

We’ll give you a free AI analysis and tell you the three things we’d fix first — in writing, for nothing.

Get in touch