Testing

The point

You don't tell the market, the market tells you: you ask questions, and the market tells you with their response or lack. A control is what's working now. Test one variable at a time.

Jay Abraham · Redondo Beach 2023 — Day 1, Session 3: Perry Marshall Quadrants Cued to 2:54 11 min 45 sec loading…

Why it matters

Direct response rests on one assumption, and it separates the businesses that compound from the ones that guess: you do not tell the market, the market tells you. Every opinion in your building about what your buyer wants is a hypothesis wearing a suit — yours, your partner's, your agency's. It costs nothing to hold and a fortune to act on. Ten headlines, ten offers, ten prices is a thousand combinations of the same product sold to the same market. Three headlines, three images, two calls to action and two layouts is thirty-six variants, and the winner routinely runs five to ten times the loser. You built nothing new. You found out which version of what you already own the market responds to. A designed grid is the opposite of changing four things at once: every combination is planned and measured, so each factor keeps its own attribution. The discipline is a single variable at a time, measured against a control — whatever is working for you now — and you graduate to the grid when your volume can carry it. Change the headline, the offer, the positioning, the proof — and you change the result. It is not only a growth instrument. It is the cheapest way to rationalize overhead. I do not want to cut a cost because it went up, if that cost is producing fifty percent of my profits — and until you have tested it, you do not know which one it is.

The mistake almost everyone makes

Testing everything at once, or testing nothing because a proper test looks like work. Both produce the same outcome — a number moved and nobody in the building able to say what moved it, which is not a result, it is a rumour.

The test: Name the variable that changed and the number you are watching. If you cannot name both in a single sentence, you are not testing, you are redecorating.

Where it shows up — 8 worked examples

Who	What happened
The hotel towel-reuse researchers	Tested a sign saying most previous guests in that very room had reused their towels. It beat the appeal to virtue, at zero added cost.
Booking.com	Did not guess its way to higher conversion. It built a machine for discovering the causes of conversion faster than any rival could.
L'Axelle	Replaced comfort-and-freshness copy with the blunt line Put an end to sweat marks. Add-to-cart actions rose 93%.
BettingExpert	Rewrote the header and button to name the benefit and left the form exactly as long as it was. Signups rose 31.5%.
CareLogger	Replaced its convenience promise with a page that made the customer's underlying pain vivid. Signups rose 31% on the reframing alone.
WikiJob	Added sober testimonials from people like the buyer. Sales rose 34%, because the buyer had been making a trust decision all along.
37signals	Added a large photo of a real person alongside a simplified message to the Highrise page. Signups rose 102.5%.
The Indianapolis protégé programme	An offer written out on a yellow pad in 1991 and tested live rather than polished. Twenty-three million dollars in six months.

What it solves

The words a business owner uses for this before anybody has told them the name of it. If one of these is a sentence you have said out loud, this is your strategy.

We changed it and something happened“Our numbers moved last quarter and I genuinely could not tell you which of the four changes did it.”One variable moved against a defined control, so the number that moves has a name attached to it and can be repeated on purpose.
The copy is fine we need more traffic“The page converts about as well as it ever has, so the answer must be to send more people to it.”The arithmetic of a thirty-six-variant grid on the page you already own, run before a pound goes on buying more visitors to it.
Nobody here agrees on the headline“We spend more time arguing about what to say than we spend saying it.”The end of the argument — a control, one challenger, and the market as the only vote in the room that counts.
We tried testing once“We ran a test a while ago, it was inconclusive, and we quietly stopped.”The discipline that makes a test conclusive: a single variable, a defined control, and the figure you are watching written down before it runs.
We are cutting costs and guessing“Costs have gone up and I am about to cut things without really knowing what any of them produce.”The same instrument pointed at overhead — what each cost is actually producing — before you cut the one carrying half your profit.
The advisor's version“My clients want me to tell them what will work, and my honest answer is that I would be guessing too.”A method you can hand a client that replaces your opinion with their market's answer, and makes the engagement measurable from the first day.

The challenge

Put one variable to the market in the next seven days.

Pick the single message that reaches the most people — the subject line, the headline on the page they land on, the first sentence of your proposal. That is your control. Write one alternative that changes exactly one element of it. Run both against the same audience in the same week. Then tell your pod which one won and by how much, before you touch anything else.

How you will know it is done Two versions run against the same audience, a single variable apart, and a percentage difference read out to your pod at the end of the seven days.

The AI layer · second pass

Once this is working, here is what to multiply

Improve the system first. Then multiply it. This panel is the second run at the strategy above, and it is deliberately useless until the first run is done.

Before you point anything at this You have run one test by hand, one variable against a control, and read the result.

What it multiplies Generating and running hundreds of variants across the thousand combinations of headline, offer and price, and reporting which single element moved the number.

The trap Declaring winners on volume rather than evidence. A model will happily run a thousand variants at once, move eight elements in each, and hand back a confident ranking nobody can reproduce, which is not data, it is a rumour with a chart on it.