What Is a Marketing Experiment?

Somewhere along my career, I fell in love with experiments because they helped me avoid arguments.

Someone in the meeting says, "I don't agree with you" but what they usually mean is "I have a strong opinion." Opinions don't represent our customers.

Yet we spend hours, days sometimes, in endless debate over what the customer really wants.

That is precisely a marketing experiment solves. It is a structured attempt to learn whether a specific change causes a measurable, observable and predictable business outcome.

That distinction sounds semantic at first. "We already have data" I hear people say. It's the origin of that data and the chain of actions from that data that makes all the difference.

What is a marketing experiment, really?

My working definition is simple. A marketing experiment is a controlled test designed to observe expected customer behaviour reacting to a change we make in the market. What follows that is how we act after getting that validation - or invalidation.

The key phrase there is 'controlled test'. We control every variable possible (independent) when allowing for one (dependent) variable to change. Why? Because we have to factor in the myriad of other influences that could change behaviour:

I'm dead serious about each of these being factors that affect behaviours:

• seasonal sales volumes

• weather

• market shifts

• the meme culture of the week

• celebrity news

• price of gas

• technology outtages

• you name it

A proper experiment has a few non-negotiables:

• a defined hypothesis

• a variable you are changing

• a measurable outcome

• a comparison or baseline

• a decision ruleset for what happens next

That last one gets ignored all the time. Teams run tests, collect data, then drift straight back into opinion. If you do not decide in advance what you plan to do given the result - then you're inviting more endless debate.

Decide in advance. Then go test.

Why some teams resist experimenting

"We're already experimenting" -- Performance teams often believe they are testing when they regularly update ads, emails, landing pages, or offers. That is normal execution. Experimentation is a disciplined rigour above execution that ties all the way up to strategy. What do we need to learn to achieve our goal of X?

"Don't know how to start" -- A test starts with a real hypothesis. When a team says "Let’s try a shorter form" or "Let’s test a new headline." To what end? What behaviour do we expect to change? More form starts? Better lead quality? Faster decision-making? If the thinking is fuzzy at the start, the learning will be fuzzy at the end.

"We get flat results" -- Metric choice is essential. There's both an art and a science to designing an effective experiment that delivers a conclusive result. The further a metric is from payment, the harder it is to justify downstream revenue. The closer we measure to revenue, the more we risk too-few conversion events to register what we call "sensitivity" or the minimum detectable effect.

"We just argue" -- That's a leader problem. Nobody likes being wrong, but if a failed test feels like a personal embarrassment people stop proposing ideas. They start suggesting safe, forgettable changes that keep the peace and teach the business nothing. This, friends, is the single biggest barrier to experimentation that exists and it's everywhere. I'm on a mission to fix it.

That is why I coach leaders to be wrong first. It gives everyone else permission to think creatively and be wrong safely. If the boss shared a rough idea, then I can too. One of those rough ideas holds the growth gem you've been aching for.

Not everything deserves to be an experiment

Some ideas are just good marketing and you should go do it:

Fundamentals, major gaps everyone can see, treating customers better, improving your product. Do those things and don't guess whether they're valuable. Measure everything so you can declare a lift - but know it's simply good practice.

A marketing experiment, on the other hand, is designed to inform a decision. The decision might be to scale, stop, refine, or investigate further. Sometimes the best outcome is not a winner. It's clarity and team alignment.

For growth-stage leaders, clarity is underrated. Experiments stops us from wasting six months on channel tactics that felt clever but never moved revenue.

For enterprise-level leaders, experiments are career-changing. They show modesty, a willingness to learn, a commitment to customers, demonstrate precision in the process, data fluency, and experiments even help us tell better stories.

Ever heard a leader start their keynote with "okay, so what we wanted to learn was..." and then a few sentences later "but what really happened was..." and did anyone, ever, look away? Experiments are captivating. There's a natural story arc. There's a shared villain, 'uncertainty', that we all want to see defeated.

What a marketing experiment looks like in practice

Here is how I usually frame one.

We have a problem. Conversion from product page to checkout is lower than expected. We believe buyer friction is coming from uncertainty around fees. Our hypothesis is that making fees clearer earlier in the journey will increase checkout starts without lowering order value.

That is already stronger than, "Let’s refresh the page, it's overdue."

From there, the experiment takes shape:

• Problem: low product page to checkout conversion

• Hypothesis: earlier fee transparency reduces hesitation

• Change: revised pricing module and supporting copy

• Primary metric: checkout start rate

• Secondary metric: completed purchase rate

• Guardrail metric: average order value

• Decision rule: roll out if uplift is commercially meaningful and guardrails hold

This is not complicated. It is disciplined.

Where marketing experiments should focus

In most businesses, the best experiments sit near moments of friction, confusion, or drop-off. That sounds obvious, yet teams still spend too much time testing cosmetic details because they are easier to change.

I tend to look in four places first:

• messaging clarity

• offer structure

• conversion friction

• audience and intent alignment

Messaging clarity matters when people do not understand what you do, why it matters, or why they should trust you. Offer structure matters when the proposition is technically sound but commercially muddy. Conversion friction matters when motivated buyers stall because forms, flows, proof, or next steps create effort. Audience and intent alignment matters when you are sending the wrong traffic into the right funnel, or the right traffic into the wrong promise.

This is where a lot of wasted budget hides.

What is a marketing experiment not?

It is not a campaign report with a positive spin.

It is not a dashboard full of numbers with no decision attached.

It is not changing creative every week because fatigue might be happening.

It is not copying what another SaaS or fintech brand did and calling that validation.

And it is definitely not a licence to throw ideas at the wall faster because AI made production cheaper.

Cheaper production without sharper thinking just gives you more ways to be wrong at scale.

How I tell if an experiment is worth running

I use a fairly blunt filter.

First, is the problem meaningful enough? A test that might improve a leading metric by 1-2% is rarely worth cross-functional delivery effort.

Second, is there an underlying and understood mechanism? I want a human reason the change should work. A reason grounded in buyer behaviour. I call this a premise, and it's essential to compound growth and future wins.

Third, can we measure the outcome with enough confidence to make a decision? If not, we may still proceed, but we should be honest that we are running an exploratory test, not a definitive one.

Fourth, what will we do if it works? And what will we do if it fails? If nobody can answer that, the experiment is not ready.

This simple discipline saves an extraordinary amount of arguing. Arguing sucks.

Why this matters beyond marketing performance

A strong experimentation practice does more than lift conversion or reduce acquisition waste. It changes how teams work together.

At Formentor Labs, this is the part I care about most. Not just running tests, but helping teams get comfortable learning in public without turning every disagreement into a political event.

A better way to think about experimentation

If you remember one thing, make it this: a marketing experiment is a decision-making tool.

Yes, it can improve performance. Yes, it can reveal hidden revenue opportunities. But its deeper value is that it helps teams replace guesswork with evidence while staying human enough to admit uncertainty.

You do not need to pretend you know the answer before the test starts. In fact, that is usually where the trouble begins.

Be wrong on purpose. Write the hypothesis down. Put a real metric against it. Let the market answer back. That is where useful growth starts.