Do you think you have a good eye for design and user experience? Do you know what will move customers to act?

Prove it.

So…how did you do? We’re thinking not too great, and that’s okay.

We talked to Deborah O’Malley about all this. She is the founder of GuessTheTest, an A/B test case study resource focused on helping digital marketers increase conversions and get new ideas and insights from testing. She says, "In CRO testing, your chances of guessing the right test are about equal to guessing the correct side of a coin toss. Don’t make assumptions.”


Feel better? We all love our biases and assumptions, but we’re with Deborah. You need to rethink yours.

Most Conversion Rate Optimization (CRO) is done to increase the conversion rate of a SaaS sign-up form or an e-commerce product page. It involves taking a critical page or conversion point, creating 3-5 variants of that same page (each with one single tweak), using a tool like Google Optimize or Optimizely to run live traffic to each of those variants, and then discovering the "winner."

CRO - done right - enables marketers to step out of their biases and actually begin to understand their customers. Still, we’ve found that CRO is largely neglected. Econsultancy reports that 50% of companies value CRO as a crucial part of their marketing strategy, but that only 1% are very satisfied with their conversion rates.

Guess the Test shares, “The average conversion rate hovered around 3% in 2020. That means of 100 visitors coming to your website, only 3 out 100 are taking the desired action you hope they’ll perform, like purchasing your product.”

Ouch. Econsultancy also found that businesses that successfully boost conversion rates perform 50% more tests. This statistic speaks for itself. More tests, of the right tests, is better.

But still, companies spend just $1 on CRO for every $92 spent on customer acquisition. Samantha Brown, the VP of Enterprise Client Services at 97th Floor, explained, "There is a huge gap between what we’re willing to pay for traffic and what we’re willing to pay to turn that traffic into customers.


Seems off. CRO should be a higher priority, so we set out to discover the major roadblocks here and how to overcome them.

We’ve got some pointers.

CRO Isn't a Tactic

CRO is a methodology, but we’ve probably all got it labeled as a tactic. Big misunderstanding.

As a manager, you focus on systems for acquisition, monetization, and retention. To improve all of these systems, you need to think of CRO as a method for innovation and not just a tactic. It’s not a phase - it’s a lifestyle, because the moment you stop testing, you’re saying “my customers aren’t living, breathing, changing humans,” or “I don’t care to keep learning from them.” Does that feel extreme? Yeah. So does not testing.

So, keep testing. Shiva Manjunath, Senior Strategist at Speero, is passionate about testing to learn. Whether or not your test is a “win” for conversion, the results are invaluable for understanding your customers. What you learn in each test should inform the next test you run. Shiva says, "The ripple effects and learnings of web testing are more impactful measurements of success than the individual metrics you move."


The CRO Shiva is talking about is more than changing the CTA button color or placing the CTA in a new location. He’s concerned about understanding his audience through the tests he runs. He’s more focused on experimentation —a mindset shift we all need to make.

Shiva Manjunath continues, “We need to unlearn CRO and relearn experimentation. We are running experiments on the website to optimize for the business KPIs and sometimes that’s conversion rate optimization. But sometimes we see CRO and think all it is is optimizing front end conversions when in reality you can run experiments on whatever you want.”

Want more expert marketing content sent right to your inbox?

Join our newsletter and never miss a beat.

Be careful to not let the title “Conversion Rate Optimization” limit your efforts. If the term “CRO” focuses marketers on optimizing for conversions as opposed to experimenting for better audience understanding, that’s a problem. Expand that definition. You’re a marketer, but you should also be a scientist. Ask questions. Create a hypothesis. Relearn experimentation, and realize that CRO is just one important facet of that.

CRO can’t be a checkbox hire. It can’t be a checkbox procedure, either. Building a culture of experimentation will pull in those amazing benefits of CRO we all hope for but don’t know how to get.

Do This: Incorporate experimentation into every aspect and role of your business. Make it clear to every team member that the questions they have about their audience can be answered.

Creatives Will Do It Best

CRO can feel scientific, but it is also an art. Ben Labay, Managing Director at Speero, explains: “There’s an art to the systems approach to CRO. If you get a big win on a landing page test for e-commerce and you get a 10% lift in transactions, that’s cool. What would happen if the sales team or the customer success team knew customer behavioral, psychological principles that went into that change in behavior? Then you could start standing on the shoulders of learnings and gain an unfair advantage.”

One definition of creativity is simply being aware of all the tools at your disposal and then knowing which tools to use and how to use them to solve your problem. One step further, it's taking all of the information and insights you gain across an organization and finding connections between them.

If CRO is limited to X% increase on Y page, no learnings are gathered and no connections are made. If CRO learnings stop after one single test or are held within one single department, connections can't be made across the organization.

Shiva agrees on this one: “There has to be a level of creativity when it comes to experimentation because you’re doing creative problem solving. You have a hypothesis that you need to test. The hypothesis can be tested in an infinite number of ways, the execution can be done in an infinite number of ways. So you have to guard rail it into specific pages and specific audiences. You have to understand how you are going to analyze a specific test. Then you have to work within the limitations of the site's ability to be modified.”

The art is also in what you do with the data. Are you finding the story in your test results? Are you really thinking with the intent to understand your customers? Is this story shared beyond you and your team? Take your learnings and share them widely. Democratize CRO and every test thereafter will compound.

Do This: Compound learnings, share insights, and get creative. Sometimes it’s the third test that gets you to the earnings. Sometimes, it’s the learnings from Sales and Customer Service experimentation that will reveal the best next step.

Get Buy-In and Get Started

Here’s Shiva again: “You can’t get anywhere if you don’t have leaders who believe in experimentation.”

So, what do you do without leader buy-in? How can you create this culture within your company?

First, stay focused on data. It’s probably a lot easier to argue data-backed decisions than CRO-backed decisions. We know they’re the same, but maybe your team hasn’t caught the picture yet. So, focus on data and show how CRO is an extension of data—a research tool—actually, the best research tool.

Ben Labay explains that there are two types of data. The first type comes from existing analytics, machine learning data and other forms of big data. This data is old when you pick it up and start to make decisions based on it. Ben warns that when using this data, “you are ripe to trip on your own cognitive biases or on your own confirmation biases.” The second type of data comes from CRO and it will confront your biases.

Ben says, “CRO and experimentation is more about intervention. It’s about coming into a situation, changing something and measuring the effect of those changes. This is “just in time” data. It is a step higher in the causal ladder to understand the mechanism behind what caused the change that you see in the data. Objectively, it’s a better type of data. It gets closer to the mechanism of why something is the way it is. You learn more precisely and more accurately and at a faster rate.”

This second kind of data is so valuable because it is intentional, living and “just in time” for you to step into your customer’s journey and really think about how the change you are testing caused the data you end up with. You’re intervening in an ongoing process, always adapting in real-time based on multiple levels of creative testing. You’re engaged with data and your audience in a whole new way. Ben wraps it all up nice when he clarifies, "Analytics is data that you see. CRO is data that you do."


Pretty compelling stuff, so we recommend you just start. Not in a rebellious way. We do not want an office coup over CRO. But what’s your role? And where can you test? Start experimenting. Use your insights to create the next test. Then be vocal about how experimentation is changing the game for you, and other teams will hop on.

Finally—and especially in a leadership role—educate.

Shiva shares, "People see experimentation as something that slows down decision making. The reality is you need experimentation to make better decisions so you don’t crash."

Shiva continues, “You need to teach people. There’s a lack of education. There are some people that just don’t want to run tests because they don’t want to be proved wrong. But honestly it just needs to be reframed as a partnership. We’re not here to prove people wrong, we’re here to make you look better.”

The right education and persistence can tip leadership towards CRO, and once they’re there you’ve got them. Jeremy Epperson, Chief Growth Officer at ​​ConversionAdvocates, says, “You don’t know how much ROI you will get on a brand new channel or campaign. Why would you hold CRO to a higher standard? There is no guarantee of results in life. You just need to make the case to get started with CRO.”

Do This: Make your case. You have a small window to prove the value of experimentation. Use low-hanging fruit opportunities to educate and prove the value of CRO quickly to get buy-in and high fives all around.

Listen, you’ve got this. Experimenting is exciting! And once you get started, the fire will catch and your organization can increase conversion and sales and everything else with this new “just in time” data.

And for your first experiment? Try dropping this article in your company’s slack channel. Start the conversation. Just see what happens.