The Eternal Struggle of A/B Testing Buttons

Have you ever heard a PM being referred to as “a person who A/B-tests the button color”?

Even though this is a rather jocular definition, sometimes we do indeed end up running numerous low-impact UI improvement experiments.

In this article, we would like to tap into the common WHYs behind this obsession with minor refinements and outline the ways that PMs can up their game and deliver higher-impact projects.

Let’s start with the WHYs

1. These are the simplest projects to execute

A typical product team consists of a product manager, a designer, an analyst, and engineers. There may also be a QA, a researcher, etc. These are all the people whose priorities are managed directly by a PM. In such a setup, the easiest project is the one that the team can deliver in complete isolation, where there’s no need to align timelines with other teams, no budget to negotiate, no non-product stakeholders to manage, and no one else to secure a commitment from.

Don’t get me wrong—often these projects may indeed be “quick wins”, but product leaders have to stay vigilant and challenge the PMs as to whether these “redesign” and “UI improvement” initiatives are truly the highest-impact ones.

All in all, product teams often end up taking on these projects because the more complex ones are simply too hard to execute—and this is the next root cause we’ll look into.

2. Dependencies are not managed in advance

If a product team gets blocked by its dependency on another team, this means the initiative has to be prioritized in someone else’s backlog.

There are plenty of reasons why this can be problematic, and the most notorious ones are as follows:

• The company’s processes are not lean enough. In the case of a large business where prioritization is done, say, once a year, there may simply be no formal process in place to re-prioritize and squeeze a project in mid-year. If a PM missed their shot at prioritizing an initiative on time, it may end up buried in the backlog for months. Meanwhile, the team will keep working on those simple projects that don’t have external dependencies.

• No clear prioritization process. In the previous example, even if a product leader did everything right and flagged an external dependency on time, there may not be a clear framework to compare the impact of two different projects.

  
  Consider this: how do you prioritize between an acquisition and a retention project? If you’re thorough enough, both projects will get quantified against the same metric (e.g. incremental revenue over 12 months), but the estimation framework needs to be solid and fair.

• Lack of upward management. Cross-department prioritization is frequently handled by the department leads and may not involve ICs. For a PM it is extremely important to manage their boss and other decision-making parties, raising awareness of their team’s blockers and dependencies.

To sum up, more often than not, a regular product team has to collaborate with others to deliver maximum impact—and managing these external dependencies is an entirely separate set of skills a PM should master.

Finally, we’d like to point out another complication that frequently limits the impact of a  product team’s effort.

3. The product team is missing the business context

It takes a lot to run a successful business. And even though some parts of a business are easy to scale and replicate, some parts are hardly scalable. For example, any marketplace product requires a dedicated team for every country the business operates in, or sometimes even for every city.

The product org, on the other hand, usually has a structure that is almost fully detached from the structure of business departments.

In this configuration, the product team may not have a single go-to business person who’d ask for a specific product feature to be developed; rather they deal with multiple stakeholders—consistent with the number of “local teams” in the org—who may not be synchronized with each other. While the product team attempts to solve customer problems with a one-size-fits-all product, this may not be the most efficient way, since the company’s business has different competitors and different maturity levels across the board.

Furthermore, the problems that the software must solve can vary dramatically depending on geography—even within a single country. The caveat that product teams sometimes encounter is that solving a market-specific problem has a limited impact, whereas a potential one-size-fits-all redesign is expected to deliver impact globally.

Now that we have covered the challenges that usually impede complex high-impact projects, here are some tips that could help you overcome those and speed up your product career.

I. Think big

Don’t take your team’s scope and setup for granted—explore large untapped problems, and don’t shy away from ambitious and complex projects that require high investment.

II. Empower your team

Even though your team might have limited resources and outreach, usually with the design and engineering help you still can conduct the “riskiest assumption test” with a proof of concept, which will empower you to further reason about the project’s priority and the need for external help.

III. Speak up and manage up

Remember that point about prioritization being unclear and managed by the bosses? Well, it is your job to draw enough attention to a problem and come up with all the hard evidence. In this journey, it is extremely important to adjust your language and the level of depth to the audience, so strong writing skills are essential. Check out “The Pyramid Principle” by Barbara Minto to learn a framework that will help you write more efficiently.

To sum up, a substantial part of the product manager’s job consists in overcoming organizational dependencies and limitations. Apart from perfecting our product sense and fluency with technologies, it is crucial to keep mastering communication and process management — because, in the end, it is what maximizes the real-world impact you can deliver as a product person.