The Metrics Trap: Are You Measuring the Wrong Things?

Boagworld: UX, Design Leadership, Marketing & Conversion Optimization

Content provided by Paul Boag and Marcus Lillington. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Paul Boag and Marcus Lillington or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.

2M ago 49:34

MP3•Episode home

This week, we’re tackling a biggie—how to avoid the pitfalls of metrics. We’ll explore why tracking the wrong numbers can send your entire organization in the wrong direction and what you should be focusing on instead.

We also have a fantastic Figma plugin to make your mockups more realistic, a thought-provoking read on AI’s impact on UX, and a listener question on handling the ever-growing number of software subscriptions. Oh, and of course, Marcus has a joke. But don’t get your hopes up.

App of the Week: Quick Fill

This week, I came across Quick Fill, a Figma plugin that helps you populate your designs with realistic-looking data. We’ve all been there—presenting a mockup to stakeholders, only to have them fixate on the fact that every name and profile picture is identical. Quick Fill solves that by auto-generating realistic text, names, addresses, and other useful content to make your designs feel more natural.

Not only does this improve stakeholder buy-in, but it also helps you spot design issues early on. Ever built a UI that looks perfect until you enter a name like “Marcus Brian John Lillington” and suddenly everything breaks? This plugin helps catch those issues before they reach development. Highly recommend giving it a go.

Topic of the Week: The Metrics Trap

Metrics can be an incredibly powerful tool—but only if you track the right things. Too often, businesses latch onto the easiest numbers to measure rather than the ones that truly reflect success. And that’s where things start to go wrong.

The Dangers of Poor Metrics

A classic mistake is tracking surface-level engagement over meaningful outcomes. For example, an insurance company might focus on the number of quotes they generate rather than the number of actual policies sold. I’ve worked with companies who obsess over lead generation, but when you dig deeper, half of those “leads” never convert into paying customers. If you optimize for the wrong metric, you optimize for the wrong behavior.

Then there’s the McNamara Fallacy, which essentially states that we measure what’s easy rather than what’s important. It’s easy to track the number of form submissions. It’s harder to track customer satisfaction, lifetime value, or the real impact of UX improvements. But those harder-to-measure metrics are the ones that truly matter.

Short-Term Thinking vs. Long-Term Success

Another issue is the obsession with short-term metrics. Quarterly targets and monthly reports drive short-term decision-making, often at the expense of long-term strategy. A UX redesign might cause a temporary dip in conversions as users adjust, but if you panic and roll everything back immediately, you’ll never see the long-term benefits.

It’s also important to balance different types of metrics. I always recommend tracking at least three key categories:

Conversion metrics (sales, sign-ups, or other key actions)
Usability metrics (error rates, time-on-task, Net Promoter Score)
Engagement metrics (return visits, time spent, interactions)

If you only track one type, you risk making short-sighted decisions that improve one number at the expense of everything else.

Making Metrics Work for You

So, what’s the solution? First, stop relying on a single “magic” number to measure success. No one metric will ever tell the full story. Instead, combine different data points to get a more holistic view.

Second, give things time before making decisions based on data. A website redesign, pricing change, or new feature will take time to bed in. I always tell clients to wait at least two weeks before reacting to data shifts—longer if it’s a major change.

Finally, use qualitative data to understand the “why” behind the numbers. Metrics can tell you what’s happening, but they won’t explain why users are struggling. That’s where usability testing, surveys, and direct feedback come in.

Reads of the Week

This week’s must-read articles focus on the evolving role of AI and its impact on UX.

Generative AI – The Power and the Glory – This article explores both the immense potential and the significant challenges that come with generative AI. It breaks down the current state of the technology and where it’s headed.
Enhancing the Experience of Using AI – A deep dive into how we can make AI tools more user-friendly, highlighting best practices for designing AI-driven experiences that feel intuitive and useful.

Both pieces offer valuable insights into the future of AI and UX. If you're keen to understand how AI is shaping our field, these are must-reads.

Listener Question of the Week

Sarah G. asks:

“In an age where literally every tool requires a subscription, how do companies of one decide what they really need—and how do they afford it all?”

Great question, Sarah. This is something I struggle with myself, and over time, I’ve developed a few key strategies:

Keep a spreadsheet – Track everything you’re subscribed to, including renewal dates and costs. It’s easy to forget about that random tool you signed up for six months ago.
Set calendar reminders – Before a renewal, ask yourself if you’re still using it. If not, cancel.
Look for multi-purpose tools – I love specialized apps, but sometimes, an all-in-one tool can save money. Notion, for example, replaces my project management software, CRM, and notes app.
Avoid annual subscriptions (unless you’re certain) – Monthly payments may seem pricier, but they give you flexibility. There’s always a better tool around the corner.
Only pay for what saves you time – Every tool should earn its keep. If it doesn’t save you significant time or money, it’s probably not worth it.

And of course, listening to this podcast keeps you up to date on the best tools so you don’t waste money testing duds!

Marcus’ Joke

To end on a high (or low, depending on your tolerance for dad jokes):

“I'm giving my chimney away for free. You could say it's on the house.”

Yep. That happened.

That’s it for this week! Thanks for tuning in, and as always, if you have questions or topics you’d like us to cover, drop us a message. See you next time!

Find The Latest Show Notes

562 episodes