FOR NEXT WAVE DESIGN LEADERS

Some hard-earned lessons to get you measuring

​

In Today’s Issue 💬

→ Practical tips to get started with measuring

→ Hot news across the industry

→ All new courses and dates announced

​

Putting metrics in place is hard. Some hard-earned lessons to get you measuring.

Hey designers, while you’re consumed with finding the right metric, you should know a few pragmatic realities of putting a metric in place. Here’s what I’ve learned over the years.

You should know the difference between measures and metrics.

Measures are numbers that give life to metrics. An Error Rate might be 10%, but did you know it has three measures?

1. # of total possible errors
2. # number of participants
3. # errors for each participant

You can’t know the error rate until you have these these measures first.

You have to baseline your measures first before you can derive a metric.

Why? Because you want to feel confident that the measure is relevant and real, not that you just got lucky. You want your measures to have business relevance AND statistical significance.

Business relevance is all about measures that connect to a business outcome. Lots of the article you read about the right metrics are only talking about business relevance. This is only half the equation. Relevant measures that aren’t real is guessing and guessing is risky!

For statistical significance, you need to think about two things:

1. The size of the sample of people
2. The variation in the people you sample with

It's #2 is where a lot of teams go sideways. When teams only sample from every type of customer, larger sampling errors occur.

Once you baseline, you have to choose a new target to achieve.

Your target is really the outcome your experiment is trying to achieve or relate to. Think about it as an if/then statement. “If we reduce errors, then expect to reach our target.”

All of the above does not happen over night.

This is perhaps the biggest reality to overcome. Getting a metric in place that you can rely on will take more than a 3-week sprint. You will have to make difficult trade-off decisions to just get a baseline measure in place.

Getting a metric in place is more than a business problem to solve.

It’s also solving people and process problems. You will have to find new ways to get measures in place while your colleagues or stakeholders are worried about slowing down or missing shipping dates.

It’s these additional factors that we, as an industry, don’t talk about a lot and it’s a disservice to us all. The more we argue about what the right metrics are, the more we delay having actual measures and metrics in place. We can only learn from them once they’re there.

TLDR: You can’t choose where to go without knowing where you are. Knowing where you are takes time and energy. There are no right metrics. There is only learning from imperfect measures and metrics to know what are more right.

IME, the benefits of doing all this work is you will have MUCH more confidence in your decisions after you have imperfect measures and metrics in place. You just have to let go that the right metric is there before you start.

​

🔥 In Other News

• ​Phoebe Gavin, journalist turned career coach, calls out companies who ask women of color to work for free
• ​Tanya Snook, founder & co-chair CanUX Conference, shares UX Design's dirty secret
• ​Anne Helen Petersen, author of CULTURE STUDY, shares an interview about how to build better bosses
• ​AIGA, shares how writing and design are more alike than you think

​

​

Drop on by

We just wrapped up our fall cohorts of Strategic Business Thinking. We had leaders join us from 5 time zones – all digging in and doing hard, vulnerable work.

The vast majority of our alumni come to us as referrals. We think it's because they feel a little less stuck and they want to help those they respect feel that way too. Focusing on where decisions, personalities, policies, and culture intertwine is hard as hell, but it's where we like to play.

We bring our values of vulnerability, honesty, and transparency to all of our courses. You have to show up, ready to share and grow. If those words scare and excite you, Second Wave Design is for you. If they just scare you, well, you can go elsewhere. No hard feelings.

We just posted all new courses and new dates on the website. If you want in, go sign up or join the waitlist. You'll be the first to know when registration opens next month! We hope to see you there.

I'm proud of you. Be good to yourself.
Ryan