Recently I went to the grand opening of Polygon Atlanta, a new events space over at Atlantic Station. I say “events space” as Polygon is explicitly not (yet another) co-working space.

What better way to kick off a space for “Web, Tech & Creative Events” than with an intro by Jared Spool?

So Jared asks the question, “Is Design Metrically Opposed?”

Nope, it isn’t.

Well that was easy. Ok, yes there’s more to it…

Is Design Metrically Opposed?

To be honest I’m not 100% certain what Jared’s answer would be. I think I know but if I had one critique about his presentation it’s that the build up was about 98% of the presentation and the takeaway was tucked in at the end, easy to miss.

My interpretation, then, is that design and metrics are not diametrically opposed at all. Rather, design is enhanced by metrics, but only the right metrics. And it’s a slippery slope, because the wrong metrics can drive you to a poorer experience, the antithesis of good design.

Take Instagram & Twitter for example. Once upon a time you saw Instagram photos nicely embedded in Twitter. Kind of like this:

If a picture is worth a thousand words, then Instagram users went way over their 140 character limit. And that was good for everyone.

But then Instagram did something odd.

It started preventing embedded photos. So now you see something like this:

What gives? This is…less good.

What gives is MAUs, which stands for Monthly Active Users. This is an important metric for Instagram. Really important. It drives their valuation and is a key success measure, among other things. Which is ok on the surface because an active user of your product is a reasonable proxy for something good happening.

But what happens when you dig below the surface?

When you could embed your Insta images directly in the Twitter feed, your friends could see your pics without directly using Instagram. And by not directly interacting with the Instagram app, you don’t get counted as a MAU. Bad for Instagram.

So now, to see the pictures you have to actually go to Instagram. Good for Instagram…well in theory. Because now you’ve created a more frustrating experience in the name of numbers. Your MAUs have gone up, but your customer delight has gone down.

Measuring (The Right) Things by Designing Your Metrics

Here’s Jared’s point: When you focus purely on the numbers, the overall value and experience get lost. You can’t just go out and start measuring numbers for numbers’ sake.

You’ve got to plan out your approach to the metrics you’re going to capture. You need to design your metrics.

“Wait, you can design metrics?!?”

Yes. And you must! The same way you design your user experience and design the UI of your app, you should also design your analytical approach. How to design the right metrics to measure is an entirely separate series of posts so I won’t go into the details now. But in short, avoid vanity metrics; focus on actionable metrics.

Quantitative vs Qualitative

The other piece that Jared brought us is the divide between qualitative and quantitative metrics.

The problem, as Jared describes, is that there shouldn’t even be a divide. The two are incredibly complementary.

Quantitative Metrics

Let’s take a poster child metric like Time on Page. What are some of the problems with Time on Page?

• Don’t take numbers as gospel – Let’s say you have high Time on Page. Is that because users are truly viewing your page for a long time, or did they just forget to close their browser tab?
• Each number by itself is just a small piece of the puzzle – In addition to Time on Page, what other metrics would help bolster your case.
• Taken out of context a number is worthless – Simply stating the Time on Page leaves me wondering whether a high or low value is good, or how far away from the target we are.
• Design your metrics with the end goal in mind – If you want to increase revenue, do you know there is a correlation between Time on Page and $$? If not, what other metrics are more closely correlated with your goal?

In the absence of perfect information metrics can be a guiding light. But they’re only that, a guide. They’re not the truth. They can be skewed. They can be misused. Or misinterpreted. Or both.

The purpose of quantitative analytics is to give you one more tool in your tool belt, nothing more.

Qualitative Metrics

Remember, qualitative and quantitative analytics should be complementary. Once you have your numbers, the qualitative results start to give you meaning.

Take a multiple choice question with 4 options. Option B wins with 75% of the picks. Pretty conclusive right?

You see where this is going.

NO, that’s not conclusive at all. All you know for sure is that B was picked the most. What you don’t know is WHY B was picked the most. And here’s the deal – that’s the part that matters the most.

Maybe everyone picked B, not because it was their favorite, but because they hated all the options and it was the least terrible of the options. Maybe there was a browser bug that obfuscated the answers to C and D (I’m looking at you IE 7). Or maybe option B really was everyone’s favorite.

When you over-emphasize quantitative analysis this leads to the value of qualitative feedback being lost. And when you do that you create an environment where metrics and design become contradicting forces.

The solution isn’t to stop measuring in the name of design, but rather to start designing how you measure.