The Digital Leader Newsletter — Strategies and Techniques for Change Agents, Strategists, and Innovators.

We are going to relentlessly chase perfection, knowing full well we will not catch it, because nothing is perfect. But we are going to relentlessly chase it because in the process we will catch excellence. I am not remotely interested in just being good. —Vince Lombardi, Green Bay Packer Hall of Fame coach (the Super Bowl Trophy is called the Vince Lombardi Trophy)

Of all the incredible leadership, strategy, and operating skills I enhanced at Amazon, the one that stands is the topic of metrics — how we measure things. We all think we are good at metrics and analytics. And we all think that we are good operators. But…do we understand how to “chase perfection”? Metrics are the fuel to feed that fire.

I get invited to participate and present to lots of executive team meetings. These meetings cover broad territory and topics, and I always try to emphasize a few key points — the importance of patience, and that transformation is as much about your personal habits and beliefs as it is about changing your organization. I stress that Amazon’s playbook is a system based on principles. It is not based on a set of steps or a single linear approach.

Inevitably, someone will raise their hand and ask me to pick the single most important play in Amazon’s playbook. I understand the need to give a starting point. So while explaining that “it’s a system,” I do answer the question. “What’s the one thing to get me started?” The critical fundamental, the “championship habit” above all else is this — metrics.

IN GOD WE TRUST. ALL OTHERS MUST BRING DATA

In 2004, I attended a senior executive team meeting at Amazon that just so happened to coincide with the Salesforce initial public offering (IPO). During the meeting, one of the other executive team members casually commented that Salesforce was the world’s largest customer relationship management (CRM) technology company.

Big mistake.

“We are the world’s largest CRM company!” THE senior Amazon leader (guess who) yelled in response.

Like a CRM company, Amazon is obsessed with managing and analyzing the data on customer interactions to improve its relationships with them. The digital nature of Amazon’s business and its focus on collecting copious amounts of data make it significantly different than the average e-commerce company. Metrics are seamlessly woven into the culture of Amazon.

W. Edwards Deming’s famous quote is a mantra: “In God we trust. All others must bring data.” Teams spend as much or more time defining and agreeing on how to measure a new feature, service, or product as they do design the feature itself. Weeks are built into project management schedules for the consideration of an operation’s inputs and outputs; to identify what data might be needed to run that operation, and to understand its complex internal workings. Metrics are the answer. Metrics are the question. Metrics are the vehicle to relentlessly pursue root causes, customer adoptions and satisfaction, and operational inefficiencies. Metrics are the way.

DID MY CUSTOMERS HAVE A GOOD DAY TODAY?

At Amazon, having a balanced, well-engineered scorecard of metrics that are consistently reviewed day over day, week over week, provides deep insights into what works and what doesn’t. It also places sole responsibility for success and failure on you as a leader.

Repeatable, consistent performance reflected in metrics is the gold standard for success. Without access to a consistent set of metrics, an Amazonleader would be flying blind, and such risky behavior is not acceptable at the company. Real-time metrics are the lifeblood flowing through the veins of Amazon. Real data and real insights from the customer experience are used continually to answer the question, “Did my customers have a good day today?” If your metrics are in place, if they are in real-time, and your team members and processes use them, this question yields a simple “yes” or “no” answer.

It takes foresight to lead by the numbers correctly. You must embed real-time metrics from the very start because they are nearly impossible to retrofit. The Amazon experience shows us that the single biggest opportunity for companies operating today is to completely rethink their concept of metrics. Most companies use what’s called “batch architecture” to record large sets of transactions or other quantitative updates and to process them periodically (daily or weekly is typical). Batch architecture is very last century. Today, you need real-time data, real-time monitoring, and real-time alarms when trouble is brewing—not lag-time metrics that hide the real issues for 24 hours or longer. Your business should operate like a nuclear reactor. If a problem arises, you need to be aware of it immediately. You need to predict it. You need to act on it.

The single biggest opportunity for companies operating today is to completely rethink their concept of metrics

SPEND MORE TIME DESIGNING YOUR METRICS

Think about where time is allocated at the executive and management levels—lots of time spent on budgeting, financial reviews, HR, compliance, and so on. These are important. But do they really help serve customers, drive operational excellence, or spark ideas for innovation? Here are three quick steps for designing your metrics:

1. Spend time talking about what your metrics should be.

2. Have meetings that use those metrics to constantly review your business. Always look for root causes. Turn metrics into action.

3. Get as granular and real-time data as possible.

Make “Metrics” a Verb

Having great metrics is ammunition, but the wars are fought in the daily combat of using metrics to drive to perfection. The theater for these battles is meetings we called “metrics meetings.” The entire operational rhythm of life at Amazon is a set of weekly metrics meetings. Typically starting with the lowest-level services and capabilities and building up to the worldwide senior meetings at the end of the week, an operational leader at Amazon attends a series of metrics meetings over the course of a week. Every week. Like clockwork.

Although they are called “metrics meetings,” they are really meetings of shared accountability where the “owners” of related metrics discuss the most recent trends and issues in their metrics. Leaders are expected to know the details of their business, to be vocally self-critical, and to discuss plans or progress for improvement. Although successes might be recognized, the general modus operandi is to “celebrate for a nanosecond” and then focus on the issues and errors in the business. Even when things are generally good, they’re never perfect.

Meetings are organized around key services, processes, customers, and groups of capabilities. For example, I ran the Marketplace metrics meetings, even though from an organizational standpoint, only a subset was part of my organization. I would include leaders from key capabilities affecting the Marketplace business, such as the catalog team, order pipeline team, and payment and fraud team.

The finance partners at Amazon play a key role in making sure all metrics, not just financial metrics, have been prepared, and they partner with the functional leader to take notes, assign responsibilities, and drive results and improvements to be delivered. The finance partners are critical in keeping the meetings “honest” and ensuring that people deliver results. I think of them as the independent accountability officers ensuring that the metrics meetings are doing their job of defining and reporting on progress to improve our performance.

INSTRUMENTATION: REAL-TIME, FINE-GRAINED DATA

Once a team’s metrics and SLAs are in place, the focus turns to data collection that will inform those metrics and SLAs. At Amazon, there are very specific standards for the quality and type of data a team should collect. Amazon’s executive team refers to those standards as “instrumentation.”

Leave a comment

Expectations on data collection—or instrumentation—at Amazon are twofold: First, data should be fine-grained in nature. You can always summarize and aggregate data, but you can’t go back and derive more detail from a data set. Second, that data should be available in real-time. You can always batch data or slow it down, but you can’t speed it up. Design for no time lags and no batch systems.

There are many reasons this is important. Let’s say a grocery company uses a refrigerated storage bin to keep its fruits and vegetables fresh. Suddenly, over the course of a day, all the lettuce goes bad. If the company has been collecting fine-grained data—for example, any changes in temperature or pressure and their timestamps—that grocery company might actually be able to figure out what it was that caused the lettuce to go bad. Otherwise, they’re stuck wondering about the possible variables and causes in that situation.

Of course, it’s not possible to collect fine-grained real-time data in every situation. There are limitations to the nature of the data that you’re able to collect in a specific situation, but at Amazon, it’s expected that you will work vigorously to achieve instrumentation.

Finally, never stop reevaluating and building metrics. Frequently check to make sure that your metrics are still explicitly linked to goals, and don’t hesitate to change or add metrics when they stop driving change in your organization.

Why?

Because good metrics create good processes that minimize bureaucracy.

Designing Metrics and Metrics Meetings—the Amazon Way

Your Homework

There’s homework to do this week. Excited?! Ask yourself (and your teams), these “simple” questions:

1. Can your core teams answer the question “Did my customers have a good day today?” with metrics and monitoring?

2. Do you have robust customer experience metrics to complement your operational and financial metrics?

3. Could you use metrics to drive better accountability and quality?

Let’s go relentlessly chase perfection.

Onward!

John

—Subscribe to The Digital Leader Newsletter for Free!

Subscribe now

About The Digital Leader Newsletter

This is a newsletter for change agents, strategists, and innovators. The Digital Leader Newsletter is a weekly coaching session focusing on customer-centricity, innovation, and strategy. We deliver practical theory, examples, tools, and techniques to help you build better strategies, better plans, better solutions — but most of all, to think and communicate better.

Share The Digital Leader Newsletter -- By John Rossman

Leave a comment

The Digital Leader Newsletter -- By John Rossman

Chasing Perfection -- Use Metrics to Build a…