How to Live in the Data as a Product Manager

The importance of defining, analyzing, and reviewing metrics as a product manager

Photo by Luke Chesser on Unsplash

Product management is both a strategic and tactical discipline, where PMs need to craft a vision, build a roadmap, execute day-to-day, and communicate widely. While intuition and experience play important roles in being a successful PM, data also plays a significant role in navigating ambiguity, iterating on the product, and growing the business.

PMs need to live in the data. They need to live and breathe their business metrics, their product KPIs, and their customer insights. They need to have metrics available at their fingertips, and be continuously analyzing their data for new insights. Finally, PMs need to use their metrics to make informed, data-driven, and objective decisions with their teams.

Through this post, I’ll walk through why data matters, provide an overview of common PM metrics, and recommend best practices for PMs to analyze and review their data regularly.

Why Data Matters

Data is at the core of every great customer experience because businesses use data to understand their customer needs and wants. Examples of this data could be market research to build the first release of a new product, beta user feedback to iterate before a public launch, or customer usage trends to identify conversion optimization opportunities.

At the core of every data-driven business should be a team of customer-centric product managers, who live and breathe that data. These PMs should treat metrics similar to how they treat requirements — fundamental elements to building any product. Ultimately, the success and performance of the product will be tied to these metrics, so PMs should spend time living in the data to define and understand their key metrics.

To start, PMs should take ownership of their data and hold themselves accountable to their business metrics. They should use data to get a daily pulse on their product space, and work with their team to course-correct when trends are moving in the wrong direction. They should use data to define a problem, and use success metrics to measure proposed solutions. And finally, PMs should use data to enable objective decision-making. Over time, living in the data will build trust with their team, stakeholders, and leaders.

Defining Your Metrics

PM metrics can be broken down into three categories: Business Metrics, Customer Usage Metrics, and Customer Satisfaction Metrics. However, it’s important to prioritize the metrics that matter most to your product. You want to collect analytics for as much as you can, but you should only measure success against what drives your product or business growth.

1. Business Metrics

At minimum, you should be able to clearly answer the following three questions when describing your product space:

• What is your company’s North Star metric?
• What is your product’s North Star metric? How does your product’s success fit into the company’s North Star?
• What are 3 KPIs (i.e. output metrics) that are crucial to the success of your product?

As a bonus, you should also understand the following:

• Revenue — How does your product contribute to the company’s revenue? How about the company’s profitability?
• Costs — What are the fixed costs associated with your product? What are the variable costs? What are your customer acquisition costs (CAC)?
• Efficiency — What are the operational defect metrics for your product? Where are the bottlenecks in your partner teams’ processes?

2. Customer Usage Metrics

After developing a fundamental understanding for the business impact of your product, you should start deeply learning about your customers. Understanding customer behavior and usage metrics are crucial to being a customer-centric, subject-matter expert for your product. Use some of the below metrics as starting points for measuring customer usage of your product:

• Adoption— How many visitors/downloads/signups do you have? Where are they coming from? What are early actions that correlate with higher retention? What types of customers does your product attract?
• Engagement & Retention—What are your daily/weekly/monthly active users? What is their frequency of use and average session time? Who are your repeat customers? What are your customer churn rates?
• Growth — What does your customer funnel look like? What are the conversion rates and bounce rates at each stage of the funnel? Where are customers spending the most time in the funnel (i.e. where are the friction points)?

3. Customer Satisfaction Metrics

Finally, as you launch your first product, add new features, or expand into new domains, you’ll want to understand how customers feel about their experience using your product. Use the below metrics to build an understanding of your target customer profile and the parts of your product that resonate most with customers:

• CSAT — How do customers rate their experience using your product? Are they satisfied? How have your satisfaction scores changed over time?
• NPS — How many promoters do you have? How many detractors? How many customers are likely to recommend your product to a friend?
• Online Reviews — How many online reviews have been posted about your business? Are there ratings that directly mention your product? How many of them are positive vs. negative?
• Customer Service Contacts — How often are customers calling your customer service team? How many support tickets are filed against your product?

Analyzing Your Metrics

Once you’ve defined the metrics that matter most to your product, you should start setting them up in analytics tools, so that you can track, analyze, and visualize your data. Note that analyzing data is a team effort, so work with your engineering team, analytics team, and data science team to build the following into your suite of product tools:

• Metrics Dashboards —Identify the business forecast targets for each of the metrics you defined above. Work with your engineering team to embed segment events for each event in your customer journey. Partner with your customer experience organization to determine where you can track satisfaction metrics. Once you’re up to speed, set up a metrics dashboard. The goal of this dashboard is for you to be able to get a pulse on the entirety of your product space, including an overview of your KPIs, a visualization of your customer journey, and a deeper layer for you to analyze any of your KPIs over a period of time.
• System Alarms — Work with your engineering team to set up alerts for metrics in your dashboard that have risk of spikes. Typically, these system alarms are intended to track negative spikes (e.g. error rates), so that you can be only notified when a metric requires your immediate attention. Set up these alarms to notify you and your engineering team in a Slack channel or whatever ticketing tool your team uses. Your customers will thank you for your ability to quickly resolve these negative spikes.
• Proactive Insights — While system alarms are for tracking negative events, proactive insights can help you identify positive opportunites for growth. Work with your user research and design team to build a library of customer behaviors, target personas, and usage insights. Take advantage of tools like Amplitude, Heap, and FullStory, which can help automatically categorize and identify these proactive insights for you. You’ll be able to identify conversion optimization patterns, determine where to iterate with A/B experiments, and learn which features impact customers most.

Reviewing Your Metrics

Finally, just defining and analyzing your metrics is not enough — you should be reviewing them regularly with your team, your stakeholders, and your senior leaders, so that you can share progress, identify new opportunities, and get support to pivot when necessary. Use some of the below mechanisms to review your metrics regularly:

• Morning Pulse — Every morning, review your metrics dashboard to understand the pulse of your business, track your product’s performance, and identify any new trends. As you get into the habit of doing this regularly, you will feel more in control of your metrics and be able to make better product decisions day-to-day.
• Daily Standups — Share key highlights from your morning pulse in your daily standups, so that you can keep your team of engineers, designers, and data scientists informed about your product performance. This will help your team feel more connected to your product, and ultimately enable more collaborative decision-making within your team.
• Weekly Business Reviews — At Amazon, teams typically run Weekly Business Reviews (WBRs) with their stakeholders and senior leaders as a mechanism to review metrics, track goal progress, and gain consensus on any business decisions. Assuming you are regularly conducting your morning pulse, you should have little additional work to prepare for these WBRs. Instead, focus on collating any product announcements, identifying callouts on hotly trending metrics (positive or negative), and proposing any changes to your roadmap that would help your product performance.
• Asynchronous Communication — As we move to a more distributed and remote workforce, asynchronous mechanisms can be powerful in providing updates on your product performance. Try sending biweekly newsletters to your stakeholders, providing automated updates in Slack, or contributing content to a company all hands presentation as a means to more broadly communicate your product’s metrics. While these do take time, they are often high reward mechanisms that celebrate the successes of your product and promote product awareness across the company.

Ultimately, as you start to live more in the data by defining, analyzing, and reviewing your metrics regularly, you’ll position yourself as a stronger product owner, more well-informed decision-maker, and better customer advocate as a result. Good luck!

My name is Zakir Tyebjee, and I publish monthly posts on the realities of product management. Comment below or reach out on LinkedIn if you have new topic suggestions — I can’t promise you I will make you the best PM, but I can promise you I will bring a realistic perspective that inspires you to be a better one 🙌