Data to Action: Product Analytics

“Product Analytics - how is it different from data analytics for your product’s user?” someone asked me a couple of weeks ago. It got me thinking why the distinction wasn’t obvious. That’s because one is a subset of the other: while data analytics provides “insights for actions,” product analytics “drives action.”

Whether you’re an enterprise or a “starting up” start up, we all need visibility into how a user is using our product, i.e. we need visibility into user behaviour to understand what’s working vs what’s not.

“How much time does an average user spend on my product?”

“Which item is sold the most?”

“Which engagement feature is the most popular?”

“When does a user exist my app?”

While it may seem tempting to find answers to these questions yourself, to answer them effectively, you need to understand how product analytics works, the instrumentation behind it, and why it’s crucial for product success. Whether you’re building something new or improving an existing part of your product, you need insights into user interactions to benchmark your growth and measure the impact of your efforts. Product analytics helps you explore (or expose 😛) this reality, which in turn can help you improve the main product itself.

The Instrumentation Behind

To get a quantitative understanding of what users are doing with your product, the first step is to build the instrumentation for product analytics, i.e., to “record” or “track” user activity.

Tracking Events

For this article, let’s use an example of an e-commerce/online shopping product.

Each time a user performs action X, you fire an event X. Essentially, you name every click/activity/action of a user and log it as an event. For instance, every time a user opens the home screen, log an event homescreen_opened with attributes like user_id, timestamp, session_number, and event_name. You can track events for actions such as cart_opened, item_added, discount_banner_clicked, checkout_initiated etc. Sounds cool and fancy that we can track each user action, we can choose to collect these events & process them in batches, or stream. Without getting into more technical details of “how to send the events”, it’s important to understand “where to send these events” to and what’s next.

Data Collection, Structuring and Aggregation

Typically, events are sent to a Data Warehouse for further processing. Raw events themselves aren’t very useful; they need to be structured into meaningful tables by aggregating and cleaning the data. Transforming / structuring data just means writing some SQL that takes these events and filters and aggregates them. Yes, you could do it as a DML statement (writing SQL queries) but doing this repeatedly, is a nightmare - especially if you’re using a managed data warehouse. Instead, to avoid future data debt, you could set up a pipeline to transform, aggregate raw event data right away, before dumping into your warehouse. Either way, you will get the same result - useable aggregate tables ready for analytics!

OG Product Analytics

Once you have the data ready - it’s yours for analytics. You might want to start by dashboarding to understand what’s the current state of your north star metrics, how is user engagement changing day on day etc. To understand your users closely, you’d want to start segmenting your users, building cohorts, understanding user activity basis these cohorts, etc. To understand the drive behind any user action, it’s useful to build a Funnel for every “user path”, this will also help you understand how power users are created, how conversions happen, what step leads to the next etc.

Further to introduce something new or improve an existing part of your product, you’d want to do some A/B testing - for which you might want to do some analyses yourself. Because even hypotheses need to be data backed :)

For this piece let’s breakdown the process for 1 stream of product analytics -

A/B testing to introduce and scale a new feature

1. Define your North Star metrics (e.g., daily_checkouts) and set a goal (e.g., increase daily_checkouts by 10%).
2. The core PM part - now define a clear hypothesis for this growth to happen - “By introducing a 5% discount banner on the cart, daily checkouts should increase by 10%”
3. The product analytics instrumentation - building the most viable implementation of this change & loading it with event analytics, i.e. each time a user clicks the discount banner, you fire discount_banner_clicked event, each time user hits checkout, you fire checkout_initiated event.
4. The engineering part - Getting your engineers to deploy the changes to a subset of the users in an A/B experiment
5. Let the experiment start!!
6. The product analytics part 2 - Breakdown the results on the basis of the data collected. And decide whether or not this experiment was successful.

How it drives Product Success

See, this wasn’t just data analytics, was it? I think with the above example, it’s clear why product analytics drives action. So let’s summarise, with product analytics, we can:

• Understand user engagement with the product, and how that impacts the key metrics,
• See in real time how users are interacting with a certain feature,
• Build hypothesis for user retention (& eventually product success) and test these via A/B testing, in form of experiments on users.
• Measure the success of features introduced via A/B
• Do all kinds of Product Analyses - building & understanding cohorts, funnels, understanding the reason for churns, and doing RCAs on major issues
• And most importantly, determine the financial impact of your product decisions - will this make me any $$

And that’s a wrap! If you don’t live and breathe data, it just becomes another tool. Product Analytics allows you to live and breathe data to improve your product (insights + action)!!