Overview

User metrics give us clear insight into the successes and failures of our product(s). They provide a means for us to hold ourselves accountable to our objectives. If you’ve never worked with user event tracking tools before then this article is a great place to get you started. In the past, I have rolled out event tracking tools at 2 different companies and trialed 4 different products. They are wonderful to have, but take time and dedication to setup properly.

Why are they important?

What you don’t measure, you can’t improve.

Validate product decisions with data, not your gut.
Gain insight into user behavior. These insights in turn drive specific in-app and marketing improvements.
Better prioritize feature improvements and bug fixes. Tracking user behavior can indicate where problem areas are.

The Difference between Usability and Engagement

Usability metrics focus on how easily people can accomplish what they’ve set out to do.

Engagement metrics focus on how much time people spend interacting with your product (best measured in a flow state) or how good they feel about it (qualitative).

Metrics to Consider

Number of Sessions per User (per day/per week/per month)
1. Measures: How often users log in (frequency)
  1. Tip: Avoid averages and take medians over means (less sensitive to outliers)
  2. Why? Once a target has been established we can use the median to identify customers that may be likely to churn, need assistance, or are having issues (proactive versus reactive).
Session Duration
1. Measures: How much time users spent interacting with the product or in a particular flow state
  1. Tip: Use medians instead of means (less sensitive to outliers)
  2. Why? Identify at risk customers and test feature usability if measuring a flow state. Establish a benchmark and analyze usage over time to identify trends.
Number of Key User Interactions per Session
1. Select user interactions that matter the most (e.g. clicks on “Publish”)
  1. Why? Validate assumptions, test “important” features and identify those that could be eliminated or simplified
  2. Tip: Look at difference cohorts of users (by user role, permission set, etc…)
  3. Idea: Compare the differences in the metric between churned and retained customers within 30 days
Feature Usage
1. Measures: How frequently a feature is used compared to assumptions or other notable features
  1. Tip: Be specific in the expected flow or path a user is expected to take. Identify bottlenecks, bounce rates, and alternative paths
  2. Why? Use to refine/improve usability or remove features
Task Success Rate
1. Continuation of Feature Usage
2. Measure: Time on task vs. expected
  1. Tip: Look at back-button usage and error rate (instead of bounce rate)
  2. Why? Use to refine/improve usability or remove features

Next Steps

Identify the top 3-5 experiences in your application (what is it you expect a user to accomplish?)
1. Breakdown by key interactions
Evaluate tracking tools (see my next article for a full run down of the tools I’ve personally evaluated)
Implement tracking tool
1. Create a naming convention for each event - very important!
2. I like to use the following - Page: Feature: Action
Set usage metric goals for all new features being rolled out
Measure
Refine