How to Analyze KPIs: Diagnose Changes and Run Actionable Reviews
Published:
Last Updated:
Category: Marketing Glossary
Authors: Shusaku Yosa
Analyze a KPI by checking its definition and data quality, separating counts, denominators, and population mix, then deciding what to do. A gap from target does not identify a cause. Changing an activity, collecting more evidence, and deferring judgment can all be valid conclusions.
Establish whether the values are comparable
Keep the indicator, target, and actual result in separate fields. If the definition is unclear, start with department and role KPI definitions. Specify success and eligibility before comparing rates based on clicks, sessions, or people.
Symptom | First check | Possible explanation | Before deciding |
|---|---|---|---|
Sudden zero | Collection, success rule, production host | Missing measurement or real decline | Do not label missing data zero outcomes |
Same-day decline | Update time, processing, weekday | Delay, small sample, real change | Recalculate comparable complete periods |
Only overall rate changes | Underlying counts and segments | Traffic or customer mix shift | Compare within-group rates and shares |
Outcome count jumps | Duplicates, cancellations, eligibility | Tracking change, quality loss, demand | Compare qualified outcomes consistently |
Align period length, time zone, production hostname, page or customer scope, cost scope, exclusions, and deduplication. Mark an unavailable indicator unmeasured rather than converting its blank cells to zero. Log changes in tracking and consent conditions.
GA4 data-freshness guidance describes processing that can take 24–48 hours in relevant circumstances. It does not guarantee that every report becomes final simultaneously. Check update status rather than assuming a realtime view and a standard report represent equivalent data.
Recalculate rates from their original counts
The following fictional periods have equal lengths, identical outcome definitions, and unchanged measurement. X and Y are mutually exclusive session groups. The numerator counts sessions with an outcome, not every outcome event.
Period and group | Sessions | Sessions with outcome | Rate |
|---|---|---|---|
A: X | 800 | 80 | 10% |
A: Y | 200 | 10 | 5% |
A: total | 1,000 | 90 | 9% |
B: X | 200 | 20 | 10% |
B: Y | 800 | 40 | 5% |
B: total | 1,000 | 60 | 6% |
X remains at 10% and Y at 5%, but Y’s larger share lowers the overall rate from 90/1,000 = 9% to 60/1,000 = 6%. That is a decline of 3 percentage points, or approximately 33.3% relative to the earlier rate. The simple average of 10% and 5%, 7.5%, is not the overall rate.
Successful sessions also fall from 90 to 60. This table alone does not establish a failed website change or weaker individual performance. Investigate why volume, destinations, customer segments, and outcome value changed. Equally, unchanged within-group rates do not automatically make the new mix desirable.
Separate observations from causal hypotheses
“The rate declined” is an observation; “the new form caused the decline” is a hypothesis. Consider measurement changes, population mix, seasonality, external conditions, and interventions. If a price change coincided with an advertising change, a simple before-and-after comparison cannot isolate one effect.
Google’s discussion of causal inference from observational data explains the need for assumptions and why good model fit does not establish correct causal effects. Consider a suitable comparison or experiment when needed. Preserve unresolved uncertainty instead of presenting a likely explanation as proven.
Record a decision, owner, and next test
A definition sheet, calculation sheet, and decision log are enough to begin; a dedicated dashboard is not a prerequisite. Put decision-relevant indicators in the overview and counts, segments, and quality checks in the diagnostic detail. Choose the number of metrics by the decisions required.
Fact and conditions | Hypothesis or gap | Current decision | Owner and next condition |
|---|---|---|---|
Fictional: form rate falls the day after a change; current data incomplete | Success-rule change and processing delay unchecked | Defer intervention assessment; inspect tracking | Measurement owner checks duplicates and rules, then recalculates a complete period |
Fictional: group rates unchanged; overall rate 9% to 6% | Mix changed; qualified outcome value unknown | Investigate allocation and quality | Acquisition owner explains mix and economics before continuing or changing |
The U.S. Federal Performance Framework provides an example of connecting accountable owners with regular data reviews. Its review cycle is not a universal business obligation. Choose a cadence that suits sales, usage, and data-processing cycles.
Define alert and follow-up conditions
- Specify the unfavorable direction, period, minimum denominator, update status, and persistence required for investigation.
- Name an owner and backup, the first checks, and an escalation route.
- Treat an alert as a prompt to investigate, not an automatic instruction to change budgets or operations.
- Record success, observation duration, and conditions for continuing, modifying, or stopping an intervention.
Verify any notification capability in the actual tool and configuration. With small counts, examine absolute results, cost, qualification, and quality alongside rates. Use the ROI and cost-effectiveness evaluation guide when the decision involves investment. The next review should establish what was learned and whether that evidence changes the decision.
