Companion to Appendix F: Reading Statistical Claims in Defensible Analytics. Use this page when a dashboard, AI visual, or chat answer makes a claim about an average, a group difference, a relationship, or a driver.
This is not a formal-inference course. It does not teach hypothesis testing, regression fitting, p-value calculation, forecasting, or model selection. It helps you decide whether a claim is ready to communicate, needs more checking, or needs qualified method review before a consequential decision.
Start with the claim
Before accepting a number, name the boundaries that make it meaningful.
What is the unit?
A ticket, customer, encounter, employee, month, transaction, or something else?
What population and denominator?
Which records are included, and is the claim per ticket, customer, dollar, month, or eligible person?
What is the key limitation?
An unresolved denominator, filter dependency, small subgroup, untested confounder, or another condition that could change the conclusion?
What is the comparison?
Against plan, a prior period, another group, or an expected range?
What is the grain?
One total, a time series, a region, a product line, or a subgroup?
What decision state?
Should the claim be communicated, investigated, or sent for qualified method review?
Choose the next move
| Decision state | When it fits | Next move |
|---|---|---|
| Communicate a bounded description | Unit, population, denominator, comparison, and material limitation are clear. | State what the data shows and name the boundary. |
| Investigate before acting | An outlier, subgroup, alternate filter, or plausible confounder could change the story. | Check the distribution, a finer grain, or a focused comparison. |
| Seek qualified method review | A consequential decision depends on an interval, significance claim, ANOVA, or model output. | Review the method, assumptions, and decision consequence with a qualified analyst. |
Read, do not overclaim
Average
An average is a starting point. Check the distribution, outliers, and subgroups before treating it as the whole experience.
Weighted average
Weights change the business question. Ask what they represent and who chose them.
Group difference
Name the direction, baseline, group sizes, and practical consequence before using it in a recommendation.
Association
Association is not cause. Check a scatterplot, subgroup patterns, and plausible confounders.
AI-generated driver
Read a ranked driver as a hypothesis for human review, not a decision by itself.
Significance label
A threshold is not a business decision rule or a measure of effect size. Ask whether the decision changes under a less favorable estimate.
Practice workbook
Statistical Claim Workbook
Use the synthetic support-ticket and reimbursement examples to inspect routine formulas for descriptive summaries, weighted averages, group differences, and correlation. Model and ANOVA sheets provide labels to read, not calculations to reproduce.
Open the workbook in Excel, Excel for the web, or Google Sheets. The formulas are visible so you can inspect the claim, then return here to choose the next check.
Worked claim check
A support dashboard says: “Tier 2 resolution time rose 34%, so staffing is the cause.”
Show worked answer
- Unit: mean resolution time per Tier 2 ticket.
- Population and denominator: unresolved until the dashboard names which Tier 2 tickets and whether reopened or unresolved tickets are included.
- Comparison: 34% higher than an unstated baseline period; name the baseline before quoting the change.
- Grain: Tier 2 aggregate. Check month, incident severity, product area, and ticket mix at the next-finer grain.
- Key limitation: staffing is only one plausible explanation. Releases, migrations, incident-heavy months, and ticket complexity could confound the claim.
- Decision state: investigate before acting. A bounded rewrite is: “Mean Tier 2 resolution time was 34% higher than [baseline] for [defined population]; investigate ticket mix, incident severity, releases, and staffing before attributing cause.”
Go deeper
When a decision requires formal study beyond Appendix F, use an authoritative resource rather than extending a dashboard calculation beyond its evidence.
| Resource | Printed URL | Purpose |
|---|---|---|
| OpenIntro Statistics | https://www.openintro.org/book/os/ | Free textbook, data, and exercises for formal study. |
| Penn State STAT 200 | https://online.stat.psu.edu/stat200/ | Open notes for statistical methods beyond this companion’s scope. |
| NIST/SEMATECH e-Handbook | https://www.itl.nist.gov/div898/handbook/ | Public reference for exploration, measurement, modeling, monitoring, and comparison. |
| Practical Statistics for Data Scientists | https://www.oreilly.com/library/view/practical-statistics-for/9781492072935/ | Applied bridge for analysts who need more depth on sampling, significance, regression, and misuse. |
Related book resources
- Chart Gallery: choose a chart once you know the question and comparison.
- Chapter Resources: living reading and activity pages for the book’s chapters.
- Online Resources: the wider living index of Microsoft, practitioner, and data-visualization sources.
- Feedback: corrections and clarifications for the print book and companion.