Chapter 9 — Practice-on-your-own solutions

The three failure modes (Chapter 9 recap). The three reconciliation failures are (a) date-filter drift, (b) measure-definition drift, and (c) source-table drift. Any of the three can produce a «close but not exactly» delta; the trick is diagnosing which without spending an hour.

Most likely: measure-definition drift. A 0.5% gap ($42.7M vs $42.9M) is the signature of measure drift — typically gross vs net, booked vs recognised, invoiced vs collected, or including-vs-excluding a small category (returns, credits, non-recurring items). Date-filter drift usually produces larger or smaller gaps depending on how many days differ; source-table drift usually produces gaps in the 5–20% range when a derived aggregate table has different filter logic baked in.

The sequence of checks (fastest first).

1. Confirm the date filter matches on both sides. Ask the finance partner: «Are you reporting on the fiscal quarter or the calendar quarter, and what is the exact end date you cut off at?» Fiscal-vs-calendar and midnight-vs-end-of-business cutoff drift explain a surprising number of «close but not exactly» deltas. If this differs, stop here — you found it.
2. Confirm you’re both querying the same base fact table, not a derived aggregate. Copilot often defaults to a pre-aggregated table (e.g. revenue_summary_daily) because it’s smaller and faster; the finance partner may be pulling from the raw revenue_line_item table with different filter logic. Ask: «Which table are you pulling from?» If different, you have source drift — likely with different definitions baked in.
3. Inspect the Add-to-page field-set on the Copilot visual carrying the $42.7M number. Click the visual, expand the «Fields» pane on the right, and read the exact measure name. In Power BI Desktop, then open Model view, find the measure, and read its DAX expression. Compare it to the finance partner’s definition. This is the check that confirms measure drift — and it takes about ninety seconds if you know where to look.

The specific Add-to-page field-set inspection. The measure Copilot picked will often be named something like [Total Revenue]. The finance-partner measure might be named [Recognised Revenue] or [Net Revenue]. Both live in the model; both look like «the» revenue measure to a casual glance; both produce different totals. The DAX expression under the hood will show the difference — usually a FILTER clause that removes returns, or a CALCULATE that scopes to a specific revenue-recognition category. Once you can name the measure the AI picked, you can either (a) switch to the finance-partner measure with one click, or (b) explicitly reconcile the difference in the report («shown here is Total Revenue ($42.9M) vs Recognised Revenue ($42.7M); the delta is returns and credits»).

The habit-forming discipline. Every Copilot-generated report that cites a number should, before shipping, be checked at least once against a known source-of-truth number from the finance partner. When they diverge, do not close the report; run the three-check sequence, name the failure mode, and reconcile explicitly. Shipping with an unresolved «close but not exactly» gap trains the audience to distrust every subsequent report from you.

Why it’s a Crystallize failure. Refine polishes a chart against a spec. If Refine has run three times and the executive still says «this is not what I wanted,» the chart is now well-polished — the executive is objecting to the chart itself, not to its polish. The executive’s objection is that the report answers the wrong question, and no amount of colour, label, or sort-order polish can convert a chart that answers the wrong question into one that answers the right question. The failure lives upstream in Crystallize, where the question was locked in.

Why the failure disguises itself as a Refine problem. Two reasons. First, Refine is the phase that produces visible symptoms — the exec objects when they look at the polished artefact, not when they read the BRD. Second, the analyst experiences the failure as «the executive keeps changing their mind,» because each Refine cycle produces a different polished version and the exec keeps rejecting all of them. But the exec is not changing their mind; they never got the question right in the first place, and now they’re trying to describe what they want by pointing at what they don’t want.

The specific reset move.

1. Stop Refine immediately. A fourth Refine cycle will produce a fourth polished-wrong chart. Halt.
2. Book a fifteen-minute meeting with the executive. Not a review of the current report — a Crystallize reset. Say up front: «I’ve been polishing the wrong report. I want to reset and make sure I’m answering the right question.» Named as a mistake rather than a defence.
3. Ask three questions. (a) «What decision are you trying to make from this report?» (b) «When you look at what I’ve shown you, what specifically is missing that would let you make that decision?» (c) «If you had to name one number or one comparison that has to be on the page for the report to work for you, what would it be?» The third question is the load-bearing one — it forces the executive to name the specific thing they’ve been indirectly rejecting the previous versions for.
4. Rewrite the BRD in front of them. Not later; there. Say «OK, so the decision is X, the missing element is Y, and the load-bearing number is Z. I’ll rebuild the report against that spec. Can we ship this version tomorrow?» Get a yes on the spec before you touch the data.
5. Rerun the full CSAR loop from Crystallize. New BRD, new prompts, new Scope, new Refine. The first Refine cycle against the correct spec usually produces a report the executive accepts without further iteration — the polish was never the problem.

Why the reset is worth the discomfort. Naming the failure as an upstream mistake feels worse than shipping a fourth polished-wrong version. It isn’t. Three failed reviews have already surfaced the pattern to the executive; a reset move is what the executive is waiting for. The reset saves the fourth cycle, the fifth cycle, and the eventual moment where the executive stops asking you for reports and asks a colleague instead.