Tuning paper: cash_30d, high_value
Generated from the Tuning Decision Log (8 decisions, sha256 ccfe1ad1c471aa9f), the TM rule registry TM-RULE-REGISTRY v1.0.0, and the independent challenges in challenges/. Every rate is PLANTED_TRUTH on a synthetic population, never a SAR outcome.
1. Background
cash_30dis proposed, not in the registry: nothing runs on it today (proposed line 15,000, D8).high_valuefires when a single transaction's amount is strictly greater than the threshold. Threshold today: 75,000 (registry v1.0.0).- Known limitation, as the registry records it: Compared on the raw amount whatever the currency, and flat across every customer: a large customer's routine payment and a small customer's fifteen-fold spike are judged by the same line. Measured on the Stage560 planted-truth population it catches 55% of planted value spikes.
- Population: 5,000 customers over 12 months from 2025-10-01, seed 20260924, 4% planted
value_spike(digest7dc1ddbc19ba9e5e).
2. Method
| Decision | Week | Evidence | What it measures |
|---|---|---|---|
| D1 | 3 | threshold_sweep | Every threshold on a grid scored against planted truth: precision, recall, F1, and the threshold a stated false-negative : false-positive cost ratio would choose. |
| D2 | 4 | capacity | The alerts a threshold produces, worked through a queue at a stated review capacity and sample ratio: backlog, the share reviewed within a stated SLA, and effective recall. |
| D3 | 5 | segments | Separate lines per customer segment fitted within a workload budget on the named population, then scored on a hold-out population the fit never saw. |
| D4 | 6 | challenger | Alternative rules fitted within the champion's workload and scored at the same workload on a hold-out, with the overlap, a negative control and the cold-start count. |
| D5 | 7 | drift | Every rule held fixed over periods drifted by a stated scenario; workload, recall, a monthly transient check, and KS/PSI on the inputs. |
| D6 | 8 | calibration | The rules blended into one ranked score worked to a budget; bands of the ranking scored on a hold-out at the same score edges; the cap and the weights swept. |
| D7 | 9 | validation | The advanced challenger with cold-start fallbacks, scored on a hold-out, on a population with the behaviour removed, and under a mean-preserving volatility drift; catches split into attributable and not. |
| D8 | 10 | capstone | A new rule taken through the whole programme: a threshold sweep and a challenger fitted within the owner's engagement terms, judged on an unused seed, under a construction stress test, under drift, against a negative control and against fixed capacity. |
Every decision's evidence is recomputed from the population on every test run and must equal what the log recorded; the log is append-only against the committed copy.
3. Results
D1 (week 3): HOLD
On the TM-SIM planted-truth population GBP 75,000 is dominated: it is not the cost-optimal threshold for any FN:FP ratio from 1 to 100, and the best F1 on the whole curve is 0.316 at GBP 187,500, catching under a third of planted cases. So a flat amount threshold is not the lever. Held rather than moved because any move is a call on what a missed case is worth, which nobody has stated, made on a population whose truth we planted -- tuning a live threshold to it would be fitting our own assumptions.
D2 (week 4): HOLD
Held at GBP 75,000 and the capacity gap named instead. On the stated assumptions -- 300 reviews a day at bank scale, TM-SIM a 1-in-150 sample (2.0 per working day), one alert per customer-month -- today's line makes 2,419 customer-months of work a year, 4.63x capacity. The queue ends the year 1,897 behind and effective recall is 0.135, none of it within 10 working days. The capacity frontier (GBP 177,500) would lift effective recall to 0.325, but the owner chose to keep the line and resource the team: holding it needs about 1,390 reviews a day at bank scale.
D3 (week 5): SEGMENT
Two lines: retail GBP 30,000, everyone else (business, corporate, correspondent) GBP 152,500. At D2's workload (2,419 customer-months a year) the two lines catch 149 of 200 planted customers on a hold-out population, against 122 for the best single line (151 vs 110 where they were fitted). One line per segment catches 154 on the hold-out, but corporate (9 planted) and correspondent (1) sit below the 10-planted floor, so four lines buy a handful more catches with two lines nobody could defend. At a 522 budget two lines (102) match or beat four (100). Decided, not applied: the registry stays at GBP 75,000 until the governed apply step, and it cannot yet express a per-segment threshold at all.
D4 (week 6): HOLD
Hold the champion at GBP 75,000 and advance the customer-baseline challenger to validation; reject amount-and-velocity. At D2's workload, on a hold-out population, the champion catches 122 of 200 planted (4.69 per 100 reviews). The customer-baseline rule (a payment over 4x the customer's own median of the previous 3 months) catches 170 (7.55 per 100) with less work (2,253 customer-months against 2,601), 63 of them ones the champion misses; its negative control shows no lift (0.9342). It is advanced, not adopted, for three reasons on the record: planted truth favours it by construction, it alerts 2.7x as many distinct customers, and 49 planted customers spike before they have enough history to be seen. Amount-and-velocity catches 88, every one of them a customer the champion already catches: it targets bursts, and the planted behaviour has none.
D5 (week 7): HOLD
Hold GBP 75,000 and name what drift does to each rule. Under the stated scenario (10% a year inflation, and non-retail business stepping up 1.6x from month 7 of year 2), with every rule held fixed: the champion's workload moves +73% then +124%; D3's two lines +48% then +85%; the champion indexed to the median payment +45% then +69% -- all past the 25% tolerance. The customer-baseline rule stays within it (+18%, +0%) because it is a ratio to each customer's own spend, but after the step its busiest month runs 1.74x its median month. PSI on payment amounts stays at or below 0.052304 ('stable') throughout: the input distribution barely moves while the tail that threshold rules live in does, so monitoring must watch each rule's alert volume, not the inputs.
- Monitor each rule's OWN monthly alert volume against its first-period level, with the stated tolerances; PSI and KS on payment amounts read stable throughout while the flat rules' workload doubled, so they are not the trigger.
- Recalibrate the flat lines (the champion and D3's two lines) at least annually. Indexing the champion to the median payment was measured and NOT adopted: it only halves the drift, because the drift sits in the tail.
- Make a step-change warm-up part of the customer-baseline rule's week-9 validation: after a segment's business steps up, its monthly workload runs up to 1.74x its median month for about a quarter, the length of its baseline window.
D6 (week 8): HOLD
Hold the rules as they run and guard the cap. The champion and Decision 4's customer-baseline rule, blended 50/50 into one ranked score and worked to the D2 budget of 2,419 reviews, catch 166 planted customers in-sample and 171 on the hold-out, against 161 and 170 for the customer-baseline rule alone and 110 and 122 for the champion; it catches 0 customers neither rule catches. The gain does not justify governing a scoring model yet. It is weight-robust (amount weights 20-60 all catch 161-168) and its ranking is monotone on the hold-out (0 inversions), but the top band over-reads out of sample (45.6% planted in-sample, 34.9% at the same score edge on the hold-out), so a score must not be read as a probability. The cap is the finding: at a cap of 100 or above it costs nothing, because every month tied at it is worked; at 75, 3,375 months tie at the cap -- more than the budget -- and the label-blind alphabetical tie-break cuts recall to 112, the champion's level. In the estate, 6 of 48 entities sit at the macro cap of 100; the estate holds too few dispositioned cases to fit weights, so its preview shows what a change does, not whether it is right.
- Watch the number of alerts tied at any score cap against the capacity that works them. A cap is free while every tied item is reviewed; once the tie exceeds capacity, the alphabetical tie-break decides who is reviewed, not the risk. Applies to the estate's macro cap of 100 as the queue grows.
- Do not present a ranked score's value as a probability. The ranking transfers to unseen data; the score-to-rate mapping at the top does not.
- Macro-score weight changes are previewed through the real scorer (tuning/macro_preview) before any governed change; no weight is changed on the estate's current three dispositioned cases.
D7 (week 9): HOLD, not yet validated
Hold GBP 75,000; the customer-baseline rule is not validated. Re-taken before commit on the independent challenge's evidence: the first draft carried the spike with a fallback forward as the candidate, compared only with the champion, and most of its fallback's extra catches fell in the simulation's first months. Scored against every rule the log holds, and again after a 3-month burn-in, only the champion and D3's two lines alone pass both stated conditions. D3's lines catch 149 on the hold-out, all attributable, against the champion's 122 (105 against 85 attributable after the burn-in), alert 1.30x the champion's customers, move +15% at worst under the volatility drift, and alert 15% of customers on pure noise. They are the candidate for the capstone. The spike with a GBP 2,000 floor and D3's lines as fallback stays the challenger to beat: after the burn-in it still catches 138 attributable against 105, but it alerts 2.91x the champion's customers, moves +55% under the volatility drift, and alerts 39% of customers on pure noise, so it fails both conditions.
- Condition 1, volatility: at sigma 0.3 over three seeds, workload must stay within D5's 25% tolerance. Stated, not derived; the size of a realistic variance drift is unknown.
- Condition 2, customers: distinct customers alerted must stay within 1.5x the champion's until the cost of a first alert is stated in its own unit (D4 challenge, point 2).
- The candidate inherits D3's challenge: its gain depends on how closely the modelled segment tracks customer size, and its non-retail line is one seed's optimum. Both must be settled on real book data before any apply.
- No live change: the registry has no rule kind for per-segment lines, and the governed apply step refuses D3 until it does.
D8 (week 10): PROPOSE
Propose a new 30-day cash rule, not applied: the finding is capacity, not a line. Re-taken before commit on the independent challenge's evidence. At the fixed 522 reviews a year, the team reviews only part of any line's alerts, so what it catches is decided by the order it works them in. Worked in signature order -- customer-months with the most deposits of GBP 4,000-10,000 in 30 days first -- the proposed line's queue catches 90 of 100 planted structurers on an unused seed, against 88 for the best rule that fits capacity on its own; with structurers who make fewer deposits, 52 against 53. Every plan tested lands in the same place at this capacity, so the engagement -- 20% fewer alerts with at most 5 points of recall lost -- cannot be met by tuning; it is D2's staffing gap again, now on a second rule. The line proposed as the target once capacity exists is GBP 15,000 (4,312 reviews against 6,207); it holds within the tolerance when structurers start lower or spread wider, but loses 8% when they make fewer deposits, and the figure is conditional on the planted pattern it was fitted to. The signature fails every stress and is used only to order the queue. On the same unused seed D7's candidate holds: D3's two lines catch 158 against the champion's 120.
- Capacity: at the fixed capacity no plan meets the engagement; the D2 staffing ask now covers two rules, and the cash rule's share of the team must be stated before any apply.
- Construction: the GBP 15,000 target and the signature's band are both conditional on the planted structuring pattern; neither may be applied without evidence from real structuring cases.
- Scale: the population plants structuring at 2%, far above any real prevalence, which flatters every rule's catches per review.
- No live change: the registry holds no cash rule, and the apply step refuses a new rule it cannot hold.
4. Recommendation
The latest decision is D8 (PROPOSE). When the cash rule's share of review capacity is stated and real cash-deposit data exists: re-fit the line and the signature on it, re-score the ordered queue at that capacity, and take any apply decision through the governed step once the registry can hold a 30-day cash rule.
What the governed apply step does with each decision today:
| Decision | Apply step |
|---|---|
| D1 | NOTHING_TO_APPLY |
| D2 | NOTHING_TO_APPLY |
| D3 | REFUSED: D3 sets per-segment lines, and the registry has no rule kind that can hold them (kinds: ['amount_above']); the step will not invent one |
| D4 | NOTHING_TO_APPLY |
| D5 | NOTHING_TO_APPLY |
| D6 | NOTHING_TO_APPLY |
| D7 | NOTHING_TO_APPLY |
| D8 | REFUSED: D8 proposes a NEW rule, 'cash_30d'; the registry holds no such rule, and adding one is a registry change the step will not invent |
5. Limitations
Every point an independent reviewer raised, with the builder's response. The full memos are committed beside the log.
D1: not independently challenged.
D2: not independently challenged.
D3: reviewer's verdict STANDS WITH CONDITIONS (challenges/D3.md).
- 1. The gain depends on an assumed link between segment and customer size (HIGH), response: ACCEPTED
- 2. The gain is a shift of work and catches from business customers to retail, and the rationale doesn't disclose it (MEDIUM), response: ACCEPTED
- 3. The non-retail GBP 152,500 figure is weakly determined (MEDIUM), response: ACCEPTED
- 4. The logged lines assume staffing that D2 said does not exist, and later evidence outperforms them (MEDIUM), response: ACCEPTED
- 5. "Two lines nobody could defend" overstates the case against four lines (LOW), response: ACCEPTED
- 6. Workload budgets are not held on the hold-out, and the guard test is minimal (LOW), response: ACCEPTED
The reviewer could not check: How payment scale relates to KYC segment on a real book, which is the parameter point 1 shows the decision depends on; Anything measured against real SAR or disposition outcomes. None exist at this volume; Whether the calendar-month workload in segments.curves matches D2's "30-day dedupe" unit exactly. The 2,419 totals agree, but I did not trace the capacity code; The full test suite, which I did not run. My measurements were in-memory runs through segments/tm_sim (the noise change was made in the running Python session only), and I wrote nothing to the repository.
D4: reviewer's verdict STANDS WITH CONDITIONS (challenges/D4.md).
- 1. The win is measured on the only behaviour the rule was designed to find (HIGH), response: ACCEPTED
- 2. "Less work" depends on counting work in customer-months (HIGH), response: PARTLY ACCEPTED
- 3. The negative control cannot fail in the way that matters (MEDIUM), response: ACCEPTED
- 4. Some "catches" are noise, and the credited figures include them (MEDIUM), response: ACCEPTED
- 5. The cold-start figure mixes a simulation artefact with a real blind spot (MEDIUM), response: ACCEPTED
- 6. The fit is decided by the budget, not by the grid search (LOW), response: ACCEPTED
- 7. "REJECT" for amount-and-velocity is really "untestable here" (LOW), response: ACCEPTED
The reviewer could not check: How either rule performs on real SAR outcomes or on typologies other than value_spike, because none exist in TM-SIM; The actual review cost of a first-time customer alert against a repeat month. The cost unit is an assumption I could only question, not measure; The week-9 validation that D4 defers to; The rest of the suite. I only ran measurement scripts (from the scratchpad, nothing written to the repo), no tests.
D5: reviewer's verdict STANDS WITH CONDITIONS (challenges/D5.md).
- 1. The ratio rule's "STABLE" rating is built into the scenario (HIGH), response: ACCEPTED
- 2. Rejecting the indexed champion depends entirely on the assumed 1.6x step (MEDIUM), response: ACCEPTED
- 3. There is no baseline for normal year-to-year variation, and recall tolerance sits inside that variation (MEDIUM), response: ACCEPTED
- 4. The tolerances that decide the verdicts have no stated source (MEDIUM), response: ACCEPTED
- 5. The warm-up is only half measured (LOW), response: ACCEPTED
- 6. The rationale's numbers and the mitigants are not tested (LOW), response: ACCEPTED
The reviewer could not check: Whether 10% inflation and a 1.6x non-retail step are realistic for any real book. The scenario is stated, not observed, and D5 says so; Behaviour against real SAR outcomes. Every rate is measured against planted truth; Drift carried across year boundaries. Each year is an independent population, so gradual customer turnover, new customers or rising transaction counts were not modelled; Whether my volatility perturbation (extra σ=0.3) is realistic in size. It shows the direction of the weakness, not how large it would be in practice.
D6: reviewer's verdict STANDS WITH CONDITIONS (challenges/D6.md).
- 1. "At a cap of 100 or above it costs nothing" belongs to this synthetic score and this budget, not to the estate (HIGH), response: ACCEPTED
- 2. The one estate fact is imprecise, and ordering costs are not measured (MEDIUM), response: ACCEPTED
- 3. The blended score's gain is within seed-to-seed noise, and the headline comparison uses unequal workloads (MEDIUM), response: ACCEPTED
- 4. D4's "favoured by construction" caveat was dropped (MEDIUM), response: PARTLY ACCEPTED
- 5. "The ranking transfers to unseen data" is shown on one sibling draw only (LOW), response: ACCEPTED
- 6. "0 inversions" rests on small counts (LOW), response: ACCEPTED
The reviewer could not check: Any real outcome data. The estate has three dispositioned cases, and every rate here is planted truth; D5's actual segmented drift scenario applied to the blended score. My 1.6× test was a stronger, uniform stand-in; The estate's real daily review capacity, which the revisit trigger depends on; Whether the typology tunable scores in escalate_queue_priority will separate tied entities as more code families feed into them. Today every tied row has priority_source = macro.
D7: reviewer's verdict STANDS WITH CONDITIONS (challenges/D7.md).
- 1. Most of what the fallback adds comes from the simulation's first three months (HIGH), response: ACCEPTED
- 2. The candidate is compared with the champion, not with D3's lines alone (HIGH), response: ACCEPTED
- 3. The two conditions have no pass mark (MEDIUM), response: ACCEPTED
- 4. The candidate is over the budget that governed D4 (MEDIUM), response: ACCEPTED
- 5. Two promises made to earlier challenges are not delivered (MEDIUM), response: ACCEPTED
- 6. One hold-out has been reused across four decisions (LOW), response: ACCEPTED
The reviewer could not check: Performance on any typology other than value_spike, or on real SAR outcomes. D4 point 1 is still open, and D7 defers it to the capstone; The real review cost of a first alert on a customer compared with a repeat month; Whether σ 0.3, or any σ, is a realistic size for variance drift; The rest of the test suite. I ran only measurement scripts from the scratchpad and wrote nothing to the repo.
D8: reviewer's verdict STANDS WITH CONDITIONS (challenges/D8.md).
- 1. "Orders the queue" is asserted, not evaluated, and at capacity the signature becomes the line in practice (HIGH), response: ACCEPTED
- 2. The GBP 15,000 figure is set by the generator, and the stress test moves the wrong dial (HIGH), response: ACCEPTED
- 3. The capacity figure is the wrong comparator (MEDIUM), response: ACCEPTED
- 4. The naive baseline is perfect by construction, and the "change" is from a rule that does not exist (MEDIUM), response: ACCEPTED
- 5. The v1→v2 revision cannot be audited (MEDIUM), response: PARTLY ACCEPTED
- 6. The decision record names the wrong population (LOW), response: ACCEPTED
The reviewer could not check: v1's actual code or the seed it was measured on. My v1 run is a rebuild from the docstring's description; Any real cash-deposit data or real structuring cases; Whether 522 was meant as the cash team's capacity or the whole team's. D8 does not say; The full test suite. I ran only measurement scripts from the scratchpad and wrote nothing to the repo.