Skip to content

ADR-0022: Neutral-prior fix A(ii) — positive rule scraped-negative → OMIT, not zero

Status: Accepted (2026-05-23) Supersedes: — Superseded by: —

Context

ADR-0012 established the neutral-prior scoring model: a 50-point baseline, moving up on positive evidence and down on deduction violations. The intent is that absence of evidence should not move the score — only positively-found evidence should.

The D2 implementation (cycle 14) handled two OMIT cases: - Case A — no live signals at all (no scraper ran this rule for this institution): write the sentinel raw_score=2.5 and exclude from the pillar weighted mean. - Case B — deduction rule ran, found no violations (clean run): same treatment. A clean deduction run is not "good governance found"; it is the absence of bad governance evidence.

A third case was left unhandled: a positive-type rule (membership/disclosure/commitment — rules where boolean=1 is the good outcome and points are added) that is scraped and returns boolean_value=0 for all sub-rules. Under the old code, pointsSum=0 for a positive rule fell through to rawScore = baseScore + 0 = 0 + 0 = 0, scoreValue=0. This made "scraped and found nothing positive" behave identically to "institution failed to join the framework" — a penalty rather than the absence of positive evidence.

Diagnostic finding (run_id=41): M&G PLC (INST-0024), an asset manager with live signals on ~7 positive-type rules all returning boolean_value=0 (not NZBA member, not PRB signatory, no SBTi, etc.), scored 0.0. Institutions with no signals at all (Case A) correctly OMIT and effectively start at neutral. M&G was being penalised relative to an unscraped institution, despite having the same absence of positive evidence. The deduction rules for M&G (G7/S6/E7) all correctly OMIT via Case B (no violations found). The score of 0.0 came entirely from the floor effect of positive rules scoring 0.

This produces a broken two-sided scale: scraped institutions with no positive membership/disclosure evidence score worse than unscraped ones. The scale intent (50+ good citizen, 50 neutral, sub-50 poor citizen) cannot function when "scraped and found nothing" is treated as a penalty.

Decision

A(ii): Positive-type rules with live signals but no positive sub-rule hits → OMIT.

Specifically, the pointsSum === 0 guard in src/scoring/sub_criterion.js is extended to cover both rule types:

  • Case B (deduction, unchanged): A deduction rule with live signals but pointsSum=0 (no violations fired) continues to write the OMIT sentinel. Deduction rules with actual violations always have pointsSum < 0; this branch never fires for a violated deduction rule.
  • Case C (positive, new): A positive rule with live signals but pointsSum=0 (no positive sub-rules fired) now writes the OMIT sentinel instead of scoring 0. The scraper ran, found nothing positive — this is neutral, not a penalty.

The condition changes from: 

if (rule.rule_type === 'deduction' && pointsSum === 0)   // Case B only
to: 
if (pointsSum === 0)   // Cases B and C

G3 special path: The evaluateG3 function (which counts NZBA/SBTi/PRB memberships and maps to a tier score) gains the same fix: when g3.rawScore === 0 (frameworks scraped but none found), write the OMIT sentinel instead of scoreValue=0.

Null-pillar baseline (composite.js): When all three pillars are null (every rule for every pillar OMIT — zero scored evidence in any direction), write score_value=50.0 (the neutral midpoint) to score_stage1_esg with confidence=0. This implements the ADR-0012 intent literally: no evidence → score at neutral. Without this, institutions with only scraped-negative positive signals and clean deduction runs would produce no composite row (or leave a stale 0.0 row), which is both misleading and breaks the before/after distribution comparison.

Rule-type boundary (confirmed before implementation)

The following 31 distinct rule IDs exist. The change applies only to POSITIVE rules:

POSITIVE (A(ii) applies — scraped-negative now OMIT): | Scope | Rule IDs | |---|---| | Financials (sector 40) | E1, E2, E3, E4, E5, E6, G1, G2, G3, G3-PRB, G4, G6, S1, S2, S3, S4, S5 | | All sectors | UN-E1, UN-G1, UN-G2, UN-G3, UN-S1 | | Non-financials | NF-E1, NF-E2, NF-E3, NF-E5-IND, NF-E6-STAP, NF-E7-STAP, NF-S1, NF-S2, NF-S3, NF-S5-CD |

DEDUCTION (unchanged — Case B preserved byte-identically): | Scope | Rule IDs | |---|---| | Financials (sector 40) | E7, E8 (inert), G5, G7, S6 | | Non-financials | NF-E4, NF-G4, NF-S4 |

No ambiguous rules. E8 is inert (no signals ever written due to source retirement in cycle 14 D3); its behavior is unchanged (Case A — no live signals → OMIT).

A deduction rule with violations always has pointsSum < 0, so the pointsSum === 0 condition never fires for a deduction rule that has actually deducted. Case B (deduction, clean) is preserved byte-identically: pointsSum=0 for a deduction rule still means no violations were found.

Consequences

Score distribution (run_id=41, 79 institutions)

Scoring re-run on the same signal data (run_id=41). Before/after headline ESG scores:

Institutions now scoring 50.0 (neutral baseline, all pillars null, confidence=0): Previously scored 0.0. These institutions had scraped-negative positive signals and no deduction violations. They were wrongly penalised. After A(ii), all their rules OMIT and the null-pillar fallback writes 50.0 (neutral). 24 institutions: M&G, Man Group, Ninety One, Colchester, Heptagon, Trium, Octopus Investments, TIAA (financials); Caterpillar, NVIDIA, Stryker, Waste Connections, Amphenol, Copart, Ferguson, Sunbelt Rentals, Brown & Brown (industrials/sector 20); RELX, Constellation Software (IT/sector 45/50); NextEra Energy, ORIT (utilities/sector 55); ICE (sector 40 non-bank); Alcon, HSBC HK.

Scale shift: Every institution rose; no institution decreased (expected: removing a floor effect can only raise or maintain scores).

Before range: 0.0–26.7 (all AMBER or RED under current RAG bands; no institution reached GREEN). After range: 20.0–80.0.

Deduction-institution interaction (see open question below): Institutions with live deduction violations (NGO-TARGETS, FOREST500) now score above 50 in most cases: - Deutsche Bank / Santander / UniCredit: 26.7 → 55.3 (2 E violations + 1 S violation) - BNP Paribas / Citigroup / Nordea: 14.7 → 44.8 (multiple E violations, S null) - BlackRock: 15.4 → 68.6 (2 E violations E7, 1 S violation S6, G null) - Invesco: 4.6 → 80.0 (1 E violation E7.4 FOREST500 only; all other rules OMIT)

These are not implementation bugs. The mechanistic explanation: before A(ii), the floor effect of zero-scoring positive rules was pulling deduction-institution scores below 50. After A(ii), those floor rules OMIT, and the deduction scores (which are 60-80 for 1-2 violations under the 5-point base model) now dominate. The deduction scoring model — where base_score=5, each violation deducts 1, and 3+ violations are needed to go below 50 — was always this shape; A(ii) just makes it visible. See open question below.

Invesco at 80.0: Flagged as notable. E7 with 1 FOREST500 violation (deforestation financier) is the only E-pillar evidence; all other E/S/G rules OMIT. E pillar = 80 (one mild deduction from max). G and S pillars are null. ESG = 80. Mechanically correct under A(ii) + the current deduction model. Does not exceed the 90+ implausibility threshold stated in the diagnostic brief.

RAG bands

Current bands: red <15, amber 15–29, green ≥30 (ADR-0004, calibrated to the run_id=29 distribution of 22 institutions where the max score was ~26.7). These bands are no longer appropriate:

  • Before A(ii): 0 institutions GREEN, ~17 AMBER, ~62 RED.
  • After A(ii): ~50 institutions GREEN (including 24 at 50.0 with confidence=0 — no evidence), ~10 AMBER, ~0 RED.

The green threshold at 30 is now well below the "no evidence" neutral baseline of 50, so green no longer indicates positive ESG behaviour — it just means "we've heard something about you." RAG recalibration is a required follow-up after this change is locked (do not change bands in the same pass as A(ii)).

Known follow-ups (NOT fixed in this commit)

Three items surfaced by this fix are explicitly deferred:

(a) Deduction scoring model — separate ADR pending. The task brief intent was: "institutions with genuine deductions should still sit sub-50." This is not met under the current model (base_score=5, -1 per violation; 3+ violations required to fall below 50 in a pillar). Before A(ii), the scraped-negative floor was masking this. Now visible: Deutsche Bank/Santander/UniCredit sit at 55.3 with 2–3 violations; Invesco at 80.0 with 1 mild E7 violation. Deduction restructuring requires its own decision and ADR — do not conflate with A(ii).

(b) RAG band recalibration — after deduction fix, not before. Current bands (red <15 / amber 15–29 / green ≥30) were calibrated to the pre-A(ii) distribution where the max score was ~26.7. After A(ii), ~50 institutions sit at or above 50; the green threshold is now below the neutral baseline. Recalibrate only after the deduction scoring model is settled, so bands are fitted to the final distribution, not an intermediate one.

(c) Lever B — NZBA/PRB applicability for asset managers — domain review pending. Excluding asset managers from NZBA/PRB membership rules via applicable_sectors narrowing is methodologically correct but requires a domain discussion. Held out of scope for this pass.

Commit status

Committed to main (2026-05-23). Score changes written to DB for run_id=41 as part of diagnostic; DB is not in git. Code landed as fix(scoring): scraped-negative positive rules score neutral, not zero (A(ii); ADR-0022).

Alternatives considered

  • Leave as-is (no change): M&G and 23 other institutions continue to score 0 due to the floor effect of scraped-negative positive rules. Rejected: breaks the neutral-prior intent of ADR-0012 and produces a scale where unscraped institutions score better than scraped ones.

  • Score 50 explicitly at sub-criterion level (not OMIT): Write score_value=50 for scraped-negative positive rules, include in the pillar weighted mean. Rejected: 50 in the mean dilutes real scores toward 50 even when there is no evidence. OMIT-and-exclude is cleaner.

  • Lever B (change applicable_sectors for NZBA/PRB rules, exclude asset managers): Correct approach for a separate domain decision. Held pending methodology discussion with Rob. Note in passing: even with Lever B, a pure-asset-manager institution with no relevant rules at all would score 50 (neutral baseline), which is the correct outcome.

References

  • ADR-0012 — Neutral-prior scoring model (coverage rationale and baseline)
  • ADR-0004 — RAG thresholds (recalibration needed as follow-up)
  • src/scoring/sub_criterion.js — Case C implementation
  • src/scoring/composite.js — null-pillar neutral baseline
  • docs/design/cycle-14-methodology-decisions.md — D2 decision record (Cases A and B)
  • Diagnostic session 2026-05-23 — dual-scoring investigation → A(ii) decision