How the scoring works
Two questions per topic, asked of two texts that the model never sees together. From the outside report alone: should this topic show up in the financial statements, and if not, why not? From the financial statements alone: what is actually there, and where, or why is it not? Code verifies every quote and compares the two answers into one outcome.
Method
We score how outside climate disclosures connect to the audited financial statements with a reasoning LLM (gpt-6-luna, medium reasoning effort, via the batch API), in two separate stages, so the model never sees both disclosure venues in the same call.
- Input: all inside and all outside paragraphs flagged as climate-related by DisclHits or DisclBERT.
- Unit: each firm-year, assessed separately for the same 9 climate topics: the seven TCFD cross-industry metric categories, targets and transition plans, and climate litigation.
Stage 1 Outside read → topic profile
The model sees only the outside paragraphs. It is called once per topic (nine calls per firm-year), with the other eight topics listed as exclusions, and returns for that topic:
- Discussed outside: whether the topic is substantively discussed, not merely mentioned in passing;
- Summary and key elements: what the firm discloses, plus up to eight concrete specifics (named assets, amounts, targets, instruments, obligations);
- Outside quotes: one to four literal quotations supporting the summary;
- FS reflection expected: yes or no. Would a reasonable, financially literate user of the financial statements expect this topic to show up there, given the outside disclosure alone? A yes rests on a triggering outside quote, which code verifies;
- Boundary reason, when no reflection is expected: the six reasons below (EFRAG Connectivity DP 2025, para 5.13) are tested in order, and the first that applies is the primary reason; a later one may be added as secondary;
- Justification: one or two sentences, in either case.
The expectation is formed before the model sees the financial statements, so it cannot be tailored to what they contain.
Stage 2 FS check → what is there, and where
The model is called once per topic discussed outside. It sees all inside paragraphs and that topic's Stage 1 profile (verified outside quotes only, never the raw outside text). It never sees the expectation, only what the outside report says, and describes what the FS contain:
- Reflection (0–3) with verbatim inside quotes, counting only content on this topic's subject matter; the quotes must address the same matter as the profile's key elements;
- Anchor points (13 FS areas, which also tell general from topical notes), and the forms, each with its own quote: qualitative, measurement, recognition, no effect;
- Why not, when the matter is not specifically reflected: a verbatim stated reason if the FS say why there is no (further) effect, and the primary (and optional secondary) boundary reason that applies on the FS facts, with a short justification.
Outcome, computed in code
| Reflected | Expected, and the FS specifically reflect it: a grounded reflection of at least the threshold with a verified content form other than a bare no-effect statement. |
|---|---|
| Boundary reason stated | Expected, not specifically reflected, but the FS state why: a verified stated-reason quote or no-effect statement. |
| Boundary reason inferred | Expected, not specifically reflected, nothing stated, but Stage 2 names a boundary reason that applies. |
| Residual gap | Expected, not specifically reflected, and neither the FS nor Stage 2 give a boundary reason. |
| Nothing expected | Stage 1 expects no FS reflection (its boundary reasons say why). |
When nothing is expected, both stages name reasons independently and the drawer shows whether they overlap; a topic reflected although nothing was expected is flagged too.
Quote-grounding
Each stage must quote the one text it saw, and code verifies every quotation against that text (ignoring case, spacing and punctuation). Unverifiable evidence downgrades the result:
- unverified outside quotes are dropped before Stage 2;
- an expectation whose triggering outside quote cannot be verified is dropped;
- FS quotes are verified against the full inside text;
- a stated reason whose quote does not verify does not count as stated;
- a form whose quote does not verify is set to no;
- a reflection score ≥ 1 without a verified inside quote is treated as 0.
As before, the prompts make clear that negative answers are valid and common: not discussed, not reflected, nothing expected.
Boundary reasons
| not fs type item | Not an FS-type information item. The item is itself not of a type ordinarily recognised or disclosed in the FS, such as a sustainability metric, target, strategy, governance or narrative disclosure. This says nothing about whether the underlying matter (e.g. the emissions behind a metric) can affect the FS (EFRAG DP 5.13 a) |
|---|---|
| outside reporting entity | Outside the reporting entity boundary. The item relates to entities, operations or assets outside the consolidated reporting entity of the FS: value-chain items, or items included under operational control that the entity does not control under IFRS 10 (EFRAG DP 5.13 d) |
| element definition not met | FS element definition not met. The matter has not given rise to an asset, liability or other financial-statement element because the relevant definition is not met, for example because no present obligation to a third party or no present right exists. A commitment to future action (a net-zero target, planned decarbonisation capex, an uncommitted restructuring) creates no present obligation, whatever its public prominence (EFRAG DP 5.13 c; IFRIC agenda decision on net-zero commitments) |
| recognition criteria not met | FS recognition or disclosure criteria not met. An asset, liability or other element exists, but it is neither recognised nor specifically disclosed because an outflow is not probable, measurement is too uncertain, or the effect falls outside the disclosure horizon (IAS 1.125, next financial year; the impairment projection period). Does not apply if the matter IS disclosed, e.g. as a contingent liability (EFRAG DP 5.13 c, g) |
| immaterial for inclusion | Immaterial for inclusion. The matter could in principle affect the FS, but no separately identifiable recognition or disclosure is required because its effect is quantitatively and qualitatively immaterial to the FS as a whole (EFRAG DP 5.13 b; IAS 1.31) |
| aggregated presentation | Aggregated presentation. There is evidence that the matter is reflected in recognised amounts or disclosures, but only within a broader line item, estimate or aggregated disclosure, so it is not separately identifiable (EFRAG DP 5.13 e) |
Reflection 0–3
Stage 2 scores how far the financial statements reflect the topic at all. Only inside content about this topic counts; adjacent climate content does not.
| 0 · No meaningful | No meaningful reflection: the financial statements do not address this topic at all, or only climate content that belongs to a different topic. |
|---|---|
| 1 · Limited | Limited: a generic or boilerplate mention only (e.g. "climate risks are considered"), nothing specific to this topic. |
| 2 · Moderate | Moderate: the topic is addressed specifically — a judgement, assumption, risk or estimation-uncertainty disclosure names it — but nothing is quantified. |
| 3 · Substantial | Substantial: specific AND quantified or recognised — an amount, sensitivity, useful life, provision, impairment or explicit no-effect conclusion attributable to this topic. |
A reflection ≥ 1 must be backed by verbatim FS quotes, checked in code ignoring spacing and punctuation; if none is found the score is treated as 0. Forms: qualitative (the matter is described without an amount or assumption (plan or instrument terms, risk disclosure, accounting policy, provision description, capital commitment)); measurement (the matter is an input or assumption to a carrying amount (impairment key assumption, useful-life basis, ecl or fair-value input, ias 1.125 estimation uncertainty)); recognition (a recognised fs amount attributable to this matter (impairment, provision, useful-life change, ecl, capitalised expenditure, an allowance line, a claims amount)); no effect (the fs explicitly conclude the matter has no material effect, or that no impairment / provision / adjustment resulted from it).
What the reviewer judges
Open a cell. The drawer shows the passages on the left, with every verified quote highlighted in place, and the two stages on the right. Per topic:
- Discussed outside. Is the topic substantively discussed in the outside passages? Confirm the model's yes/no or mark it wrong.
- FS reflection expected. Yes or no, judged from the outside text alone. The model's answer is marked, with its justification and trigger quote underneath.
- Boundary reasons (outside), when no reflection is expected: every reason that applies, or none; other takes a short label.
- Reflection. Set your own 0–3 for how the financial statements reflect the topic. Use the highlighted FS quotes.
- Boundary reasons (FS), when the topic is not (specifically) reflected, with the stated reason quote if there is one.
- Anchor points and form, when there is some reflection. Confirm the model's chips as shown, or toggle them to correct it.
Anything left untouched is recorded as unrated. Ratings are append-only and attributed to your account; saving again adds a new entry. Ratings made under an earlier scoring contract are retired: still listed, but not counted.
The grid
Rows are firm-years, columns the 9 outside climate topics. A cell shows the reflection score for topics discussed outside, or "–" when the topic was not discussed (no Stage 2 call). A dark corner marks topics where Stage 1 expects an FS reflection; a red bar marks a gap without good reason. The outcome is shown read-only in the drawer and is not rated.