Truflation as a proxy for US CPI¶

Project: Algebris_Case_Study_Shi_Yuchen
Python: 3.12.14

All raw snapshot hashes verified.

Rebuild: success_with_warnings

Forecast cutoff: 2026-10-18 | target month: 2026-10 | scheduled release: 2026-11-10
Selected model: Truflation + gasoline (OLS)
Information set: {
  "target": "2026-10",
  "asof": "2026-10-18",
  "last_cpi_month": "2026-09",
  "steps": 1,
  "y_last": 3.737684729064039,
  "y_prev": 3.3965478924364856,
  "truf_t": 2.8972765625,
  "truf_last": 2.3885172,
  "truf_days": 16,
  "d_truf": 0.5087593625000002,
  "momentum": 0.34113683662755356,
  "gap_last": 1.3491675290640393,
  "gas_t": 40.43115423854942,
  "gas_last": 37.72323494535617,
  "d_gas": 2.7079192931932496,
  "gas_weeks": 2,
  "actual": null,
  "actual_release": "2026-11-10"
}

{
  "synthetic": true,
  "status": "accepted",
  "extraction_key": "dc809af1a9315af924b8c2df1792f2c2802b5d3ecd6a294823ab424e5f58c754",
  "event_count": 1,
  "notice": "Entirely synthetic offline fixture; no actual ISW report was acquired and no real LLM was called."
}
{
  "extraction_key": "dc809af1a9315af924b8c2df1792f2c2802b5d3ecd6a294823ab424e5f58c754",
  "article": {
    "canonical_url": "https://example.invalid/synthetic-isw-demo",
    "title": "SYNTHETIC DEMO: fictional conflict and inflation report",
    "publication_date": "2026-03-02",
    "retrieved_at": "2026-10-18T13:57:48.291571Z",
    "source_kind": "synthetic_fixture",
    "raw_sha256": "b6c764dcc2b737a3e46cfd231f6f7b8c9b263847eb61f9b38e23030b082976fb",
    "text_sha256": "ae3a70cd13f1f7efd15f863f69b57171e29edd59e3b28c8e0ed595e53d427888",
    "parser_version": "html-paragraphs-2-browser-dom",
    "paragraphs": [
      {
        "paragraph_id": "p0001",
        "text": "This document is an invented software test fixture. It is not an ISW article and describes no actual event.",
        "links": [],
        "reference_ids": []
      },
      {
        "paragraph_id": "p0002",
        "text": "Fictional Port Aurora's operator said oil loading was temporarily suspended on March 1, 2026.",
        "links": [],
        "reference_ids": []
      },
      {
        "paragraph_id": "p0003",
        "text": "The fictional operator gave no interruption duration, capacity loss or verified volume estimate.",
        "links": [],
        "reference_ids": []
      },
      {
        "paragraph_id": "p0004",
        "text": "A fictional analyst said a prolonged interruption could require longer shipping routes. This is a conditional assessment.",
        "links": [],
        "reference_ids": []
      }
    ],
    "references": {},
    "acquisition_metadata": {},
    "narrative_character_count": 420,
    "reference_character_count": 0,
    "body_selector": "article",
    "media_links": [],
    "parsing_warnings": []
  },
  "extracted_at": "2026-10-18T13:57:48.298084Z",
  "model_id": "synthetic/fake-client-v1",
  "prompt_version": "conflict-inflation-3",
  "schema_version": "1.0",
  "payload": {
    "schema_version": "1.0",
    "relevant": true,
    "summary": "SYNTHETIC DEMO: a fictional port interruption illustrates structured extraction.",
    "events": [
      {
        "event_type": "shipping_disruption",
        "claim_status": "attributed_claim",
        "event_date": "2026-03-01",
        "date_text": "March 1, 2026",
        "actors": [
          "Fictional Port Aurora operator"
        ],
        "location": "Fictional Port Aurora",
        "commodities": [
          "oil"
        ],
        "summary": "SYNTHETIC: the fictional port operator reported a temporary loading suspension.",
        "quantities": [],
        "evidence": [
          {
            "paragraph_id": "p0002",
            "quote": "Fictional Port Aurora's operator said oil loading was temporarily suspended on March 1, 2026."
          }
        ],
        "impacts": [
          {
            "channel": "shipping_freight",
            "direction": "up",
            "horizon": "weeks",
            "materiality": "unknown",
            "basis": "analyst_inference",
            "rationale": "If an actual interruption persisted, detours could raise freight costs; this fictional example cannot quantify CPI."
          }
        ],
        "confidence": "high",
        "uncertainties": [
          "Entire event is fictional; duration and volumes are unspecified."
        ]
      }
    ],
    "limitations": [
      "Synthetic fixture only; no real ISW article or live LLM output."
    ]
  },
  "attempts": [
    {
      "attempt": 1,
      "model_id": "synthetic/fake-client-v1",
      "validation_version": "exact-quotes-2",
      "request_id": "synthetic-call-1",
      "usage": {
        "synthetic": true
      },
      "response_sha256": "d89849bc13d1abb9ad3a6ba5912e7e18bc6a4ed110d528b388ece7589384bdb5",
      "response_text": "{\"schema_version\": \"1.0\", \"relevant\": true, \"summary\": \"SYNTHETIC DEMO: a fictional port interruption illustrates structured extraction.\", \"events\": [{\"event_type\": \"shipping_disruption\", \"claim_status\": \"attributed_claim\", \"event_date\": \"2026-03-01\", \"date_text\": \"March 1, 2026\", \"actors\": [\"Fictional Port Aurora operator\"], \"location\": \"Fictional Port Aurora\", \"commodities\": [\"oil\"], \"summary\": \"SYNTHETIC: the fictional port operator reported a temporary loading suspension.\", \"quantities\": [], \"evidence\": [{\"paragraph_id\": \"p0002\", \"quote\": \"Fictional Port Aurora's operator said oil loading was temporarily suspended on March 1, 2026.\"}], \"impacts\": [{\"channel\": \"shipping_freight\", \"direction\": \"up\", \"horizon\": \"weeks\", \"materiality\": \"unknown\", \"basis\": \"analyst_inference\", \"rationale\": \"If an actual interruption persisted, detours could raise freight costs; this fictional example cannot quantify CPI.\"}], \"confidence\": \"high\", \"uncertainties\": [\"Entire event is fictional; duration and volumes are unspecified.\"]}], \"limitations\": [\"Synthetic fixture only; no real ISW article or live LLM output.\"]}",
      "status": "accepted"
    }
  ],
  "human_review_status": "unreviewed"
}
{
  "asof": "2026-10-18T13:57:48.310335+00:00",
  "include_synthetic": true,
  "report_count": 1,
  "event_mentions": 1,
  "days": {
    "2026-03-02": {
      "report_count": 1,
      "relevant_report_count": 1,
      "event_mentions": 1,
      "channel_direction_mentions": {
        "shipping_freight:up": 1
      },
      "synthetic_reports": 1
    }
  },
  "events": [
    {
      "extraction_key": "dc809af1a9315af924b8c2df1792f2c2802b5d3ecd6a294823ab424e5f58c754",
      "event_index": 0,
      "source_url": "https://example.invalid/synthetic-isw-demo",
      "publication_date": "2026-03-02",
      "available_at": "2026-10-18T13:57:48.299084+00:00",
      "last_observed_at": "2026-10-18T13:57:48.291571+00:00",
      "observation_id": "5814abe54740447db8ace6b10246de83",
      "observation_time_basis": "fetch",
      "first_extracted_at": "2026-10-18T13:57:48.298084+00:00",
      "source_kind": "synthetic_fixture",
      "event_date": "2026-03-01",
      "claim_status": "attributed_claim",
      "summary": "SYNTHETIC: the fictional port operator reported a temporary loading suspension.",
      "evidence": [
        {
          "paragraph_id": "p0002",
          "quote": "Fictional Port Aurora's operator said oil loading was temporarily suspended on March 1, 2026.",
          "paragraph_links": []
        }
      ],
      "impacts": [
        {
          "channel": "shipping_freight",
          "direction": "up",
          "horizon": "weeks",
          "materiality": "unknown",
          "basis": "analyst_inference",
          "rationale": "If an actual interruption persisted, detours could raise freight costs; this fictional example cannot quantify CPI."
        }
      ],
      "human_review_status": "unreviewed"
    }
  ],
  "quarantined_attempts": 0,
  "failed_source_urls": [],
  "limitations": [
    "Counts are report mentions, not unique independently verified events.",
    "Latest observation of accepted content per URL is used; failed revisions can leave the last successful observation stale.",
    "No-report days are unknown coverage, not zero conflict or zero inflation pressure.",
    "Publication-date bins are descriptive; use available_at for point-in-time applications.",
    "Economic direction is a conditional qualitative assessment, not a CPI forecast."
  ]
}

"""Versioned extraction contract. Provenance is supplied by code, never by the LLM."""
from __future__ import annotations

from datetime import date, datetime
from typing import Literal

from pydantic import BaseModel, ConfigDict, Field, model_validator

SCHEMA_VERSION = "1.0"


class StrictModel(BaseModel):
    model_config = ConfigDict(extra="forbid", str_strip_whitespace=True, allow_inf_nan=False)


class Evidence(StrictModel):
    paragraph_id: str = Field(pattern=r"^p\d{4}$", description="ID in the supplied article.")
    quote: str = Field(min_length=8, max_length=500, description="Exact excerpt from that paragraph.")


class Quantity(StrictModel):
    """A source-reported magnitude, not an LLM-calculated economic effect."""

    metric: str = Field(min_length=1, max_length=100)
    value: float | None = Field(description="Use null when only a qualitative statement is available.")
    unit: str = Field(min_length=1, max_length=80, description="Preserve source units; never silently convert.")
    qualifier: Literal["exact", "approximate", "at_least", "at_most", "unknown"]
    time_period: str | None = Field(max_length=100, description="Source-stated period, else null.")
    evidence: Evidence


class InflationImpact(StrictModel):
    channel: Literal["energy_supply", "shipping_freight", "food_fertilizer", "insurance_risk",
                     "expectations_fx", "demand", "other"]
    direction: Literal["up", "down", "ambiguous", "none", "unknown"]
    horizon: Literal["days", "weeks", "months", "unknown"]
    materiality: Literal["low", "medium", "high", "unknown"]
    basis: Literal["source_assessment", "analyst_inference"] = Field(
        description="Use analyst_inference unless the report itself states the economic assessment.")
    rationale: str = Field(min_length=10, max_length=600,
                           description="Explain conditional transmission, not a numeric CPI forecast.")


class ConflictEvent(StrictModel):
    event_type: Literal["energy_asset_damage", "supply_disruption", "shipping_disruption",
                        "sanctions_policy", "military_escalation", "deescalation", "other"]
    claim_status: Literal["reported_observation", "attributed_claim", "source_assessment", "forecast"]
    event_date: date | None = Field(description="Exact event day only if supported; otherwise null.")
    date_text: str | None = Field(max_length=150, description="Original timing phrase, else null.")
    actors: list[str] = Field(max_length=20)
    location: str | None = Field(max_length=160)
    commodities: list[Literal["oil", "natural_gas", "refined_fuels", "food", "fertilizer", "other"]] = Field(max_length=6)
    summary: str = Field(min_length=10, max_length=600)
    quantities: list[Quantity] = Field(max_length=10)
    evidence: list[Evidence] = Field(min_length=1, max_length=8)
    impacts: list[InflationImpact] = Field(max_length=7)
    confidence: Literal["low", "medium", "high"] = Field(
        description="Confidence in faithful extraction, not probability that the underlying claim is true.")
    uncertainties: list[str] = Field(max_length=10)


class ExtractionPayload(StrictModel):
    schema_version: Literal["1.0"]
    relevant: bool
    summary: str = Field(min_length=10, max_length=1000)
    events: list[ConflictEvent] = Field(max_length=60)
    limitations: list[str] = Field(max_length=20)

    @model_validator(mode="after")
    def relevance_matches_events(self):
        if self.relevant != bool(self.events):
            raise ValueError("relevant must be true exactly when extracted events are present")
        return self


class Paragraph(StrictModel):
    paragraph_id: str = Field(pattern=r"^p\d{4}$")
    text: str = Field(min_length=1)
    links: list[str]
    reference_ids: list[str] = Field(default_factory=list)


class SourceReference(StrictModel):
    reference_id: str
    label: str
    text: str
    links: list[str]
    alias_of: str | None = None


class Article(StrictModel):
    canonical_url: str
    title: str
    publication_date: date | None
    retrieved_at: datetime
    source_kind: Literal["isw_http", "local_html", "browser_dom", "synthetic_fixture"]
    raw_sha256: str = Field(pattern=r"^[a-f0-9]{64}$")
    text_sha256: str = Field(pattern=r"^[a-f0-9]{64}$")
    parser_version: str
    paragraphs: list[Paragraph] = Field(min_length=1)
    references: dict[str, SourceReference] = Field(default_factory=dict)
    acquisition_metadata: dict = Field(default_factory=dict)
    narrative_character_count: int = Field(default=0, ge=0)
    reference_character_count: int = Field(default=0, ge=0)
    body_selector: str = "unspecified"
    media_links: list[str] = Field(default_factory=list)
    parsing_warnings: list[str] = Field(default_factory=list)

    @model_validator(mode="after")
    def aware_retrieval(self):
        if self.retrieved_at.utcoffset() is None:
            raise ValueError("retrieved_at must include a timezone")
        return self


class StoredExtraction(StrictModel):
    extraction_key: str
    article: Article
    extracted_at: datetime
    model_id: str
    prompt_version: str
    schema_version: str
    payload: ExtractionPayload
    attempts: list[dict]
    human_review_status: Literal["unreviewed"] = "unreviewed"


def strict_json_schema() -> dict:
    """Use required-but-nullable fields and closed objects for provider strict mode."""
    return ExtractionPayload.model_json_schema()

Estimated at peak uncached prices: CNY 2/M input and 8/M output tokens. Includes pilots and repairs; actual provider billing may be lower.
Counts are report mentions after automated semantic review. Repeated reports can describe one incident. Dates are publication dates; October collection prevents a historical trading backtest. Human verification and extraction-accuracy measurement remain outstanding.

{
  "event_id": "2026-02-28-iran-update-evening-special-report-february-28-2026-e4dfe81f508d:e002",
  "title": "Iran Update Evening Special Report: February 28, 2026",
  "source_url": "https://www.criticalthreats.org/analysis/iran-update-evening-special-report-february-28-2026",
  "retrieved_at": "2026-10-18T12:17:09.681000Z",
  "raw_event": {
    "event_type": "shipping_disruption",
    "claim_status": "attributed_claim",
    "event_date": null,
    "date_text": null,
    "actors": [
      "MarineTraffic",
      "United States",
      "Israel",
      "Iran"
    ],
    "location": "Strait of Hormuz",
    "commodities": [
      "oil"
    ],
    "summary": "Commercial ship traffic in the Strait of Hormuz dropped by 70 percent in response to the combined US and Israeli strike campaign on Iran, according to ship tracking platform MarineTraffic. The passage notes 20 percent of global oil supply transits the strait.",
    "quantities": [
      {
        "metric": "commercial ship traffic decline in Strait of Hormuz",
        "value": 70.0,
        "unit": "percent",
        "qualifier": "exact",
        "time_period": null,
        "evidence": {
          "paragraph_id": "p0039",
          "quote": "Commercial ship traffic dropped by 70 percent in the Strait of Hormuz in response to the United States and Israel’s combined strike campaign on Iran, according to ship tracking platform MarineTraffic."
        }
      },
      {
        "metric": "share of global oil supply passing through Strait of Hormuz",
        "value": 20.0,
        "unit": "percent",
        "qualifier": "exact",
        "time_period": null,
        "evidence": {
          "paragraph_id": "p0039",
          "quote": "Twenty percent of the global oil supply passes through the strait."
        }
      }
    ],
    "evidence": [
      {
        "paragraph_id": "p0039",
        "quote": "Commercial ship traffic dropped by 70 percent in the Strait of Hormuz in response to the United States and Israel’s combined strike campaign on Iran, according to ship tracking platform MarineTraffic."
      },
      {
        "paragraph_id": "p0039",
        "quote": "Twenty percent of the global oil supply passes through the strait."
      }
    ],
    "impacts": [
      {
        "channel": "shipping_freight",
        "direction": "up",
        "horizon": "weeks",
        "materiality": "high",
        "basis": "analyst_inference",
        "rationale": "A reported 70 percent drop in commercial transits through a chokepoint carrying about a fifth of global oil supply could raise freight and insurance costs and tighten energy logistics, conditional on the disruption persisting."
      },
      {
        "channel": "energy_supply",
        "direction": "up",
        "horizon": "weeks",
        "materiality": "medium",
        "basis": "analyst_inference",
        "rationale": "Reduced tanker transits through Hormuz could constrain crude flows and support oil prices if shippers continue avoiding the strait; the passage does not quantify price effects."
      }
    ],
    "confidence": "medium",
    "uncertainties": [
      "Traffic figure is attributed to MarineTraffic via the report, not independently observed.",
      "No event date is given for the traffic drop; article publication date is not used as event date.",
      "Duration and persistence of the disruption are unknown."
    ]
  },
  "analysis_event": {
    "event_type": "shipping_disruption",
    "claim_status": "attributed_claim",
    "event_date": null,
    "date_text": null,
    "actors": [
      "MarineTraffic",
      "United States",
      "Israel",
      "Iran"
    ],
    "location": "Strait of Hormuz",
    "commodities": [
      "oil"
    ],
    "summary": "Commercial ship traffic in the Strait of Hormuz dropped by 70 percent in response to the combined US and Israeli strike campaign on Iran, according to ship tracking platform MarineTraffic. The passage notes 20 percent of global oil supply transits the strait.",
    "quantities": [
      {
        "metric": "commercial ship traffic decline in Strait of Hormuz",
        "value": 70.0,
        "unit": "percent",
        "qualifier": "exact",
        "time_period": null,
        "evidence": {
          "paragraph_id": "p0039",
          "quote": "Commercial ship traffic dropped by 70 percent in the Strait of Hormuz in response to the United States and Israel’s combined strike campaign on Iran, according to ship tracking platform MarineTraffic."
        }
      },
      {
        "metric": "share of global oil supply passing through Strait of Hormuz",
        "value": 20.0,
        "unit": "percent",
        "qualifier": "exact",
        "time_period": null,
        "evidence": {
          "paragraph_id": "p0039",
          "quote": "Twenty percent of the global oil supply passes through the strait."
        }
      }
    ],
    "evidence": [
      {
        "paragraph_id": "p0039",
        "quote": "Commercial ship traffic dropped by 70 percent in the Strait of Hormuz in response to the United States and Israel’s combined strike campaign on Iran, according to ship tracking platform MarineTraffic."
      },
      {
        "paragraph_id": "p0039",
        "quote": "Twenty percent of the global oil supply passes through the strait."
      }
    ],
    "impacts": [
      {
        "channel": "shipping_freight",
        "direction": "up",
        "horizon": "weeks",
        "materiality": "high",
        "basis": "analyst_inference",
        "rationale": "A reported 70 percent drop in commercial transits through a chokepoint carrying about a fifth of global oil supply could raise freight and insurance costs and tighten energy logistics, conditional on the disruption persisting."
      },
      {
        "channel": "energy_supply",
        "direction": "up",
        "horizon": "weeks",
        "materiality": "medium",
        "basis": "analyst_inference",
        "rationale": "Reduced tanker transits through Hormuz could constrain crude flows and support oil prices if shippers continue avoiding the strait; the passage does not quantify price effects."
      }
    ],
    "confidence": "medium",
    "uncertainties": [
      "Traffic figure is attributed to MarineTraffic via the report, not independently observed.",
      "No event date is given for the traffic drop; article publication date is not used as event date.",
      "Duration and persistence of the disruption are unknown."
    ]
  },
  "ai_review": {
    "event_id": "2026-02-28-iran-update-evening-special-report-february-28-2026-e4dfe81f508d:e002",
    "article_id": "2026-02-28-iran-update-evening-special-report-february-28-2026-e4dfe81f508d",
    "event_index": 2,
    "raw_event_sha256": "281a5b08f671c22ff44bdb77c525572a72117d21060b0ad291df528f1a1ef31f",
    "disposition": "include_with_annotation",
    "issues": [
      "quantity_context"
    ],
    "annotation": "Faithful MarineTraffic-attributed traffic claim. The 70% figure concerns commercial ship traffic with no baseline/window specified; the 20% oil-supply share is background exposure, not a measured 20% supply loss. No realised price change follows from these numbers.",
    "overrides": {},
    "source_paragraph_ids": [
      "p0039"
    ],
    "checks": {
      "relevance": true,
      "quotation": true,
      "attribution": true,
      "event_type": true,
      "event_date": true,
      "quantities": true,
      "full_cited_paragraph_context": true
    }
  },
  "human_review_status": "unreviewed"
}