fix(xbrl): an amended annual report is still an annual report; cash flow gets its named period views (#1311)

* fix(xbrl): an amended annual report is still an annual report; cash flow gets its named period views

GH #1226 (edgartools-osrt) — entity_info classified the report by comparing the
whole dei:DocumentType against '10-K' and '10-Q', while testing the amendment
with a substring. A 10-K/A therefore failed both report tests and passed the
amendment one, so Shopify's amended annual report came back amendment=True with
annual_report=False — a classification that contradicts itself and the filing,
which tags dei:DocumentAnnualReport true.

The form list this decision needs already existed as its own tuple in
period_selector.py (annual_form_types, /A variants included): one rule, two
implementations, and the copy that never set the flag was the correct one. Both
now call edgar.xbrl.core.is_annual_document_type and its siblings, which strip
the suffix before classifying.

That widens annual_report to 20-F, 40-F, 10-KT, 10-KSB and 11-K, where only
'10-K' set it before. period_selector is the only consumer of the flag and it
already ORed those forms in through its own tuple, so period selection does not
move: entity_info is identical across all eight directories in
data/xbrl/datafiles.

GH #1253 (edgartools-qao5) — get_period_views("CashFlowStatement") returned an
empty list because the statement type had no entry in STATEMENT_TYPE_CONFIG, so
the named views every other primary statement offers were unreachable and
to_dataframe(period_view=...) had no name to accept.
determine_periods_to_display already handles cash flow and income statements in
one branch, so the entry mirrors the income statement's. Across all eight
fixtures the cash-flow view list now equals the income-statement view list,
including the two where both are legitimately empty.

hatch run test-fast: 7,217 passed.

Fixes #1226
Fixes #1253

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test(1253): read the fixture list from the checkout, not from a hand-written list

data/xbrl/datafiles/aes is present locally and untracked, so naming the eight
directories by hand passed here and failed CI with "Directory not found". The
sweep now takes the directories that actually carry a presentation linkbase,
and asserts there are at least five of them so it cannot quietly measure
nothing — which is the failure mode an untracked fixture usually produces.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: dgunning

changelog.d/1226.fixed.md

@@ -0,0 +1 @@
+**An amended annual report was not classified as an annual report.** `entity_info` compared the whole `dei:DocumentType` against `10-K`, so Shopify's 10-K/A reported `amendment=True` alongside `annual_report=False`, contradicting the filing's own `dei:DocumentAnnualReport`. The amendment suffix is now stripped before classifying, and 20-F/40-F annual reports are recognised too. (GH #1226)

changelog.d/1253.added.md

@@ -0,0 +1 @@
+**Named period views for cash flow statements.** `get_period_views("CashFlowStatement")` returned an empty list because the statement type had no entry in the period-view configuration, so `to_dataframe(period_view=...)` had no name to accept. Cash flow now offers the same named views as the income statement, which selects from the same periods. (GH #1253)

edgar/xbrl/core.py

@@ -572,6 +572,39 @@ def is_point_in_time(period_type: Optional[str]) -> Optional[bool]:
     return period_type == 'instant'
 
 
+#: Document types that are annual reports, without the amendment suffix.
+ANNUAL_REPORT_FORMS = frozenset({'10-K', '10-KT', '10-KSB', '20-F', '40-F', '11-K'})
+
+#: Document types that are quarterly reports, without the amendment suffix.
+QUARTERLY_REPORT_FORMS = frozenset({'10-Q', '10-QT', '10-QSB'})
+
+
+def base_document_type(document_type: Optional[str]) -> str:
+    """The document type with its amendment suffix removed: '10-K/A' -> '10-K'."""
+    if not document_type:
+        return ''
+    return document_type.split('/')[0].strip().upper()
+
+
+def is_amendment_document_type(document_type: Optional[str]) -> bool:
+    """Whether a dei:DocumentType carries the amendment suffix."""
+    return bool(document_type) and '/A' in document_type.upper()
+
+
+def is_annual_document_type(document_type: Optional[str]) -> bool:
+    """Whether a dei:DocumentType is an annual report, amended or not.
+
+    An amended annual report is still an annual report. Comparing the whole
+    string against '10-K' made Shopify's 10-K/A report amendment=True alongside
+    annual_report=False, which contradicts its own dei:DocumentAnnualReport
+    (GH #1226).
+    """
+    return base_document_type(document_type) in ANNUAL_REPORT_FORMS
+
+
+def is_quarterly_document_type(document_type: Optional[str]) -> bool:
+    """Whether a dei:DocumentType is a quarterly report, amended or not."""
+    return base_document_type(document_type) in QUARTERLY_REPORT_FORMS
 #: The XBRL ``decimals`` sentinel for a value reported exactly, to unlimited
 #: precision. It is not a number and is never interchangeable with 0, which
 #: says the value is rounded to the unit.

edgar/xbrl/parsers/instance.py

@@ -12,7 +12,14 @@
 from lxml import etree as ET
 
 from edgar.core import log
-from edgar.xbrl.core import NAMESPACES, classify_duration, duration_days
+from edgar.xbrl.core import (
+    NAMESPACES,
+    classify_duration,
+    duration_days,
+    is_amendment_document_type,
+    is_annual_document_type,
+    is_quarterly_document_type,
+)
 from edgar.xbrl.models import Context, Fact, XBRLProcessingError
 
 from .base import BaseParser
@@ -838,11 +845,13 @@ def get_dei(*names):
                 except Exception:
                     pass
 
-            # Flags based on document_type
+            # Flags based on document_type. An amended annual report is still
+            # an annual report, so the suffix is stripped before classifying
+            # rather than compared away (GH #1226).
             dt_val = self.entity_info['document_type'] or ''
-            self.entity_info['annual_report']    = (dt_val == '10-K')
-            self.entity_info['quarterly_report'] = (dt_val == '10-Q')
-            self.entity_info['amendment']        = ('/A' in dt_val)
+            self.entity_info['annual_report']    = is_annual_document_type(dt_val)
+            self.entity_info['quarterly_report'] = is_quarterly_document_type(dt_val)
+            self.entity_info['amendment']        = is_amendment_document_type(dt_val)
 
             log.debug(f"Entity info: {self.entity_info}")
         except Exception as e:

edgar/xbrl/period_selector.py

@@ -14,7 +14,7 @@
 from datetime import date, datetime
 from typing import Any, Dict, List, Optional, Tuple
 
-from edgar.xbrl.core import duration_days
+from edgar.xbrl.core import duration_days, is_annual_document_type
 
 logger = logging.getLogger(__name__)
 
@@ -257,15 +257,12 @@ def _select_duration_periods(periods: List[Dict], entity_info: Dict[str, Any], m
     # Some filings (like GE 2015 10-K) report fiscal_period='Q4' even for annual reports
     is_annual_report = entity_info.get('annual_report', False)
     document_type = entity_info.get('document_type', '')
-    annual_form_types = (
-        '10-K', '10-K/A', '10-KT', '10-KT/A',  # Standard and transition annual reports
-        '10-KSB', '10-KSB/A',                   # Small business (legacy)
-        '20-F', '20-F/A',                       # Foreign private issuers
-        '40-F', '40-F/A',                       # Canadian issuers
-    )
-
-    # Consider it annual if: fiscal_period == 'FY' OR it's flagged as annual OR it's an annual form type
-    is_annual = fiscal_period == 'FY' or is_annual_report or document_type in annual_form_types
+
+    # Consider it annual if: fiscal_period == 'FY' OR it's flagged as annual OR it's an annual form type.
+    # The form list used to live here as its own tuple, which is the same rule
+    # entity_info's annual_report flag is derived from (GH #1226).
+    is_annual = (fiscal_period == 'FY' or is_annual_report
+                 or is_annual_document_type(document_type))
 
     # Filter for annual periods if this is an annual report
     if is_annual:

edgar/xbrl/periods.py

@@ -58,6 +58,31 @@
             }
         ]
     },
+    # A cash flow statement selects the same duration periods an income
+    # statement does (determine_periods_to_display handles the two together),
+    # but it had no entry here, so get_period_views() returned [] for it and
+    # the named views were unreachable through to_dataframe(period_view=...)
+    # (GH #1253).
+    'CashFlowStatement': {
+        'period_type': 'duration',
+        'max_periods': 3,
+        'allow_annual_comparison': True,
+        'views': [
+            {
+                'name': 'Three Recent Periods',
+                'description': 'Shows three most recent reporting periods',
+                'max_periods': 3,
+                'requires_min_periods': 3
+            },
+            {
+                'name': 'YTD and Quarterly Breakdown',
+                'description': 'Shows YTD figures and quarterly breakdown',
+                'max_periods': 5,
+                'requires_min_periods': 2,
+                'mixed_view': True
+            }
+        ]
+    },
     'StatementOfEquity': {
         'period_type': 'duration',
         'max_periods': 3,

tests/issues/regression/test_issue_1226_amended_annual_report.py

@@ -0,0 +1,92 @@
+"""Regression test for issue #1226.
+
+GitHub Issue: https://github.com/dgunning/edgartools/issues/1226
+
+`XBRL.entity_info` classified the report by comparing the whole
+`dei:DocumentType` against `'10-K'`, so Shopify's 10-K/A came back with
+`amendment=True` alongside `annual_report=False` — a classification that
+contradicts itself and the filing, which tags `dei:DocumentAnnualReport` true.
+
+An amended annual report is still an annual report. The suffix is now stripped
+before classifying, and the form list `period_selector` kept as its own tuple
+for the same decision is now the one both use
+(`edgar.xbrl.core.is_annual_document_type`).
+"""
+
+import pytest
+
+from edgar.xbrl.core import (
+    is_amendment_document_type,
+    is_annual_document_type,
+    is_quarterly_document_type,
+)
+from edgar.xbrl.parsers import XBRLParser
+
+INSTANCE = """<?xml version="1.0" encoding="UTF-8"?>
+<xbrl xmlns="http://www.xbrl.org/2003/instance"
+      xmlns:xbrli="http://www.xbrl.org/2003/instance"
+      xmlns:dei="http://xbrl.sec.gov/dei/2024"
+      xmlns:xlink="http://www.w3.org/1999/xlink">
+  <context id="c1">
+    <entity><identifier scheme="http://www.sec.gov/CIK">0001594805</identifier></entity>
+    <period><startDate>2024-01-01</startDate><endDate>2024-12-31</endDate></period>
+  </context>
+  <dei:DocumentType contextRef="c1">{document_type}</dei:DocumentType>
+  <dei:AmendmentFlag contextRef="c1">true</dei:AmendmentFlag>
+  <dei:DocumentAnnualReport contextRef="c1">true</dei:DocumentAnnualReport>
+  <dei:DocumentFiscalPeriodFocus contextRef="c1">FY</dei:DocumentFiscalPeriodFocus>
+  <dei:DocumentFiscalYearFocus contextRef="c1">2024</dei:DocumentFiscalYearFocus>
+  <dei:EntityRegistrantName contextRef="c1">SHOPIFY INC.</dei:EntityRegistrantName>
+</xbrl>
+"""
+
+
+def _entity_info(document_type):
+    parser = XBRLParser()
+    parser.parse_instance_content(INSTANCE.format(document_type=document_type))
+    return parser.entity_info
+
+
+def test_amended_annual_report_keeps_both_halves_of_its_identity():
+    """The report's own case: Shopify's 10-K/A, accession 0001594805-25-000039."""
+    info = _entity_info("10-K/A")
+
+    assert info["entity_name"] == "SHOPIFY INC."
+    assert info["document_type"] == "10-K/A"
+    assert info["annual_report"] is True
+    assert info["amendment"] is True
+    assert info["quarterly_report"] is False
+
+
+def test_an_unamended_annual_report_is_unchanged():
+    info = _entity_info("10-K")
+
+    assert info["annual_report"] is True
+    assert info["amendment"] is False
+
+
+def test_an_amended_quarterly_report_is_still_quarterly():
+    info = _entity_info("10-Q/A")
+
+    assert info["quarterly_report"] is True
+    assert info["amendment"] is True
+    assert info["annual_report"] is False
+
+
+@pytest.mark.parametrize("document_type,annual,quarterly,amendment", [
+    ("10-K", True, False, False),
+    ("10-K/A", True, False, True),
+    ("10-KT", True, False, False),
+    ("20-F", True, False, False),
+    ("20-F/A", True, False, True),
+    ("40-F", True, False, False),
+    ("10-Q", False, True, False),
+    ("10-Q/A", False, True, True),
+    ("8-K", False, False, False),
+    ("", False, False, False),
+    (None, False, False, False),
+])
+def test_document_type_classification(document_type, annual, quarterly, amendment):
+    assert is_annual_document_type(document_type) is annual
+    assert is_quarterly_document_type(document_type) is quarterly
+    assert is_amendment_document_type(document_type) is amendment

tests/issues/regression/test_issue_1253_cashflow_period_views.py

@@ -0,0 +1,75 @@
+"""Regression test for issue #1253.
+
+GitHub Issue: https://github.com/dgunning/edgartools/issues/1253
+
+`get_period_views("CashFlowStatement")` returned `[]` even for a filing whose
+cash flow statement had been resolved and whose ordinary period selector had
+already chosen comparable duration columns. `CashFlowStatement` was simply
+absent from `STATEMENT_TYPE_CONFIG`, so the named views other primary
+statements offer were unreachable, and `to_dataframe(period_view=...)` had no
+name to accept.
+
+`determine_periods_to_display` already handles cash flow and income statements
+in one branch, so the entry mirrors the income statement's: duration periods,
+three of them, with the same two named views.
+"""
+
+from pathlib import Path
+
+import pytest
+
+from edgar.xbrl import XBRL
+
+DATA = Path(__file__).resolve().parents[3] / "data" / "xbrl" / "datafiles"
+
+# Not every directory under data/xbrl/datafiles is committed -- `aes` is
+# present locally and untracked, so naming the eight by hand passed here and
+# failed in CI. Read what the checkout actually has, and assert below that it
+# is enough for the sweep to mean something.
+FIXTURES = sorted(path.name for path in DATA.iterdir()
+                  if path.is_dir() and any(path.glob("*_pre.xml")))
+
+
+def test_the_fixture_sweep_is_not_empty():
+    assert len(FIXTURES) >= 5, f"too few committed XBRL fixtures to sweep: {FIXTURES}"
+
+
+@pytest.mark.parametrize("fixture", FIXTURES)
+def test_cash_flow_offers_the_same_named_views_as_the_income_statement(fixture):
+    """The two statements select from the same periods, so they must agree."""
+    xbrl = XBRL.from_directory(DATA / fixture)
+
+    income = [view["name"] for view in xbrl.get_period_views("IncomeStatement")]
+    cash_flow = [view["name"] for view in xbrl.get_period_views("CashFlowStatement")]
+
+    assert cash_flow == income
+
+
+def test_a_named_view_can_be_passed_to_to_dataframe():
+    xbrl = XBRL.from_directory(DATA / "aapl")
+
+    views = xbrl.get_period_views("CashFlowStatement")
+    assert views, "expected named cash-flow views for the AAPL fixture"
+    assert "Three Recent Periods" in [view["name"] for view in views]
+
+    statement = xbrl.statements.cash_flow_statement()
+    frame = statement.to_dataframe(period_view=views[0]["name"])
+
+    period_columns = [column for column in frame.columns if "20" in str(column)]
+    assert period_columns, "the named view produced no period columns"
+    assert len(period_columns) <= views[0].get("max_periods", 3) + 1
+
+    # A view is only useful if the numbers come with it.
+    operating = frame.loc[frame["concept"] == "us-gaap_NetCashProvidedByUsedInOperatingActivities"]
+    assert not operating.empty
+    assert operating[period_columns].notna().any().any()
+
+
+def test_every_view_reports_the_period_keys_it_selected():
+    xbrl = XBRL.from_directory(DATA / "aapl")
+
+    for view in xbrl.get_period_views("CashFlowStatement"):
+        assert view["name"]
+        assert view["description"]
+        assert view["period_keys"], f"{view['name']} selected no periods"
+        assert all(key.startswith("duration_") for key in view["period_keys"])