Replies: 30 comments 19 replies
-
Statement StitchingAn example of my prompts to get statement stitching to work The whole point about stitching statements is to get a longer series of data over time. So when we stitch the 2024, 2023, and 2022 statements together we should see columns 2024, 2023, 2022, 2021, Claude Code (AI)✻ Thinking… This request is asking me to rethink the purpose and implementation of statement stitching in the XBRL2 module. Let me break down the key points:
This is a more comprehensive view of statement stitching than what was implemented in the current fix. The current fix just addressed the immediate bug by replacing get_statement_data with The user wants me to think about the purpose and what users want from stitching, not necessarily to implement all of these changes right now. ⏺ You're absolutely right about the purpose of statement stitching. Let me rethink this from a user perspective: Statement stitching should create a comprehensive multi-period view across multiple filings, focusing on:
The render functionality is important but secondary - we should focus on getting the data structure right first, then worry about displaying it effectively. The stitching process needs to handle all periods across all statements, not just the most recent periods from each statement. This is especially valuable for analyzing long-term trends, particularly |
Beta Was this translation helpful? Give feedback.
-
Rendering of Stitched StatementsA couple more prompts to tidy up the rendering but it is rendering multiple years now. |
Beta Was this translation helpful? Give feedback.
-
|
@dgunning this looks good Dwight! |
Beta Was this translation helpful? Give feedback.
-
New XBRL2 API for stitched statementsfrom edgar.xbrl2 import *
from edgar import *
c = Company("AAPL")
filings = c.latest("10-K", 4)
xbrls = XBRLS.from_filings(filings)
xbrls.statements.income_statement() |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight, I am trying to run this but I am getting the below response. Is this up yet and working? Please let me know. Thank you. Traceback (most recent call last): Cell In[16], line 1 NameError: name 'XBRLS' is not defined from edgar import company, set_identity c = Company("AAPL") Cell In[17], line 8 NameError: name 'XBRLS' is not defined |
Beta Was this translation helpful? Give feedback.
-
|
Just released 3.13.1. Please see the notes https://github.com/dgunning/edgartools/blob/main/edgar/xbrl2/README.md |
Beta Was this translation helpful? Give feedback.
-
|
Dwight can we not export to excel? I keep getting this error: Traceback (most recent call last): Cell In[26], line 41 Cell In[26], line 19 in export_to_excel AttributeError: 'StitchedStatement' object has no attribute 'to_pandas' Please let me know your thoughts. Thank you. David |
Beta Was this translation helpful? Give feedback.
-
|
This should work income_df = xbrls.statements.income_statement().to_dataframe()
balance_df = xbrls.statements.balance_sheet().to_dataframe()
cash_flow_df = xbrls.statements.cash_flow_statement().to_dataframe()I realized there is a |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight thank you for your hard work. How do we get these two to look identical? (but the dates). Meaning when we go from single to multi pull it goes haywire. Here is the basic which is perfect. Income Statement: Line Item Sep 28, 2024 Sep 30, 2023 Sep 24, 2022 But when we change the code to this: from edgar.xbrl2 import * c = Company("AAPL") It looks like the below. Granted I am using Spyder so it's not going to come out correctly but even when I transfer it to excel a ton of data is still missing. You are very close. How do we get all the data together? People can build trading strategies when the XBRL data is complete and comprehensive like this. This would be an alternative to FactSet almost. Line Sep 28, Sep 30, Sep 24, Sep 25, Sep 26, 28, Sep 29, |
Beta Was this translation helpful? Give feedback.
-
|
Working on this now. I was focusing on which columns are selected in stitched statements and now I'll turn attention to the rows. Should be in the next release - maybe tomorrow |
Beta Was this translation helpful? Give feedback.
-
|
Released 3.13.2 with fixes for these [3.13.2] - 2025-03-14Changed
|
Beta Was this translation helpful? Give feedback.
-
|
Thx Dwight! Will this fix the issues with the multiple years? |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight I updated the release. I am getting the below output error now. Any thoughts? Traceback (most recent call last): Cell In[6], line 71 Cell In[6], line 49 in export_to_excel Cell In[6], line 38 in statement_to_dataframe AttributeError: Unable to convert statement to DataFrame. Please consult the documentation. |
Beta Was this translation helpful? Give feedback.
-
|
Dwight is the library working correctly? |
Beta Was this translation helpful? Give feedback.
-
|
Yes. Your code is probably wrong. See this notebook with statement.to_dataframe examples https://github.com/dgunning/edgartools/blob/main/notebooks/XBRL2-Rewrite-of-XBRL.ipynb |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight so I am trying to use this multi statement for XBRL but I am not getting all the data. Below is the code I am using I also included a snippet of the excel file. Is there something I am doing wrong?
Code: from edgar.xbrl2 import * Set pandas display options (optional)pd.set_option("display.max_columns", None) def export_to_excel(xbrls, file_path, max_periods=9): Create a Company object and retrieve filingsc = Company("AAPL") Define the file path in your Documents folderdocuments_folder = os.path.join(os.path.expanduser("~"), "Documents") Export the stitched statements to Excel using the new featuresexport_to_excel(xbrls, output_file) |
Beta Was this translation helpful? Give feedback.
-
|
Good morning, Dwight. I hope you are doing well. Can you give me your thoughts here? Does this look correct?
|
Beta Was this translation helpful? Give feedback.
-
Released 3.13.3 with some minor updates to XBRL2Some updates to the notebook XBRL2-Rewrite-of-XBRL.ipynb [3.13.3] - 2025-03-15Changed
|
Beta Was this translation helpful? Give feedback.
-
|
Hello Dwight, I take a look to your code but no trace of a XBRLS class ...! Maybe I made a mistake somewhere? |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight I just tried running this again with the update from 48 minutes ago. I tried getting nine years using the code below and here are the results: Balance Sheet: Cash Flow Statement: CODE: Set pandas display options (optional)pd.set_option("display.max_columns", None) Create a Company object and retrieve filings (e.g., the latest 9 10-K filings)c = Company("AAPL") Stitch statements across multiple filings for a multi-year comparison.The max_periods parameter sets how many periods (e.g., years) are compared.income_trend = xbrls.statements.income_statement(max_periods=9) Convert each stitched statement to a pandas DataFrame using the new to_dataframe() method.df_income = income_trend.to_dataframe() Define the file path in your Documents folder and export the DataFrames to an Excel file.documents_folder = os.path.join(os.path.expanduser("~"), "Documents") with pd.ExcelWriter(output_file) as writer: print(f"Export completed. Stitched statements saved to: {output_file}") |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for testing. I'll take a look later today |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight, any news on this? A new release or anything coming soon? Thank you. |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight just wanted to touch base. Is this being looked at still? I hope you are doing well. @dgunning David |
Beta Was this translation helpful? Give feedback.
-
Release 3.13.4 - 2025-03-18Added
Changed
Added documentation on Facts to Added Fact Queries notebook |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight, when trying to run your example "Multi-Period Analysis with XBRLS" : I have this ImportError: cannot import name 'XBRLS' from 'edgar.xbrl2.xbrl' ...? |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight, Good afternoon. I hope you are doing well. I tried running the updated code but I am getting the below error. Can you let me know when this is up and running? I also included the code I used below as well. Thank you! Traceback (most recent call last): Cell In[5], line 2 ImportError: cannot import name 'XBRLS' from 'edgar.xbrl2.xbrl' \ Code: from edgar import Company Get multiple filings for trend analysis (last 3 annual reports)company = Company('AAPL') Create a stitched view across multiple filingsxbrls = XBRLS.from_filings(filings) Access stitched statementsstitched_statements = xbrls.statements Get multi-period statements for the income statement and balance sheetincome_trend = stitched_statements.income_statement() Convert stitched statements to DataFrames using the new to_dataframe() methodincome_df = income_trend.to_dataframe() Define the output file path (saving in the current directory)output_file = "AAPL_Stitched_Statements.xlsx" Export the DataFrames to an Excel file with separate sheetswith pd.ExcelWriter(output_file) as writer: print(f"Export completed. Excel file saved as: {output_file}") |
Beta Was this translation helpful? Give feedback.
-
|
That should be from |
Beta Was this translation helpful? Give feedback.
-
|
More precisely: from edgar.xbrl2.stitching import XBRLS .. thank's |
Beta Was this translation helpful? Give feedback.
-
|
Hey Dwight, I just updated the code will the financial labels be in order eventually and when pulling multiple years there are still years missing.
|
Beta Was this translation helpful? Give feedback.
-
Release 3.14.5 [3.13.5] - 2025-03-19Fixed
|
Beta Was this translation helpful? Give feedback.
-
|
Hi, I'm wondering if it wouldn't be a good idea to use values to identify different concepts. Let me explain: |
Beta Was this translation helpful? Give feedback.
-
|
in _integrate_statement_data, you refer to the label to list the corresponding concepts, that seems ok (if label don't change) ...in this case why worry about the concept? |
Beta Was this translation helpful? Give feedback.
-
|
Good points. Before I implement each feature I ask Claude to generate a design. Here is the design for Statement Stitching https://github.com/dgunning/edgartools/blob/main/edgar/xbrl2/docs/stitching-statements.md As you see it uses StandardConcepts to help in the stitching, and it does so because I suggested it when prompting for the design. Your logic of stitching by value is likely also valuable but did not occur to me at the time. Maybe we can incorporate it. I want to maintain not just the code but the design docs of a great XBRL parsing system |
Beta Was this translation helpful? Give feedback.
-
Release 3.13.6 - 2025-03-20Fixed
Added
|
Beta Was this translation helpful? Give feedback.
-
|
Hi, as David08-08, impossible to get IncomeStatement for duration_2018-09-30_2019-09-28 (and before) it seems to be a parsing problem, because only 2 items ('concept': 'us-gaap_AllocatedShareBasedCompensationExpense' and 'concept': 'us-gaap_EmployeeServiceShareBasedCompensationTaxBenefitFromCompensationExpense') are visible in stitching/_integrate_statement_data for this period... and should be in cash flow ? |
Beta Was this translation helpful? Give feedback.
-
|
when looking at stitching/ stitch_statements for duration_2018-09-30_2019-09-28 we can see a wrong filtered_statement: |
Beta Was this translation helpful? Give feedback.
-
|
it seems that the problem is in XBRL / get_statement_by_type where it take the first statement matching the given type... |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for checking. This is fixed now, I'm just testing. There was a bit of code that did in |
Beta Was this translation helpful? Give feedback.
-
|
Fixed in 3.13.9 
|
Beta Was this translation helpful? Give feedback.
-
|
Nice, thank's a lot. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, why using "max_periods" ? when requesting filings we ask for a number but stitching cut this number after (if i understand) with max_periods . I think there is a reason, but i don't see why, could you explain ? |
Beta Was this translation helpful? Give feedback.
-
|
Good point. I think we can remove XBRL2 is intended to replace Financials and MultiFinancials |
Beta Was this translation helpful? Give feedback.
-
|
How many statements would you stitch? 10? 20? Do you want to remove |
Beta Was this translation helpful? Give feedback.
-
|
I think the important point is the linearity and integrity of the data, For example if for a ticker we can get 2024 to 2000 but with a gap in 2004 where the data is incomplete or unusable, in my opinion I prefer limiting the results to 2024 => 2004 (better for charting and statistics). Warren Buffet said 10 years of analyzes is a minimum ... so ! |
Beta Was this translation helpful? Give feedback.
-
|
I am going to change What do you mean by date columns - you mean the column headers e.g. 'Sep 30, 2024'? What is standard pandas format? |
Beta Was this translation helpful? Give feedback.
-
|
Thank for the changes, I will test when I find a moment. The column headers are str format. By "standard date format" I mean for example pd.Timestamp witch is easily convertible in python.datetime : cf ( https://pandas.pydata.org/docs/reference/api/pandas.Timestamp.html ) and used in pd.DatetimeIndex ... |
Beta Was this translation helpful? Give feedback.
-
|
This release includes major refactoring to improve make the code more maintainable, major performance improvements, and rendering of segmented statements. There are a lot of code changes so good change I've broken something. So the next few releases will be quality focused [3.13.10] - 2025-03-25Added
Changed
|
Beta Was this translation helpful? Give feedback.
-
|
We should plan the switchover from XBRL to XBRL2. Here is what I am thinking Cutover Plan
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I'm grateful for the interest in edgartools and I like that the community finds it useful.
One area of concern is the number of open issues that have to to with XBRL and Financials. At the moment there are 12 open issues relating to XBRL and Financials which is half of the current open issues. This is mostly due to the unbelievable complexity of the XBRL spec and how thousands of companies report financial data, but also due to me having to both be an expert in the area as well as an expert in coding, and also due to some naive decisions I made in the current XBRL implementation.
In order to get things right I'm taking advantage of the rapid improvements in AI capability and specifically Claude Code released 2 weeks ago. In short - it is amazing - I used it to rewrite Insider Transactions over one weekend. I also pay for and use Windsurf AI (similar to Cursor) but Claude Code is better.
Now I decided to use it to rewrite XBRL financials and test and improve it over the course of a few weeks until it becomes good enough to replace the current features. The current XBRL issues will be fixed on the new XBRL2
You are invited to help test the new functionality. It's rough around the edges but I'm actively making changes and will take your suggestions
Also Claude Code is very good but very expensive. The Insider Transaction rewrite cost maybe $7 in Anthropic credits and so far XBRL rewrite being larger probably $25 with some more way to go. (Plus being Canadian I factor in the exchange rate and being culturally more miserly) . This is a lot to spend on a tool released 2 weeks ago. $10 in Anthropic credits goes a long way to speeding things up so you could But Me a Coffee which would speed things up and improve the quality and features.
Thanks,
Dwight
Beta Was this translation helpful? Give feedback.
All reactions