Skip to content

[DataGridPremium] Nested Lazy Loading#21043

Open
MBilalShafi wants to merge 28 commits intomui:masterfrom
MBilalShafi:nested-lazy-loading
Open

[DataGridPremium] Nested Lazy Loading#21043
MBilalShafi wants to merge 28 commits intomui:masterfrom
MBilalShafi:nested-lazy-loading

Conversation

@MBilalShafi
Copy link
Copy Markdown
Member

@MBilalShafi MBilalShafi commented Jan 23, 2026

Fixes #14527

Summary

This PR adds server-side nested lazy loading for Data Grid Pro/Premium. Lazy loading now works with tree data and row grouping by fetching visible root and child skeleton ranges through the Data Source layer.

It also adds support for defaultGroupingExpansionDepth and isGroupExpandedByDefault in nested lazy loading, including groups loaded later while scrolling.

Motivation

Flat lazy loading could fetch root rows as they entered the viewport, but nested data needs parent-scoped ranges. Without a dedicated strategy, tree data and row grouping could not lazily fetch descendants,
revalidation could not target the right parent, and collapsing expanded parents could try to delete skeleton rows through updateRows().

Changes

  • Added the LazyLoadedGroupedData data source strategy.
  • Added useGridDataSourceNestedLazyLoader for tree data and row grouping.
  • Added nested skeleton row handling and parent-scoped fetches.
  • Added row grouping support using groupFields, group paths, and grouping fields.
  • Added automatic expansion support for defaultGroupingExpansionDepth and isGroupExpandedByDefault.
  • Added follow-up visible skeleton scanning after auto expansion, including rows loaded later by scroll.
  • Added nested revalidation and polling support with dataSourceRevalidateMs.
  • Added showChildrenLoading to suppress child loading indicators during background revalidation.
  • Preserved nested selection/expansion when revalidating same-id rows.
  • Replaced changed-id rows under the correct parent.
  • Fixed collapse cleanup so skeleton rows are regenerated without calling updateRows() with invalid delete rows.
  • Added explicit errors for unsupported unknown root/children counts.

Docs

  • Added Nested lazy loading demos for tree data and row grouping.
  • Added Group expansion demos for tree data and row grouping.
  • Added dynamic revalidation demos for plain data, tree data, and row grouping.
  • Split tree data and row grouping revalidation into separate examples.

Behavior Notes

  • Nested lazy loading supports viewport lazy loading, not infinite loading.
  • The root rowCount and each parent getChildrenCount() must be known.
  • skipFallbackRevalidation only applies to the extra scan after automatic expansion; normal viewport and polling revalidation still run.

Tests

  • Added Pro browser coverage for nested tree data lazy loading, default expansion, isGroupExpandedByDefault, scroll-loaded groups, nested polling, cache-aware polling, same-id updates, changed-id replacement, and
    collapse behavior.
  • Added Premium browser coverage for row grouping lazy loading, grouping fields, final-level leaves, default expansion, isGroupExpandedByDefault, and collapse behavior.

Preview

https://deploy-preview-21043--material-ui-x.netlify.app/x/react-data-grid/server-side-data/lazy-loading/#nested-lazy-loading

In Progress

  • Documentation improvements
  • Settle on cache invalidation strategy: The current approach that is being followed in the lazy loading could not be used here because in case of expanded group rows, invalidating everything outside viewport would cause them to collapse resulting in flickers
  • Support defaultGroupingExpansionDepth and isGroupExpandedByDefault
  • Test coverage
  • Edge cases

@MBilalShafi MBilalShafi added scope: data grid Changes related to the data grid. feature: Tree data Related to the data grid Tree data feature feature: Row grouping Related to the data grid Row grouping feature feature: Server integration Better integration with backends, e.g. data source labels Jan 23, 2026
@mui-bot
Copy link
Copy Markdown

mui-bot commented Jan 23, 2026

Deploy preview: https://deploy-preview-21043--material-ui-x.netlify.app/

Updated pages:

Bundle size report

Bundle size will be reported once CircleCI build #790195 finishes.

Generated by 🚫 dangerJS against 8b10e5a

@MBilalShafi MBilalShafi added the type: new feature Expand the scope of the product to solve a new problem. label Jan 23, 2026
@github-actions

This comment was marked as outdated.

@github-actions github-actions Bot added the PR: out-of-date The pull request has merge conflicts and can't be merged. label Jan 28, 2026
Resolved import conflicts by:
- Using 'type' keyword for type-only imports (consistent-type-imports rule)
- Preserving branch-specific imports needed for nested lazy loading feature

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged. label Jan 29, 2026
@mbiggs-gresham
Copy link
Copy Markdown

mbiggs-gresham commented Feb 10, 2026

I've been testing these changes in our product, and it seems pretty good as far as I can tell. We have lazy loaded groups with more fetched as you scroll around, along with lazy loaded children also adding more as you scroll around.

It would be nice to get this as it's a nice feature. 👍

@MBilalShafi
Copy link
Copy Markdown
Member Author

@mbiggs-gresham Thank you for the feedback.

I've been trying to implement a better cache invalidation strategy before we can proceed with this PR. I've just opened a PR for that matter (#21282) and as soon as it's merged, I plan to port the same to this PR and open it for review.

Thanks for your continued interest and patience. 🙏

@mbiggs-gresham
Copy link
Copy Markdown

That's great news, thanks for the update. We're really keen to get this feature.

One issue I have discovered is that defaultGroupingExpansionDepth={-1} doesn't expand the groups by default when the lazyLoading flag is set to true. Is this expected?

@github-actions github-actions Bot added the PR: out-of-date The pull request has merge conflicts and can't be merged. label Feb 24, 2026
@github-actions

This comment was marked as outdated.

@github-actions github-actions Bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged. label Feb 25, 2026
@diogoaoliveira
Copy link
Copy Markdown

Having an official nested lazy loading solution for tree data would be truly wonderful for us ! It would simplify our code a lot and make our implementation much more reliable in production.

Do you have any rough idea of when this might ship ? @MBilalShafi . Thanks for pushing this forward.

@code-infra-dashboard
Copy link
Copy Markdown

code-infra-dashboard Bot commented Apr 18, 2026

Deploy preview

Bundle size

Bundle Parsed size Gzip size
@mui/x-data-grid 🔺+303B(+0.07%) 🔺+85B(+0.07%)
@mui/x-data-grid-pro 🔺+10.3KB(+2.01%) 🔺+2.34KB(+1.57%)
@mui/x-data-grid-premium 🔺+10.7KB(+1.57%) 🔺+2.49KB(+1.26%)
@mui/x-charts 0B(0.00%) 0B(0.00%)
@mui/x-charts-pro 0B(0.00%) 0B(0.00%)
@mui/x-charts-premium 0B(0.00%) 0B(0.00%)
@mui/x-date-pickers 0B(0.00%) 0B(0.00%)
@mui/x-date-pickers-pro 0B(0.00%) 0B(0.00%)
@mui/x-tree-view 0B(0.00%) 0B(0.00%)
@mui/x-tree-view-pro 0B(0.00%) 0B(0.00%)

Details of bundle changes


Check out the code infra dashboard for more information about this PR.

@BrentFarese
Copy link
Copy Markdown

@MBilalShafi much more green 🤩! We've been struggling with our hand-rolled solution for a couple weeks. Gonna love an official solution. Thank you!

@oliviertassinari oliviertassinari added the plan: Premium Impact at least one Premium user. label Apr 23, 2026
@MBilalShafi MBilalShafi marked this pull request as ready for review May 1, 2026 10:39
@MBilalShafi MBilalShafi requested review from a team May 1, 2026 10:39
@MBilalShafi
Copy link
Copy Markdown
Member Author

The PR is now generally ready for review, I'll be keep polishing a few things here or there in the meantime.

CC @mui/grid @mui/cse

@mbiggs-gresham
Copy link
Copy Markdown

@MBilalShafi That's excellent news thanks. I've tried the latest version but seem to get an error.

We have a grid with only a depth of 1. If i expand the very first group, then scroll down (roughly 80 parent rows), then expand that 80th row, it fetches it's children, if i then collapse it the following occurs:

chunk-IENC7LCU.js?v=b5d4df48:29645 Uncaught TypeError: Cannot read properties of undefined (reading 'id')
    at handleTreeNode (chunk-IENC7LCU.js?v=b5d4df48:29645:56)
    at chunk-IENC7LCU.js?v=b5d4df48:29649:9
    at Array.forEach (<anonymous>)
    at handleTreeNode (chunk-IENC7LCU.js?v=b5d4df48:29647:21)
    at getVisibleRowsLookup2 (chunk-IENC7LCU.js?v=b5d4df48:29667:7)
    at Object.applyStrategyProcessor (chunk-IENC7LCU.js?v=b5d4df48:19118:12)
    at getVisibleRowsLookupState (chunk-IENC7LCU.js?v=b5d4df48:20558:25)
    at chunk-IENC7LCU.js?v=b5d4df48:20582:38
    at Object.setState (chunk-IENC7LCU.js?v=b5d4df48:19156:18)
    at Object.current (chunk-IENC7LCU.js?v=b5d4df48:20576:20)
    at EventManager.enhancedHandler (chunk-IENC7LCU.js?v=b5d4df48:3253:29)
    at EventManager.emit (chunk-IENC7LCU.js?v=b5d4df48:27301:18)
    at Object.publishEvent (chunk-IENC7LCU.js?v=b5d4df48:27394:40)
    at chunk-IENC7LCU.js?v=b5d4df48:35372:27
    at chunk-IENC7LCU.js?v=b5d4df48:35637:5
    at Object.current (chunk-IENC7LCU.js?v=b5d4df48:3946:5)
    at EventManager.enhancedHandler (chunk-IENC7LCU.js?v=b5d4df48:3253:29)
    at EventManager.emit (chunk-IENC7LCU.js?v=b5d4df48:27301:18)
    at Object.publishEvent (chunk-IENC7LCU.js?v=b5d4df48:27394:40)
    at Object.setRowChildrenExpansion (chunk-IENC7LCU.js?v=b5d4df48:23524:20)
    at handleClick (chunk-IENC7LCU.js?v=b5d4df48:33465:22)
    at handleClick (chunk-PTXFZRAC.js?v=b5d4df48:202:24)
    at executeDispatch (react-dom_client.js?v=4dc101c5:13622:11)
    at runWithFiberInDEV (react-dom_client.js?v=4dc101c5:997:72)
    at processDispatchQueue (react-dom_client.js?v=4dc101c5:13658:37)
    at react-dom_client.js?v=4dc101c5:14071:11
    at batchedUpdates$1 (react-dom_client.js?v=4dc101c5:2626:42)
    at dispatchEventForPluginEventSystem (react-dom_client.js?v=4dc101c5:13763:9)
    at dispatchEvent (react-dom_client.js?v=4dc101c5:16784:13)
    at dispatchDiscreteEvent (react-dom_client.js?v=4dc101c5:16765:62)
    at HTMLDivElement.sentryWrapped (@sentry_react.js?v=7d4b303b:14650:17)

This seems recent and I didn't get this previously. Is that stacktrace useful?

@MBilalShafi
Copy link
Copy Markdown
Member Author

MBilalShafi commented May 1, 2026

@MBilalShafi That's excellent news thanks. I've tried the latest version but seem to get an error.

We have a grid with only a depth of 1. If i expand the very first group, then scroll down (roughly 80 parent rows), then expand that 80th row, it fetches it's children, if i then collapse it the following occurs:

chunk-IENC7LCU.js?v=b5d4df48:29645 Uncaught TypeError: Cannot read properties of undefined (reading 'id')
    at handleTreeNode (chunk-IENC7LCU.js?v=b5d4df48:29645:56)
    at chunk-IENC7LCU.js?v=b5d4df48:29649:9
    at Array.forEach (<anonymous>)
    at handleTreeNode (chunk-IENC7LCU.js?v=b5d4df48:29647:21)
    at getVisibleRowsLookup2 (chunk-IENC7LCU.js?v=b5d4df48:29667:7)
    at Object.applyStrategyProcessor (chunk-IENC7LCU.js?v=b5d4df48:19118:12)
    at getVisibleRowsLookupState (chunk-IENC7LCU.js?v=b5d4df48:20558:25)
    at chunk-IENC7LCU.js?v=b5d4df48:20582:38
    at Object.setState (chunk-IENC7LCU.js?v=b5d4df48:19156:18)
    at Object.current (chunk-IENC7LCU.js?v=b5d4df48:20576:20)
    at EventManager.enhancedHandler (chunk-IENC7LCU.js?v=b5d4df48:3253:29)
    at EventManager.emit (chunk-IENC7LCU.js?v=b5d4df48:27301:18)
    at Object.publishEvent (chunk-IENC7LCU.js?v=b5d4df48:27394:40)
    at chunk-IENC7LCU.js?v=b5d4df48:35372:27
    at chunk-IENC7LCU.js?v=b5d4df48:35637:5
    at Object.current (chunk-IENC7LCU.js?v=b5d4df48:3946:5)
    at EventManager.enhancedHandler (chunk-IENC7LCU.js?v=b5d4df48:3253:29)
    at EventManager.emit (chunk-IENC7LCU.js?v=b5d4df48:27301:18)
    at Object.publishEvent (chunk-IENC7LCU.js?v=b5d4df48:27394:40)
    at Object.setRowChildrenExpansion (chunk-IENC7LCU.js?v=b5d4df48:23524:20)
    at handleClick (chunk-IENC7LCU.js?v=b5d4df48:33465:22)
    at handleClick (chunk-PTXFZRAC.js?v=b5d4df48:202:24)
    at executeDispatch (react-dom_client.js?v=4dc101c5:13622:11)
    at runWithFiberInDEV (react-dom_client.js?v=4dc101c5:997:72)
    at processDispatchQueue (react-dom_client.js?v=4dc101c5:13658:37)
    at react-dom_client.js?v=4dc101c5:14071:11
    at batchedUpdates$1 (react-dom_client.js?v=4dc101c5:2626:42)
    at dispatchEventForPluginEventSystem (react-dom_client.js?v=4dc101c5:13763:9)
    at dispatchEvent (react-dom_client.js?v=4dc101c5:16784:13)
    at dispatchDiscreteEvent (react-dom_client.js?v=4dc101c5:16765:62)
    at HTMLDivElement.sentryWrapped (@sentry_react.js?v=7d4b303b:14650:17)

This seems recent and I didn't get this previously. Is that stacktrace useful?

@mbiggs-gresham Thank you for the early feedback.
Would you mind adding a reproduction example based on the lastest built package? I tried to follow the mentioned flow but couldn't reproduce the issue.

Update: I added a defensive check on the place likely causing issue due to some outdated nodes. Would appreciate having a double check at your end.

@mbiggs-gresham
Copy link
Copy Markdown

mbiggs-gresham commented May 1, 2026

@MBilalShafi Sorry it's taken me a while to get a reproduction working with the bug, but have managed to do so here.

Feel free to clone the repo, run yarn install then yarn dev. Follow the on screen instructions.

I'm not sure i'd take what claude's analysis/workaround has said as necessarily correct.

I'll try your updated version too.

Updated version gives

chunk-7RWOS667.js?v=b51776c6:2514 Uncaught TypeError: Cannot read properties of undefined (reading 'type')
    at getTreeNodeDescendants (chunk-7RWOS667.js?v=b51776c6:2514:12)
    at getTreeNodeDescendants (chunk-7RWOS667.js?v=b51776c6:2526:30)
    at getTreeNodeDescendants (chunk-7RWOS667.js?v=b51776c6:2526:30)
    at chunk-7RWOS667.js?v=b51776c6:24617:25
    at Object.setState (chunk-7RWOS667.js?v=b51776c6:19156:18)
    at Object.current (chunk-7RWOS667.js?v=b51776c6:24612:20)
    at EventManager.enhancedHandler (chunk-7RWOS667.js?v=b51776c6:3253:29)
    at EventManager.emit (chunk-7RWOS667.js?v=b51776c6:27301:18)
    at Object.publishEvent (chunk-7RWOS667.js?v=b51776c6:27394:40)
    at chunk-7RWOS667.js?v=b51776c6:35375:27
    at chunk-7RWOS667.js?v=b51776c6:35640:5
    at Object.current (chunk-7RWOS667.js?v=b51776c6:3946:5)
    at EventManager.enhancedHandler (chunk-7RWOS667.js?v=b51776c6:3253:29)
    at EventManager.emit (chunk-7RWOS667.js?v=b51776c6:27301:18)
    at Object.publishEvent (chunk-7RWOS667.js?v=b51776c6:27394:40)
    at Object.setRowChildrenExpansion (chunk-7RWOS667.js?v=b51776c6:23524:20)
    at handleClick (chunk-7RWOS667.js?v=b51776c6:33468:22)
    at handleClick (chunk-PTXFZRAC.js?v=b51776c6:202:24)
    at executeDispatch (react-dom_client.js?v=dbc38bb0:13622:11)
    at runWithFiberInDEV (react-dom_client.js?v=dbc38bb0:997:72)
    at processDispatchQueue (react-dom_client.js?v=dbc38bb0:13658:37)
    at react-dom_client.js?v=dbc38bb0:14071:11
    at batchedUpdates$1 (react-dom_client.js?v=dbc38bb0:2626:42)
    at dispatchEventForPluginEventSystem (react-dom_client.js?v=dbc38bb0:13763:9)
    at dispatchEvent (react-dom_client.js?v=dbc38bb0:16784:13)
    at dispatchDiscreteEvent (react-dom_client.js?v=dbc38bb0:16765:62)
    at HTMLDivElement.sentryWrapped (@sentry_react.js?v=793c3867:14650:17)

@MBilalShafi
Copy link
Copy Markdown
Member Author

MBilalShafi commented May 1, 2026

@mbiggs-gresham Thank you for providing the reproduction example.

It seems that the custom cache being used in the demo is not using the cache keys properly.
I tried to comment it out and it seems to solve the issue: https://stackblitz.com/edit/fsh1mpjy

Can you check if the issue still reproduces for you in the attached StackBlitz example?

@mbiggs-gresham
Copy link
Copy Markdown

@MBilalShafi I think i've finally gotten to the bottom of this, and apologise for all the noise/confusion. It seems we had multiple groups/parents that had the same child rows under them and therefore the same id's. eg a child can be in multiple different groups.

It's been incredibly difficult to try to track the root cause down but also probably a good exercise. This didn't seem to be a problem before, so not sure what changed to cause this. I don't know if the grid would even be able to detect this and report it or if it's just a case of not providing it with duff data.

Our main app doesn't even have a custom datasource cache but the repro produced the same error we were seeing. Overall very happy with the new group grid, i guess we'll just have to try to generate more unique id's when a child is under multiple parents.


The demo below exhibits the behavior of the nested lazy loading with tree data.

{{"demo": "ServerSideTreeDataNestedLazyLoading.js", "bg": "inline"}}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like sorting breaks children expansions – both in tree view and row grouping.
Same with filtering.

Screen.Recording.2026-05-01.at.19.45.45.mov

isAutoGenerated: false,
groupingField: props.treeData ? null : (groupFields[parentPath.length] ?? null),
serverChildrenCount: childrenCount,
childrenFromPath: {},
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Accommodate the change of #22312 here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature: Row grouping Related to the data grid Row grouping feature feature: Server integration Better integration with backends, e.g. data source feature: Tree data Related to the data grid Tree data feature plan: Premium Impact at least one Premium user. scope: data grid Changes related to the data grid. type: new feature Expand the scope of the product to solve a new problem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[data grid] Implement server-side data source with nested data lazy loading

7 participants