Db v4 move graph folder by arienandalibi · Pull Request #2624 · Pometry/Raphtory

arienandalibi · 2026-05-27T08:19:26Z

What changes were proposed in this pull request?

Move graph_folder.rs constants (directory and file names regarding exported graphs) out of the raphtory crate and into raphtory-api.

Why are the changes needed?

Following previous changes, we needed access to GRAPH_META_PATH in pometry-storage, where we can't import from raphtory.

Does this PR introduce any user-facing change? If yes is this documented?

No

How was this patch tested?

Tests still pass

Are there any further changes required?

No

…storage crate.

…icate DIRTY_PATH in raphtory-graphql/src/data.rs

github-actions

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite	Current: `c7554f8`	Previous: `9823ef7`	Ratio
`lotr_graph/num_edges`	`4` ns/iter (`± 0`)	`0` ns/iter (`± 0`)	`+∞`
`lotr_graph/num_nodes`	`4` ns/iter (`± 0`)	`1` ns/iter (`± 0`)	`4`
`lotr_graph/has_node_nonexisting`	`5` ns/iter (`± 0`)	`2` ns/iter (`± 0`)	`2.50`
`lotr_graph/graph_latest`	`3` ns/iter (`± 0`)	`0` ns/iter (`± 0`)	`+∞`
`lotr_graph_materialise/materialize`	`8058377` ns/iter (`± 18953`)	`1564816` ns/iter (`± 35303`)	`5.15`
`lotr_graph_window_100/num_nodes`	`13` ns/iter (`± 0`)	`5` ns/iter (`± 0`)	`2.60`
`lotr_graph_window_100_materialise/materialize`	`8052686` ns/iter (`± 28660`)	`1669150` ns/iter (`± 10700`)	`4.82`
`lotr_graph_window_10/has_node_existing`	`146` ns/iter (`± 10`)	`62` ns/iter (`± 11`)	`2.35`
`lotr_graph_window_10_materialise/materialize`	`3335194` ns/iter (`± 8224`)	`971980` ns/iter (`± 4278`)	`3.43`
`lotr_graph_subgraph_10pc/has_node_nonexisting`	`5` ns/iter (`± 0`)	`2` ns/iter (`± 0`)	`2.50`
`lotr_graph_subgraph_10pc_materialise/materialize`	`1984805` ns/iter (`± 25113`)	`334634` ns/iter (`± 1287`)	`5.93`
`lotr_graph_subgraph_10pc_windowed/has_node_existing`	`149` ns/iter (`± 10`)	`62` ns/iter (`± 14`)	`2.40`
`lotr_graph_subgraph_10pc_windowed_materialise/materialize`	`1210545` ns/iter (`± 32083`)	`230399` ns/iter (`± 2617`)	`5.25`
`lotr_graph_window_50_layered/num_edges_temporal`	`150713` ns/iter (`± 2479`)	`70121` ns/iter (`± 7586`)	`2.15`
`lotr_graph_window_50_layered/has_node_existing`	`396` ns/iter (`± 24`)	`129` ns/iter (`± 12`)	`3.07`
`lotr_graph_window_50_layered/has_node_nonexisting`	`5` ns/iter (`± 0`)	`2` ns/iter (`± 0`)	`2.50`
`lotr_graph_window_50_layered/graph_latest`	`83760` ns/iter (`± 1282`)	`36649` ns/iter (`± 916`)	`2.29`
`lotr_graph_window_50_layered_materialise/materialize`	`29913162` ns/iter (`± 72765`)	`3488825` ns/iter (`± 24948`)	`8.57`
`lotr_graph_persistent_window_50_layered/num_edges_temporal`	`651472` ns/iter (`± 7627`)	`192686` ns/iter (`± 1569`)	`3.38`
`lotr_graph_persistent_window_50_layered/has_node_existing`	`445` ns/iter (`± 442`)	`174` ns/iter (`± 83`)	`2.56`
`lotr_graph_persistent_window_50_layered/has_node_nonexisting`	`5` ns/iter (`± 0`)	`2` ns/iter (`± 0`)	`2.50`
`lotr_graph_persistent_window_50_layered/iterate_exploded_edges`	`3499404` ns/iter (`± 35541`)	`1659940` ns/iter (`± 19402`)	`2.11`
`lotr_graph_persistent_window_50_layered/graph_latest`	`125522` ns/iter (`± 1888`)	`57549` ns/iter (`± 4809`)	`2.18`
`lotr_graph_persistent_window_50_layered_materialise/materialize`	`52778741` ns/iter (`± 272325`)	`5298035` ns/iter (`± 147912`)	`9.96`

This comment was automatically generated by workflow using github-action-benchmark.

… over. Now, we have many import errors because things from raphtory can't be imported (like GraphError)

…_v4_move_graph_folder

….rs file over. Now, we have many import errors because things from raphtory can't be imported (like GraphError)" This reverts commit 2cab13c.

…build the GraphMetadata in Raphtory

… on Metadata.

# Conflicts: # raphtory/src/db/graph/views/filter/model/degree_filter.rs

…ata to update metadata instead of previous refresh_disk_graph_metadata.

…to disk

ljeub-pometry

the Drop implementation for storage should be gone, the metadata write can now be part of the flush in the disk storage

ljeub-pometry · 2026-06-22T16:28:05Z

            if let Some(disk_path) = self.disk_storage_path() {
-                let disk_path = disk_path.to_path_buf();
-                storage::refresh_disk_graph_metadata(
-                    &disk_path,
-                    self.count_nodes(),
-                    self.count_edges(),
-                )
-                .map_err(|err| MutationError::from(err).into())?;
+                if let (Some(data_folder), Some(graph_dir)) = (
+                    disk_path.parent(),
+                    disk_path.file_name().and_then(|name| name.to_str()),
+                ) {
+                    let meta = Metadata {
+                        path: graph_dir.to_string(),
+                        meta: build_graph_metadata(self),
+                    };
+                    meta.write_atomic(data_folder, &data_folder.join(GRAPH_META_PATH))
+                        .map_err(|err| MutationError::from(StorageError::from(err)).into())?;
+                }
            }


this isn't needed here either once you have the actual logic fixed in the storage

arienandalibi added 3 commits May 27, 2026 03:21

Return GRAPH_META_PATH definition to graph_folder.rs and out of each …

f928edb

…storage crate.

Move graph_folder constants out of raphtory and into raphtory-api.

4209b36

Now that the pub constants are in raphtory-api, we can avoid the dupl…

9d1ddfc

…icate DIRTY_PATH in raphtory-graphql/src/data.rs

github-actions Bot reviewed May 27, 2026

View reviewed changes

arienandalibi added 2 commits May 28, 2026 02:13

Fix import

c62e445

Merge branch 'db_v4' into db_v4_move_graph_folder

b1e13b5

arienandalibi marked this pull request as ready for review June 10, 2026 06:30

ljeub-pometry requested changes Jun 10, 2026

View reviewed changes

Comment thread raphtory/src/serialise/graph_folder.rs

arienandalibi and others added 11 commits June 12, 2026 04:34

Instead of moving constants only, move the whole graph_folder.rs file…

2cab13c

… over. Now, we have many import errors because things from raphtory can't be imported (like GraphError)

Merge branch 'db_v4' into db_v4_move_graph_folder

adede86

Merge remote-tracking branch 'origin/db_v4_move_graph_folder' into db…

e92a0c3

…_v4_move_graph_folder

Revert "Instead of moving constants only, move the whole graph_folder…

f2ce071

….rs file over. Now, we have many import errors because things from raphtory can't be imported (like GraphError)" This reverts commit 2cab13c.

Run rustfmt

8b826a9

Move GraphMetadata and Metadata to raphtory-api crate. Still need to …

526fbf0

…build the GraphMetadata in Raphtory

Update write_metadata on InnerGraphFolder to use the new write_atomic…

a911f24

… on Metadata.

Merge branch 'db_v4' into db_v4_move_graph_folder

df18adf

# Conflicts: # raphtory/src/db/graph/views/filter/model/degree_filter.rs

Update Drop and flush on graphs to use InnerGraphFolder's write_metad…

ca3f9f8

…ata to update metadata instead of previous refresh_disk_graph_metadata.

Stop creating InnerGraphFolder, instead create Metadata and write it …

abe4b65

…to disk

Merge branch 'db_v4' into db_v4_move_graph_folder

c1f616d

ljeub-pometry requested changes Jun 19, 2026

View reviewed changes

Comment thread raphtory/src/db/api/storage/storage.rs Outdated

Remove Drop impl on Storage. remove test for it.

c7554f8

ljeub-pometry reviewed Jun 22, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Db v4 move graph folder#2624

Db v4 move graph folder#2624
arienandalibi wants to merge 17 commits into
db_v4from
db_v4_move_graph_folder

arienandalibi commented May 27, 2026

Uh oh!

github-actions Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

ljeub-pometry left a comment

Uh oh!

Uh oh!

ljeub-pometry Jun 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

arienandalibi commented May 27, 2026

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change? If yes is this documented?

How was this patch tested?

Are there any further changes required?

Uh oh!

github-actions Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

⚠️ Performance Alert ⚠️

Uh oh!

Uh oh!

ljeub-pometry left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ljeub-pometry Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot left a comment •

edited

Loading