HI :) First of all, it's a great library, thank you for putting your effort into it.
I want to use DAG for ETL workflow purposes, with our architecture, we need to be able to proceed to vertex only if all of the vertex's parents finished their "job".
This is why running over the graph with BFS/DFS would not work for us.
To do so, I can think about 2 options with this lib:
-
GetChildren, GetRoots, DeleteVertex:
After the vertex completes its mission, we will find all of its children and we can delete the vertex from the graph. Now we will search again for roots, if a new root appears, a new vertex will start to run.
Complexity: with this approach, we need to save and track all of the roots (hash table).
The problem here is that GetRoots() returns all of the roots and we will have to iterate over all of them each time to find the new root.
Let’s take an example of 2 layers graph with 2n vertices, n roots, n leaves, each root connected to one leaf => O(n^2)
-
GetVertices, GetChildren, GetParents, DeleteVertex:
Pre-processing step addition. Iterating over all of the vertices to add “dependencies” meta-data (inbound edge, part of the vertex struct).
Complexity: we have to use GetVertices() → O(n) and iterate over them to get the parents num with GetParents for each vertex. Let’s take an example of 2 layers graph with 2n vertices, n roots, n leaves, each root connected to all n leaves => O(n^2)
We can improve the complexity of the second option if we could get the parents num (len(inboundEdge)) => O(n) instead of the actual edges.
Please current me if I'm wrong or if there is any other way.
Thanks a lot!
HI :) First of all, it's a great library, thank you for putting your effort into it.
I want to use DAG for ETL workflow purposes, with our architecture, we need to be able to proceed to vertex only if all of the vertex's parents finished their "job".
This is why running over the graph with BFS/DFS would not work for us.
To do so, I can think about 2 options with this lib:
GetChildren, GetRoots, DeleteVertex:
After the vertex completes its mission, we will find all of its children and we can delete the vertex from the graph. Now we will search again for roots, if a new root appears, a new vertex will start to run.
Complexity: with this approach, we need to save and track all of the roots (hash table).
The problem here is that GetRoots() returns all of the roots and we will have to iterate over all of them each time to find the new root.
Let’s take an example of 2 layers graph with 2n vertices, n roots, n leaves, each root connected to one leaf => O(n^2)
GetVertices, GetChildren, GetParents, DeleteVertex:
Pre-processing step addition. Iterating over all of the vertices to add “dependencies” meta-data (inbound edge, part of the vertex struct).
Complexity: we have to use GetVertices() → O(n) and iterate over them to get the parents num with GetParents for each vertex. Let’s take an example of 2 layers graph with 2n vertices, n roots, n leaves, each root connected to all n leaves => O(n^2)
We can improve the complexity of the second option if we could get the parents num (len(inboundEdge)) => O(n) instead of the actual edges.
Please current me if I'm wrong or if there is any other way.
Thanks a lot!