I have two ideas come to mind.
-
Write the .dag file to a memory based file, which should greatle improve read/write times. Only noticeable on large datasets I imagine.
- This should be easy given rclone. I'll test next time I have a giant dataset to work with.
-
Can the .dag file be skipped entirely? Specically, can the flat file be skipped, and can the data be streamed from the database directly to the cytoscape client?
- I feel like the answer is yes, but also, I'm still debugging a fair amount, and having the flat file to poke around in is helpful.