Incident raygrid bottleneck #14
Description
The migrate_event function in this package has a serial section that computes the incident P wave travel times in a 3D volume. That section takes about 2 minutes to run. Not a serious concern when the number of workers is small given the serial time to process the plane wave summation (not done in parallel) takes about 120 s per component. Serial that is typically several hours. However, with a reasonable cluster that could be reduced to something as low as around 5 minutes. 2 minutes is a large fraction of 5.
I think of two solutions to this problem:
- Write an application that precomputes the travel time volumes and stores them in files with indexing via mongodb.
- An in memory version could be created in parallel as a preparatory step for running a full data set through migrate_event. i.e. right now the model is parallelism only within migrate_event with the complete processing a loop over all events.
Approach 2 would be cleaner, but (a) much more complex, and (b) prone to creating a new memory problem since the travel time field volume are not exactly tiny.
mspass-team members please comment. Not a top priority but something we need to ultimately address to make this package work efficiently.