Spot instances

[thodson-usgs]

Has anyone investigated using spot instances? If not, I would.

For example,
https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/jobruns-flink-decommission.html

[yuvipanda]

Nope, but I think they sound like a great idea.

[thodson-usgs]

#11 enables spot instances
#14 discusses the two main configurations necessary to make Flink failure tolerant

• a restart strategy for task manager failures.
• a shared store for metadata used to recover from job manager failures