Improve streaming

[mr-devs]

According to Twitter's page about consuming streaming data we want to be able to do the below:

• Establish an HTTPS streaming connection to the filter stream endpoint.
• Asynchronously send POST requests to the filter stream rules endpoint to add and delete rules from the stream.
• Handle low data volumes – Maintain the streaming connection, detecting Tweet objects and keep-alive signals
• Handle high data volumes – de-couple stream ingestion from additional processing using asynchronous processes, and ensure client-side buffers are flushed regularly.
• Manage volume consumption tracking on the client-side.
• Detect stream disconnections, evaluate and reconnect to the stream automatically.

[yang3kc]

This is very things get complicated.

[mr-devs]

This is very things get complicated.

I agree. I don't think that we need all of these but probably we do need to deal with the low data volumes and disconnections - which may be easier.