Considering alternatives

[amotl]

LorryStream's evolution stalled a bit, so we might look into reusing
work from other authors, like FastStream or Proton. 🙇

[amotl]

mq-bridge by @marcomq looks like an excellent gem. 🙇

mq-bridge is an asynchronous message library for Rust. It connects message brokers, databases, files, HTTP/WebSocket endpoints, and in-memory channels behind one small set of traits.

It is not only a forwarder. A route can transform, filter, fan out, retry, rate-limit, deduplicate, or turn a request into a response before the message reaches the next system. The core is built on Tokio and keeps the transport details at the edge, so application code can mostly work with CanonicalMessages and handlers.

One pip install mq-bridge-py away, it will also be an excellent candidate to bring streaming support to omniload. ✨
LorryStream is not actively maintained anyway.

[amotl]

mq-bridge has been unlocked in omniload v0.5.0.. Thank you, @marcomd. 💯

/cc @hlcianfagna, @wierdvanderhaar, @hammerhead, @bgunebakan, @florinutz

[amotl]

This patch updates the README to refer to omniload/mq-bridge. 🙇

• README: Refer to mq-bridge and omniload projects #251 (preview)

[amotl]

Arroyo, a distributed stream processing engine written in Rust, also provides quite a bunch of connectors and formats.

• https://github.com/ArroyoSystems/arroyo
• https://doc.arroyo.dev/connectors/
• https://doc.arroyo.dev/connectors/formats/

For data conversion, Arroyo offers user-defined functions (UDFs) written in Rust and Python.

• https://doc.arroyo.dev/udfs/

However, Arroyo was acquired by Cloudflare. It's still open and development seems to be active, but the last release was more than half a year ago.

• https://www.arroyo.dev/blog/arroyo-is-joining-cloudflare/
• https://www.arroyo.dev/blog/arroyo-0-15-0/