Streaming utility to build largely-scalable, highly available, fast, and secure distributed streaming APIs!
- Simple reason is that distributed systems are cool because of many reasons and some of these reasons are largely-scalability, high availability, secure, and fast.
- With this project, I attempt to go in-depth on how to build a system that grows in functionality as well as users and the team developing it.
- It's my attempt to broaden my knowledge and make it strong by developing this real-world end-to-end product.
- Simplicity
- Strongly typed and compiled
- Compiles to a single binary with no external dependencies
- Fast and lightweight
- Good coding practices
- Excellent support for network programming and concurrency
- Easy to deploy
- Go 1.16+
| Parameters | Type | Usage |
|---|---|---|
| DataDir | string | Path to the directory to store data locally. |
| BindAddr | string | The address that Goutube will bind to for communication with other members on the ring. |
| RPCPort | int | The port that Goutube will bind to for the member's RPC server. |
| ReplicationPort | string | If this server allows replication (by setting the ParticipationRule). This is the port where leader and its follower will communicate to sync with each other. (eventually consistent) |
| NodeName | string | Unique node name to identify this member. |
| SeedAddresses | []string | Addresses of other members to join the ring upon start up. |
| VirtualNodeCount | int | Number of virtual nodes to create on the ring for this member. |
| ACLModelFile | string | Path to the model file for authorization using casbin |
| ACLPolicyFile | string | Path to the policy file for authorization using casbin |
| ServerTLSConfig | *tls.Config | To configure the server for authentication. |
| PeerTLSConfig | int | To configure the client for authentication (to connect to other servers for replication, forwarding, etc.) |
| LeaderAddresses | []string | Hash function to calculate position of the server on the ring. |
| Rule | ParticipationRule | Participation rule to define the boundary and extension of the server. |
| MaxChunkSize | uint64 | Size of the chunk that can be processed by this server. |
| MultiStreamPercent | int | Percents of the number of followers to return upon GetMetadata request? |
Feel free to create a PR, I’m more than happy to review and merge it.
- Onboard videos and documentation
- Clean code, full test coverage and minimal tech debt
Feel free to create an issue, if you have any problem running this distributed system or any suggestions.