r/learnpython•
DxNovaNT

Suggestions regarding the Distributed Queue

So, I was building a distributed queue for learning purposes, purely in Python.

Repo link :- https://github.com/Lumen-EIP/Distributed-Queue

Architecture :- You can find the diagram in the README

Although it's working, I think it's kind of too far from how real it works. Although I don't want to implement the exact same thing but I want to make it close enough. So I want your suggestion to improve the existing architecture and fix issues that you guys caught in the current architecture.

Currently I used a json file as a queue. There are 2 brokers one for the consumer and one for the publisher. Broker Manager is the common link between brokers, publishers and consumers. I try to re create the distributed systems by creating separate processes which kinda represents separate services and used async operations to represent how data send through I/O or socket.

Posted Mar 30, 2026, 6:19 AM UTC

10
Loading comments…