Bypassing the Python event loop for token-aware rate limiting with PyO3/Tokio

I’ve been exploring cross-runtime concurrency boundaries and built a PyO3 extension that does token-aware rate limiting. I wanted to see how cleanly I could isolate a Redis network layer completely outside of Python's single thread to minimize performance degradation under load.

Disclaimer: This is basically a proof of concept. It's basically tied to another experimental crate I am working on (axum-rate-limiter), and so it's not super configurable or abstracted as of now. Could you use in production? Probably, but why?

That being said, the raw performance under a 100-concurrency flood on a heavy, dynamically rerouted endpoint turned out pretty efficient:

• Pushed 1,128 req/sec without dropping a connection.
• Fastest response hit 15.3 ms.
• Fails closed instantly with immediate 429 rejections to protect downstream application logic.

The cool part: I benched a naked, no-op /health endpoint (literally just returning {"status": "ok"}) on the same machine, and it maxed out at 1,496 req/sec.

The fact that crossing FFI boundaries, handling memory pinning, and doing a multi-threaded Tokio to Redis round-trip only costs ~370 req/s, proves that the Rust integration added almost non existent overhead.

If you're interested to in checking out the project go to:

https://github.com/MordechaiHadad/rustgate