Global Data Store of Active Requests & Current State?
I have a next.js / express.js / PostgreSQL project with a fairly complicated dynamic dashboard. When users select different filters on the dashboard, it triggers a gnarly 600 line hook to trigger that fetches new data from the backend and essentially keeps a queue of the requests. The data fetches take seconds / minutes to complete due to the size of the datasets and the complexity of the calculations on the backend.
My express.js API further communicates to my containerized Python Flask & C++ APIs that live elsewhere when querying for certain metrics. These containers run even more complicated machinery on even bigger data sets.
I need some sort of global data store that will track the current state of each user's dashboard and all of their active requests. So if user A changes their selected metric, there should be functionality that should cancel all of the code running on the Express.js, Flask, and C++ services for the old metric's request. Because these queries are so complex I need to cancel any that aren't actively being selected for to improve performance.
I have a version 1 of this but its really messy and only works for the express.js backend, not for the c++ / Flask backends. Its just a simple in memory activeRequests object that I define globally in my express.js project.
Has anyone solved a similar problem to this?
Thanks!
Posted Nov 12, 2024, 8:09 PM UTC