Long running GET request results in the connection dropping
I'm looking for input on how I can further troubleshoot this. Issue is distance related and I don't know enough about networking and internet architecture to troubleshoot this. Any pointers on where I can start researching would be great!
Here's the problem:
- I have a server in Virginia running my REST client.
- The API I'm connecting to is in Oregon (3rd party vendor, I have no control over this)
- The GET request sent from Virginia to Oregon can take anywhere from 5-10mins before the REST client gets a response back from the API, which is usually just a 200 OK or some other HTTP response code. No body.
- I have a separate web console that I can watch to see when the API is done doing the job it was asked to do. When the API finishes it's task, that's when it's supposed to send a 200 OK back to the REST client that's waiting.
What's actually happening is this:
- REST client sends the GET request, from Virginia.
- API starts doing job specified in the GET request. Finishes in 5-10mins and I can see this from my separate web console.
- At this point, I'd expect my REST client to have gotten a 200 OK back from the API. But I get nothing! The REST client is just waiting and keeps waiting until it hits the timeout period
I tried different REST clients. I get the same result with Insomnia, and two simple REST apps I made in C# and Go.
If I send the GET request somewhere closer to the API servers, like Oregon, I always get a response back.
The 3rd party API has no way of sending callbacks and no API calls exposed that would allow me to programmatically check the status of a job that I previously sent. I know, very annoying.
Anybody have an issue similar to this before? What are some things I can look at to try and explain this behaviour and resolve it?
Posted Apr 20, 2018, 3:52 PM UTC
It doesn't look like a good design. I would have have given a success response immediately and given the task to different thread for processing. And another API would give me the status for this process. So producer polling this would tell me when the prices would be over.
I totally agree with you, but sadly I have no control over the design of the API and I have no way of polling the job status through their API.
From a networking standpoint, is there anything I can do to ensure the REST client doesn't lose the connection? Thanks for your response!