Source
r/learnpython•
IonRed

Looping Python Application, and timed GET requests.

Hey guys,

I need to build some logic into my python engine where it does not stop after a computation is ran but circles back around, runs another GET request to the endpoint and does the computation, etc again. What is the best way to go about doing this?

I assume one way to do this would be a while loop and if it does not = true then return to top. But sense there is no goto in python I am not sure how to go about setting this up.

here is a paste bin with the current code (NOTE: get requets, etc are commented out for the time being)

http://pastebin.com/UZ2Xc0mR

Posted Jan 19, 2016, 9:17 PM UTC · edited

122
Th3BaconNation•

The way I do this is I use a function for the gets and computations, and based on what it returns either the script tries again or stops. I use a while loop similar to this:

while True:  
    check = function()  
    if "end condition" in check:  
        break  
    time.sleep(7)
2
IonRedOP•

Thanks for this. I assume the (7) in time.sleep() is seconds? can I set it to miliseconds by doing like:

time.sleep(.5)

?

1
i_can_haz_codeUsually Wrong.•

If you hammer a web server every half second, you may end up on the special kids list at whatever company. Just saying.

1
IonRedOP•

Endpoint can handle it, had long discussions with the people that handle the actual web side and they said to go for it.

There is a unique use case here where 2 seconds even would be too long.

1
i_can_haz_codeUsually Wrong.•

They can handle that level of volume but don't have an api?

1
i_can_haz_codeUsually Wrong.•
canary = 1 # This just sets a variable to one. I name it canary, because I will bring it low on some successful operation.
while canary != 0: # If it is the case that canary is not equal to 0, do these actions
    try:        
        foo()    # The action which, if successful, would end the loop.
        canary = 0  # set canary low, to end the loop
    except:    # [u/elbiot](/u/elbiot) is correct this should catch one, and only one exception. 
        time.sleep(1) # something has to be here. It can be pass, but if it is pass you will just hammer the server.
0
elbiot•

except SpecificException

1
IonRedOP•

could you expand some more on this, or how I would implement it with the code in my pastebin?

1
i_can_haz_codeUsually Wrong.•

I am not entirely clear on what you are trying to do. Am I correct in assuming you wish to run basically your whole script over several data sets? Or are you trying to run forever?

I commented my code above to make it more obvious what I was doing.

If you want to run some n number of iterations, you could do something like:

wrap the whole script using nested functions like this so that foo() is basically your script sans imports.

def foo():
    def action1():
        pass
    def action2():
        pass
    action1()
    action2()

Use range to run foo some number of times like this:

for i in range(n):
    foo()

or to run for ever/until stopped:

while True:
    foo()

No matter what avoid the temptation to do something like this:

>>> def foo():
    ...     foo()

Or you will end up hitting the maximum recursion depth, and see this gem.

RuntimeError: maximum recursion depth exceeded

I hope that was helpful, if I can clarify anything please let me know. :-) If I can get a more clear idea of your goal, I may even take a shot at implementing it for you. :-)

1
IonRedOP•

Basically what I want to do is the following:

  1. Place this on a server and have it always run new data sets via the get request.

  2. It will run the entire script (the GET, the Computation, and the POST every time).

Basically that is the main functionality. The goal here is to have this auto loop every time.

1
i_can_haz_codeUsually Wrong.•

Are we talking a *nix server?

How "up to date" do you want it to be?

If the answers are yes and within a minute, I'd say just do it with cron and schedule it to run once a minute.

If the answers are yes/no or every second or so, I would say implement your logic as nested functions and call the outer function inside a while True loop.

If the answers are no, every minute or so, I would vote for task scheduler.

If you do either the cron, or task scheduler options make sure to have your script check if the acript is already running and either kill them or exit cleanly.

I can take a shot at wrapping that in nested functions tomorrow morning if you like. It should be fast. On mobile now or I would. :-)

something like this?

1