Looking for help with Zabbix and python script
Hello all,
I'm currently working on implementing some custom checks in Zabbix (monitoring solution), and I've run into a problem which resists all my attempts to fix it.
So, here's the thing: I'm periodically running a python script triggered by a Zabbix check. The script pulls some data from external servers, and then feeds it into zabbix via it's zabbix_sender utility.
zabbix_sender works like this: you feed it one or multiple lines in '<hostname> <zabbix key> <json formatted string>' format, and it sends these lines to the zabbix server for processing. Now, here's the problem.
When I manually run the script, direct it's output into a file, and then call zabbix_sender from the shell with that file as input, it works just fine. However, when the script is triggered by the zabbix server, for some reason, the data it feeds in via zabbix_sender is cut short somewhere in 2/3rds of the json string, so the server rejects it as invalid json data. Now, I'm completely stumped by this. In the shell, I'm using exactly the same command as in my subprocess.call() below, both my regular user and the 'zabbix' user have the same shell, and even when i switch user to 'zabbix', and run the command manually, it works just fine.
I have no idea, why there is a difference (and a pretty strange one at that) when i run the exact same command with the exact same input file through the shell and through a shell spawned by my script.
If anyone has any ideas, what might be wrong, please let me know. The relevant code is below:
try:
# create a temporary file, write data, pass the filename into zabbix_sender, close file
f = tempfile.NamedTemporaryFile()
logger.debug('Created temporary file {}'.format(f.name))
f.write(data)
command = '/usr/local/bin/zabbix_sender -c {} -i {}'.format(config, f.name)
logger.debug('Calling zabbix sender: {}'.format(command))
subprocess.call(command, shell=True)
f.close()
# die on error
except Exception as e:
print('Error: ' + str(e))
sys.exit(1)
Thanks for any advice.
Posted Aug 27, 2015, 11:37 AM UTC
Hmm. Couple thoughts on things to try. First, move your
f.closeto be before you call the zabbix sender. It could be that the entire write has not been committed, or something, before close. I would not expect this to be a problem, but it might be.Second, grab the return value from your
subprocess.calland print it out. It may give you some indication what is happening.Third, consider a pure python solution, like this module: https://github.com/jbfavre/python-zabbix
This would allow you to avoid having to subprocess out a system call.
It's my best guess with the available information, actually.
Better yet, OP should use a
withblock to manage the file.Good suggestion. Yeah, best guess is either the close or some environmental variable not making it to subprocess.call. I'd give even odds on this, just because how much caching could there really be for a bit of json.
Also for the OP, if you still have problems, check out this link:
http://stackoverflow.com/questions/7127075/what-exactly-the-pythons-file-flush-is-doing
for some ideas on taking it to the next level if the write still is not finished even after closing.
Hey, thanks for the reply. I'll give the first two suggestions a try, and if it doesn't work, I'll try the python module. I've been planning to use ZabbixSender if i can't get it to work, but I like the one you linked more.
Thanks again!
Cool. Report back if this solves your problem!
Allright, so I tried implementing the
withcontext manager, but no luck, the json string is still clipped. I will probably try to call zabbix_sender multiple times with smaller chunks of the json string, but I'm not sure if Zabbix won't mark the rest of the items as not discovered in between batches.If that fails, I'll try it with pure python, I've already wasted enough time on the shell and subprocess and I'm getting slightly annoyed, heh.
Thanks for the suggestions anyway!