Handling of unavailable services on startup

So this is issue has been following me through all my HASS installations (RPI, NUC, docker, systemd): Home Assistant does not handle connection failures when starting up.

Meaning that if influxdb or deconz start later than HASS, it will never reconnect as setup fails, so I'll have to manually check HASS whenever I reboot so that it started up correctly.

I do have a UPS and the server should never restart by itself, but still, I do want to leave HASS unattended after I configure it.

Most of the solutions I've found stops HASS from starting if dependencies are not available. (the best solution was to combine docker-compose depends_on and systemd - but recreating the container on startup wasn't always working reliably for some reason)

Also, I do want my camera feed and UI to work even if my lights are unavailable so that's a no go.

Has anyone raised this issue anywhere? (I could barely find anything, which kind of baffles me, it's the first thing you guard against when connecting to anything)

How does everyone else handle this? (Does hass.io handle addon startup order?)

Posted Aug 21, 2018, 10:00 PM UTC

810
bwduncan•

I just set systemd dependencies so home assistant starts after mqtt and that does it for me.

3
neruzOP•

Yeah, for the simplest cases, that works. But for example, influxdb's self-monitoring db gets quite large, and on an RPI, it might take 5-10 seconds to start. (though I've had 40+ too. Disabled it in the end)
For a while I delayed HASS startup with a sleep so it always starts up after, but that is just lazy.

1
bwduncan•

Can you do the same thing and make home assistant start after influx?

1
neruzOP•

Unfortunately, no. Basically you'll have to know which components are unable to reconnect, wait for them to come up with a custom script (sleeping is not a solution) which basically stops HASS from working.
I could create a timeout, so if in 1 min, influx doesn't come up, HASS starts anyway. But then the handling script becomes too complex for what it provides, and it doesn't even fix the issue. It's not good enough.

I guess the only solution would be to fix HASS's components to support retrying the setup.

1
bwduncan•

I think what you're saying is that you don't want HA to wait forever for influx because it might fail. In that case I think you can configure systemd so that HA is After= influx but doesn't Requires= it. Systemd will detect that influx has failed and start HA anyway. You're right that HA failing to retry is bad...

2
maniaman268•

Support for this was added to the core a while back but a majority of the components haven’t been updated to utilize it. My understanding is that each integration needs updated to raise a PlatformNotReady exception if it can’t connect and then HA will continue on and try to re-setup the platform again later

2
neruzOP•

Added more than 1 year ago according to the git history. Seems to work perfectly. This is the solution, thanks!

Edit: Unfortunately, that is only for platforms, not components. Replicated the retry code for the component setup, looks like it's working so far.

1
databoy2k•

I can only +1 on this one. I've recently swapped my zoneminder cameras from "Motion" to "off" (unless I turn on motion detection). If HA boots while they're all on "off" it can't actually access them again. I need to re-enable all cameras before rebooting.

Would be happy to learn a fix.

1
neruzOP•

That is a different issue, but quite related. HomeAssistant should be able to detect changes to components while running, not just when setting up. Unfortunately that probably needs restructuring of the architecture.

What I do currently, is use docker's health check and an outside process can restart it whenever it's unhealthy. (autoheal works, but I don't recommend it)

For example, this works for influxdb as it prints a persistent notification (deconz does not):
For deconz, and your issue, you'd have to check if certain entities exist.

check.sh

#!/bin/bash
set -e
response=$(curl -s -X GET -H "x-ha-access: ${API_PASSWORD}" -H "Content-Type: application/json" http://localhost:8123/api/states/persistent_notification.invalid_config)
if [ 0 -eq $? ] &&  [ '{"message": "Entity not found."}' == "${response}" ]; then
  exit 0
else
  echo "Home Assistant could not start: ${response}"
  exit 1
fi

docker-compose.yml

healthcheck:
  test: ["CMD", "/check.sh"]
  interval: 30s
  timeout: 10s
  retries: 5
1