Losing a heart during Elliotts 14 heart event

Hey so this may be a stupid question but I'm rather new to this game and I just got to Elliott's 14 heart event and he's been gone for about 4 days and my farmer just lost a heart with him. I'm now at 13 and I'm not sure if that's normal? Is it just because while he's gone I physically can't talk or give gifts to him? Just curious if I'm doing anything wrong or if this is totally normal and all will be well when he gets back. All feedback is appreciated, thanks!