My partner doesn't seem to like my child

I'm so confused with what to do with my situation as my partner seems to not like my son , he is 2 and he struggles to talk and he has stated in the past that he has struggled to make a bond with him because of worries of his biological father coming back, he refuses to interact with him or take him anywhere with me and it really upsets me, I'm on a lower income than he is and we are supposed to take a trip soon and wants me to fully cover for him which is understandable it's not his child but I asked him if I fall short would he potentially help out with food and he said no. ( I would obviously just give whatever food I am offered to my son) but it just really hurts, is there anything I can do or say to make it better?