Cat started on 5mg Clomicalm

So I started my very anxious cat on 5mg clomicalm 5 days ago. He’s sleeping a lot more and eating a bit less, which I understand is pretty typical and should go away as he adjusts. But he is way less affectionate and interested in being around me, which is incredibly unusual as he was very clingy and needed more attention than I could give him prior to taking the medication.
I don’t know what to do. He’s like a totally different cat. I would be ok with waiting this out if I knew it was going to get better but at this stage I am seriously questioning whether I am doing the right thing, it’s distressing to watch.
At this stage I feel a little silly taking him to the vet especially if this is a pretty common side effect, and money is a consideration. So if anyone has any experiences or ideas about what is happening with him hearing about it would be much appreciated! Thank you!