Hanging curtain rods on plaster/brick walls

Hi all. I bought a 1930s brick home last year and I've had a nightmare getting curtain rods to stay up. The exterior is solid brick. The interior is several layers of paint over plaster I believe. There's no central air only hot water radiators throughout so temperature and humidity fluxuates quite a bit.

I have no inate handyman knowledge so I was hoping for an easy solution and want to avoid drilling into the plaster and brick if possible. I'm afraid of causing damage to the wall because frankly I have no idea what I'm doing with tools.

My first approach was these heavy duty clamp adhesive curtain holders. The first time I stuck them up I didn't let them sit long enough so some of them dropped a couple weeks after installing. Some of the original ones are still up.

I then found this thick adhesive mounting tape and applied that for the curtain mounts that fell down. Some of those lasted a week some are still up. The ones that fell I noticed the adhesive remained on the mount but some layers of paint came off. Not sure if humidity is playing a part in the curing process but long story short I can't seem to trust the adhesive route - the paint, plaster, or adhesive tape are unreliable.

I've looked into options of mountaing directly into the wood window trim. Ideally I could mount above the trim though onto the wall itself since mounting to the trim is honestly a bit ugly.

TLDR Are my only options mounting to the wood trim or drilling into the plaster and brick? Anything easier or less damaging to the exterior wall?