Advice for smoking in garden without bothering neighbours quite so much?

Hello! I live with a partner who doesn't like the smell of weed. We're lucky to have a garden flat in the city, so I will smoke out there - 1-2 joints a day, usually spread into 2-6 "sessions". I'm quite committed to smoking actual joints - I like the ritual. Sometimes it's straight weed, sometimes I mix in marshmallow leaf. My partner prefers it if I don't smoke inside while he's home/up, but I do smoke inside when he's at work (2 weeks out of 4) or in bed (he's an early bird and I'm a night owl).

I've been made aware that my upstairs and next door neighbours are suffering with the smell coming in when their windows are open. Our part of the building is very sheltered and we don't get much of a breeze to carry the smell away.

I was wondering if anyone has any tips for or experience in reducing the smell? For example, is there anything I can plant, burn alongside, buy, or give to my neighbours that might help? The gardens are all currently empty and unplanted (the flats are newly built and gardens are yet to be completed). My next door neighbour and I discussed whether a higher fence and/or higher plants around the perimeters might help keep the smell in. But that doesn't help the couple upstairs!

Any advice much appreciated, but tried and tested tips are preferred. Thank you!