Whats wrong with my gearbox?

Should I worry if my 535d sometimes jerks when shifting?

Its a 2005 535d e60 rwd with the 6 speed automatic.

It happens rarely when the car and gearbox are warm and a bit more frequently when these components are cold. Still even when its cold it doesnt happen that often. Maybe if I drive in the city and its constant gear changing in stop and go traffic then it happens once every 10-15 minutes. I little jerk when shifting. It feels like when you shift a manual car with no rev matching and jump off the clutch.

Important that this does not happen at all in higher gears at higher speeds, it only does this when you are starting from a standstill. It starts smoothly and then jerks one and continues smoothly. I dont know if it jerks from first to second, or if it jerks in first when it grabs and then the shift into second is smooth (idk if thats how an automatic works, this is my first automatic)

Also i have noticed a pattern. If I concentrate on 1 thing I can drive the car fully smoothly with it never jerking. If i always either start the car sporty (meaning slightly bigger gas from standstill so the gearbox can use the rpms while shifting) or I always start smooth with gentle throttle and steadily increase it until the first shift then it always shifts smoothly. This jerk only happens if I start gently than add a bit of gas quickly or the other way around if I start with a bit more throttle than slightly decrease it quickly (not fully) then it happens.

Is this a fault with my gearbox mechanism? Or is it something else? Maybe a fluid change would solve it? Maybe I dont have enough fluid? I dont really know all that much about automatics.

The car never has problems shifting at any throttle and any speed except the scenario I described. Thats the only scenario where it acts up.

I brought it to a mechanic, who test drove it and just said that my gearbox is bad. Overreacting like any mechanic I ever met. Than he added that a fluid change might solve it, or it might make it worse to the point where I would need a new gearbox.

So what do you guys think? Should I take the risk of changing the fluid, or just live with it? Will it get worse?

Any tips or recommendations are welcome.

Thanks!