Light doesn't fully turn off after replacing with Caseta

I had my electrician out recently to do some work and asked about moving some of my switches somewhere more convenient. He said he could, but it would be simpler to just install some Lutron Caseta switches so that I can put the remotes wherever I want. He knows I am capable of doing some minor electrical and isn't one to sell me services I don't need, so it seemed like an easier, cheaper option.

My house is from the mid-1950s and while we have had the electrician rewire all of the receptacles upstairs (downstairs were added later), the lighting is still using the original cloth wire that is, in many cases, not grounded. The boxes are all metal and where there are grounds, they are at least connected to the boxes. Anyway, here is rough sketch of the wiring as it was in the box:

Wiring diagram

The grey off to the left are all the white wires bundled together, presumably those are the neutrals. Two black wires came in, one from each side, and were bundled together with a jumper that went to the left switch. A third black wire connected to the same spot, then a jumper from that switch to the right switch. Red wire coming from the right side was connected to the left switch and finally a black wire connected to the right switch. I didn't draw it, but there is a bare wire connected to the box in this case.

When I replaced it, I connected the white wires of the Caseta switches to that bundle of white wires, the black wires to the bundle of black wires (I removed the jumper and just put them all together, though, there's barely room in the box as is), red wire of the left switch to the red wire in the box and the red wire of the right switch to the remaining black wire. I'm just counting on being screwed into the box for ground. Blue wires aren't used, so I snipped them off. Again, limited room.

I assume this is all wired correctly as it works. The lights go on and off just fine, but the one connected to the right switch never goes fully off. I'm guessing it's a dimmable LED bulb as it stays on as if it was on the lowest possible dimmer setting. When I swapped it to a non-dimmable LED, it just flashed occasionally instead. I tried swapping the black and red connections on the right switch, which still worked but did not improve the situation. From what I've read, it could be that these switches have a small amount of current flowing so the remotes can work or it could be that some current is leeching from somewhere else in the circuit or it could be that something somewhere is wired backwards. All of these seem possible, as I said the wiring here (that wasn't done by my electrician) is a mess, but this didn't happen with the regular switches before. What should I be looking for and what are my options for getting this to work properly?

And before someone says it, yes, I've asked my electrician about it, but if it's a simple fix, I'd rather not pay $400 to have him come back out to fix it for me.