Transcode from H264 flattens skin and textures too much

I have a bunch of 720p (~4000kbps/30fps) and 1080p (~7500kbps/30fps) videos in H264. They look worse than what I'd expect from these bitrates, so I suspect they've probably been reencoded from a lower to a higher (current) bitrate. It's not CRF since the bitrates are pretty stable around those values. I want to bring them down to something more reasonable for what they are. The lighting is pretty flat and the original camera wasn't the greatest quality. I tried to transcode to AV1 with ffmpeg and libsvtav1 to about half the filesize, by adjusting CRF by trial and error. The results look pretty good on their own, but when played side by side with the source there is a very obvious loss of digital noise and what's worse - loss of detail on skin and some surfaces. It seems to be worse on the 720p.

All tests were done at preset 4. These are combinations I've tried, but it just resulted in increased bitrates approaching the sources and still flattening skin and textures:

• variance boost strength - up to 4 and octile down to 2
• 10 bit
• ac-bias, pretty high
• tune VQ
• lowering CRF
• enable-overlays=1 - not sure what it does, read it in a guide
• scd=0 - since there are no cuts
• scm=0 - made the encode go faster

It seems that AV1 just smooths everything no matter what. Is this because the sources already have been compressed a lot? Are there any settings that can preserve textures better? It's weird, because the sources already have visible compression artifacts and digital noise, but AV1 kinda puts a blanket over them and I can't find the settings to preserve those without the files getting even bigger than what I have now in H264. I though I'd ask here before wasting more time in experiments, grain synthesis being the next step.

Any suggestions? This is my first attempt with AV1, I'm more used to the older codecs.