Mesh distortion problems in explicit analysis, I am confused

I am a master student, and mostly self-taught in abaqus, so please bare with my stupidity :)
I am trying to simulate a tumour growing inside a brain. I have a skull, a brain and a tumour part. I am simulating the growth of the tumour by thermal expansion. The tumour grows fine, and if I use a coarse mesh on the brain part, the simulation runs fine and I get my results. There is large compressive distortion of the brain material. I want to refine the mesh to get more accurate results, but when I refine the mesh, it aborts with error: Deformation ratio exceeds wave speed, or excessive distortion/zeroMass. I am using explicit step, and I am applying the thermal expansion gradually over 5 seconds. (when I use a shorter time frame like 2s or 3s i get the deformation ratio error)
I am very confused about both these errors, because when I have a coarser mesh, I get away with lots of distortion! But when I refine the mesh, it suddenly can't handle distortion any more.. Can any of you explain this to me, please?

This is the results with the coarse mesh, but I am aiming to compare shape of the brain, so the coarse mesh is messing with my results.

This is the results with a trace of tumour and brain to make it easier to see. The pink is the tumour, if you were in doubt. As you can see the brain tissue behaviour is affected by the large elements...

This is the results

This is the initial model before thermal expansion of the tumour. The shape of this tumour is cut out of the brain so it fits well. They are connected with a tie constraint, and a contact initialisation is applied to deal with the slight over-closure that occurred.

I have tried triangle mesh, quad mesh, with/without ALE, with/without distortion control etc. but since I struggle to understand what is happening I also struggle to understand what I should do to fix the issue.

Can you help me, please?