TTC After Mirena

So I've been reading a few places that saw that you should wait a while before TTC after the removal because the lining might be too thin and you might miscarry? I had my IUD in for about 1.5 years. I am a little worried that if I get pregnant I might miscarry. My dr had said I could start right away but why would they tell me that if there was that risk? Any help is appreciated.