Will my SSA stop if I miss a random appointment?

So I see was on the phone with an agent at social security and she scheduled me an appointment. I'm really stuck on not going. Just haven't been out the house at all lately. Will I get in trouble if I miss it?