Am i wrong for thinking to limit my contact with sc/st batchmates?

All this sc/st act bs going on has been getting to me and today I realised that I talk very openly with anyone without a filter and I'm scared I might make some edgy joke. I'm scared my life gets ruined just bcz of something I say or a joke i make .

Like I'm not on equal footing with them ,they have a weapon to legit destroy my life for no reason and thinking about this scares me.