Fear of being dismissed if standing up for others

Sorry if this is posted in the wrong place.

I want to ask about standing up for others and calling out behaviors that are out of line. I want to get involved and do this, and had many situations in which I bad the opportunity to say something but chose to do nothing or not speak out for fear of being dismissed not just by the one who is doing the wrong but by the other person. I am small physically and have a disability and slight slurred speech. I am afraid of being swiftly dismissed like someone thinking 'I don't want someone like this person defending me' out of embarrassment or 'who tf are you, get lost'.

Is this just all in my head, or should someone like me remain passive?