How to deal with kids baseball coach?

My 11 year old son absolutely loves baseball the problem is he isnt athletic. He has had a few games where he was on fire but mostly he has anxiety and gets in his own head leading to lots of strike outs. Despite that he likes to do little league. He has come home from multiple games crying that he was embarrased but he always goes back.

We just started fall ball. We live in a small district and not a lot of kids play baseball this time of year so our team ends up being all diffrent ages and abilities. Last night my son came home from Practice #1 saying he was quiting because he is too bad (like ususal). I have anxiety and hate going to games. Im the type of person who doesnt like to do things I don't immediatly excel at so I dont really understand why he still plays and enjoys it. However I appreciate he keeps at it and its really good for him to get the excercsise and the friendship.

Today the coach accidentally texted the parent group chat that the team was bad and 6 of the kids have no business playing baseball. He ended it saying he was glad 3 of our games got cancelled. He immediatly apologized and no one else responded. I assumed he is talking about my kid and got so upset I had to leave work to cry in the parking lot for 20 minutes. Honestly I want to just pull him. I dont want to see that coach and everytime my son strikes out im going to be think that everyone else is saying "that coach was right". I dont want to tell my son what was said.

This isnt a travel league. I was under the impression little league was just for fun? Is it unfair to have a really bad kid on the team? Our first game is Monday. Do we just go and act like nothing ever happened.