AIO boyfriend made disparaging comment that had a prejudicial undertone

My bf was trying to find parking and was festive very frustrated because there was no parking. I happened to be on the phone with him and he hung up abruptly . When he called back he said sorry “this fing illegal immigrant guy was acting crazy and honking at me while trying to park” like very casually. This comment made me feel very uncomfortable and I got silent and detached as we were heading to our location. He then walked away quickly because he said I clearly was annoyed with him and told me to get over it because he didn’t mean it in a derogatory way. I pressed further and questioned in what other way he could have meant it. This ended up escalating into a big argument where he kept telling me to get over it. Is this a huge red flag? Or should I assume that he didn’t mean what he said.