Waiting to brush + green tea?

So, I'm trying to wait at least 20 minutes after I eat/drink something to brush because I'm worried about enamel damage. But, I know that plain green tea isn't acidic (which is the culprit here) does that mean I could take my aligners out, drink it, and brush immediately after? Sugar free green tea. I don't want to drink with my aligners in. Thank you!!