UberX but XL shows up

If I'm with a large group, we'll call a couple of ubers. We call an UberX first and a large vehicle (think mini van) shows up. Can we put 6 passengers in there then? Or can the driver deny that