Aggregate in concrete to be drilled

I am building a pergola and am laying 6 foundation pads which I will bolt pergola post supports to. My question is, if I am going to drill into the concrete for the support bolts, do I need to choose a specific size of aggregate? If there are big chunks of stone in the mix well that make drilling into it a problem?