Too many faces/reference images?

Is there a point where there are just too many people and face reference images that it would really slow down facial recognition?

Like if I had 20-30 named people with 30 images each, will my system just take too long to identify someone?

(On an i7 8700k, 5gb DDR4 ram allocated to frigate)