Do i have to wait for the Adhan to end?

while i was praying in the Imam Bargah at the time of Zuhar, a woman came to me and said that i am supposed to wait for the Jamat to end the prayer, THEN i can pray my Fardh namaz. is it true that everytime i have to wait or i have to assume that jammat has done their prayers and now i can pray. she also said that if i want to pray on time ill have to join the jammat like behind the men... can anybody confirm me?