Why did none of Aragorns many ancestors take the throne of Gondor?

Elrond has been raising generation after generation of heirs to the throne who held proof of their heritage, why did they all spend their ~200 years of life hanging out with the rangers of the north guarding the borders of The Shire while Gondor slowly dwindled?