Is gold actually even that rare?

So I know that there's a whole thing surrounding diamonds and how "big diamond" stockpiles and hoardes the diamond supplies in order to keep the prices high and avoid diamond-inflation, but is the same the true with gold?

I know it's considered a rare metal, but is it in the same way that diamonds are? Because as humans, we've constantly valued gold as a metal above all else. But is that due to inferior mining methods?

Is the whole thing about gold being rare down to a mix of cultural labels (gold being labelled as holy in some religions) and historical assumptions?

Is there "big gold" controlling everything?