Does the Earth have less mass than before our species became "advanced?"

Based on the laws of conservation of matter and energy, because there are now seven billion of us, and because of the amount of matter we have transformed into "man-made" objects (e.g. roads, buildings, etc.), does the Earth now have less mass than before the advent of our species? That is, in theory, if we were to remove humans (since our species, as organisms, use up more matter and energy than other species in terms of our size and numbers) and the largest man-made objects, would the Earth be smaller or have less matter than it did a billion years ago?

EDIT: Sorry, would it be better to post this in r/askscience? Just found that subreddit after already posting this. :\