What is the best way to evaluate if benchmarks are no longer the best?

For a long time, we developed benchmarks and evaluated them against it. Recently we see a lot more Chinese models performing really well in those benchmarks but the general feedback from users seem to be, don't trust the benchmarks. This sort of reminds me of the infamous Dieselgate. However now the question is, how are you managing to evaluate the models?

I have seen some people mentioning to use some questions only you know and never publish it. I think this could work but it seems more anectodal to me.

Are there other tricks you use?