Tandon is besmirching my intended major.

At 2014 Tandon fully merged with nyu and coincidentally NYU's math ranking dropped a staggering 8 places (http://www.shanghairanking.com/World-University-Rankings/New-York-University.html). From past years before the full merger, there was some fluctuation over rising or falling ~1 place in rankings. It's no coincidence that at 2014, NYU's math fell from 13th place to 21st place and its still falling leading me to believe NYU's brand as a legitimate good math school is irrecoverable due to the administration's shortsighted decision to merge. I predict in the near future, less talented math students will apply to NYU due to its fall from grace and it will start a cycle between worse math students, worse faculty, worse funding, worse research, and even greater fall in math ranking. If I decide to pursue a major in math I will constantly have to dispel the assumption that I am a Tandon student majoring in math whenever the subject of majors come up. This is a hassle and it gives off a bad first impression that "I was too lazy and unmotivated in high school to apply myself resulting in me attending Tandon".

Is it worth it to major in math even if I have to put up with the hassel? Should I switch majors to philosophy, it has a similar method of logical thinking. Or should I transfer to a school that doesnt butcher its own math program's reputation?

P.S. I being completely honest here, not some PC bullshit about oh every school is good/has its strengths as long as you work hard. Rank is there for a reason and you can ignore the statistics as much as you want but don't try to sell me on your PC ideals ( rank doesnt matter, you're pretentious, don't look at prestige, etc). If you're gonna spout some PC crap or use some ad-hominem attack, i will just ignore it. I would appreciate an answer to my question with legitimate reasons.