Leaving my firm for the government, should I reveal the agency if they ask?

3rd year at v50 about to give notice to my firm that I am leaving for a litigating federal agency. If they ask where I am going, are there any downsides to telling them?

My firm has a lot of attorneys who move to/come from the government.