No firm goal?

Is it a dumb goal to be an attorney and never work at a law firm? Just seems like firm culture and politics aren’t a good fit for me. I feel like I wouldn’t fit in very well at most firms regardless of size for various reasons. What would I be missing out on or lacking as my career progresses if I never worked at a firm?