What exactly is the fourth dimension?

After reading "The End of Eternity", I felt that the fourth dimension was time. But after reading "The Three-Body Problem", I started to think it might be more like an extra spatial direction that we simply cannot perceive.

So now I’m a bit confused—are we talking about time as the fourth dimension, or a physical direction beyond length, width, and height? Or could it be that both interpretations are valid in different contexts, like physics vs. sci-fi worldbuilding?