How do you know when it's time to leave a job that's "fine" but going nowhere?

My job is stable, decent pay, no real complaints. But I haven't grown in a while and I feel stuck. The comfort makes it hard to leave. For those who left a comfortable-but-stagnant job, how did you know it was time, and did you regret it?