Why did Jaime try to escape from Brienne?

He was going to Kings Landing and was safer with her anyways. It doesn't make much sense.