The main character can travel back in time but completely forgets all details of his/her future time that don't fit with where he/she has travelled to but they still remember who they are to some extent so its not full amnesia.
This solves the problem of time travellers trying to interfere with or complete timelines and the bigger picture but they still fufill their place in the time line while not causing a massive disturbance in the place. Since time has changed they sound rather foreign but this explains away any incongruousness and confusion.
Possibly this is caused by the brain tethering you to your timeline by the facts that you know about your current time. However babies can rough