My theory on time travel. What is your view?

Theory: Time travel can only occur in one direction (into the past). The future is mapped out by every millisecond of action that takes place on this world (butterfly effect). The person travelling back in time would be unable to return because the traveler assumes the body, mind and soul of him or herself in that previous time. That traveler has no recollection of the life that they have and will eventually live and is in perpetual and everlasting stating of living the same life. As soon as he or she reaches that moment when they time travel back, the process continues. Their life ceases to exist after that point of travel but the world continues to go on. So there would be an infinite amount of the same universes as each passing time travel takes place.

We would never know if time travel is possible because the subject wouldn't be able to relay information.

This is just something I came up with before falling asleep last night. Thought it was interesting. I want to know your takes on time travel and if you even think it's possible.