Prisoner of Azkaban time questions

I know playing with time is a major part of POA; but it doesn't quite make sense to me:

If Harry's birthday is at the end of July, Aunt Marge comes for a week so probably left August 6. Then Harry stays in Diagon Ally for 2 weeks and then he sees Ron and Hermione and the next day they go to Hogwarts (September 1.)

Does someone understand this?