Council tax between houses.

We get the keys on Thursday and won't be fully moving into the property until a week or two later. How does council tax work in this time? I'm getting the impression I'll have to pay for council tax on both properties until we move out of our current house (which is rented)?

Sorry if this seems like a daft question, but I've been told different things by different people and I'm a first-time buyer.