[TOMT] Book, childrens fantasy, 1 day is a year

I read this book late 90s however I picked up the book from the library so it was probably a few decades older. The basic gist was a boy and a girl in a large house/castle with a bunch of other people. Each day had the seasons and each dinner was a Christmas feast. Towards the end of the book the children realise that each day they were in the house was a full year that passed in the real world. I had always thought this book was part of the Chrestomanci series by Dianna Wynne-Jones but I'm on the last book in the series at the moment and there's no indication that the story line will match my TOMT. I've also had a look through her other works and none match. Does anyone know what book I'm remembering?