Old gothic horror

Hi, first post. So, the movie was I believe in B&W. A man inherites an old mansion in which his ancestor lived so he moves there. The town down the mansion is full of miserable people - children are born without eyes, some are completely deranged etc. The townsfolks believe the man's ancestor was doing witchcraft and cursed the town. They try to chase the man away while he seems to fall under the influence of the mansion and his deceased ancestor... Crowen or Crowden or something similar. Watched is as a kid, looked it all over the place and found nothing. Does it ring the bell to anyone? Tnx.