How important are the cultists of the Dead Three ?

Hi! I've been toying with the idea of giving Baphomet more ground to stand on. Would it break the adventure too much if I replace the Dead Three cultists with Baphomet ones ?