Is it bad practice to ambush a players camp when they are asleep? (Player agency question)

I am running daggerheart right now, a homebrew campaign based on the Drylands campaign frame. I have been hinting to my players that in the village they are in, a cabal of criminals have taken an interest in them and are trying to stalk them.

They have not pursued any of them, and I intend on introducing this group as antagonists.

Since they are not damaged, and have set up camp outside of the village they were staying in, I wanted to stage an ambush from the group. I am wondering if you had any input on how to stage this.

There are NPC guards that are taking the first "shift", and I was considering having them awake to these guards in conflict with their pursuers.

I do not want to railroad them into this scenario, and they absolutely do not have to clear all of the enemy PCs, as I am hoping they come up with a creative way to thwart them. I just feel like staging an ambush while they are asleep could be considered removing player agency.

Trying to keep this general and not get into lore specifics, but would love your inputs.