Losing Manpower

Currently playing as Japan, I'm not at war with anyone and im not producing any ships. For whatever reason my manpower is draining and i cant find the cause. Do agencies affect manpower? Or has anyone else had this issue?