Non-Availability Scheduling

My SSA keeps scheduling me outside my approved availability hours. He said that when doing the schedule, it doesn't tell my availability, that he has to look elsewhere for that. Is he telling me the truth? Thanks.