Was I in the wrong for this??

This is my second week ever being a dasher and I had an order to a house but when I got there he told me to come in through the front door - go upstairs and it was a door on the left.

when I confirmed he wanted me to go inside the house he said it was an apartment house but I said I did not feel comfortable doing that.

Is that okay to do as a dasher with house apartments like that? a normal apartment complex is okay but I haven't been in a situation like that before