Do mousepads affect direction of mouse?

I'm asking this because I have recently started using my mouse for work. (procrastinating rn). I also have a mouse pad I've been using, however, if I move it directly straight to the left or right on the mousepad, it also moves down diagonally as I move straight to the left. However, when I take out the mousepad and use it on the hard surface with no mousepad, it move directly straight as it should be, but with the mousepad it moves diagonally down with it. Thoughts? Comments? Could it be the pattern of the mouse pad I use? Thanks.