Are some languages just objectively harder for native English speakers than others?

We often hear that the "easiest" language is the one you are most motivated to learn, and that difficulty is subjective. But does that hold up when you look at time commitment?