Why do we say "it's a piece of cake" when cake isn't that easy to make?

I've always been puzzled by the phrase "it's a piece of cake" to mean something is easy. I mean, have you ever tried baking a cake from scratch? It can be quite complicated! There are measurements, mixing techniques, and baking times to consider. So, where did this saying come from? Did people actually find it easy to make cakes in the past, or is there another origin to this expression? I just can't wrap my head around why we associate cake with simplicity when it can be a real challenge!