Action for new classical guitarist

I'm relatively new to classical guitar and have just measured my action for the first time. The low E is 5mm at the 12th fret and tapers to 3mm at the first. The high E is 4mm at the 12th and tapers to 2.5mm. I've seen people mention a 4/3 ratio, is that high E to low E or 12th to first? And how does one go about lowering the action? Is this something that can be done with the truss rod or does the saddle need to be shaved?