Saranac (Knitty) sweater - modification help needed

I'm working on the Saranac sweater from Knitty for my boyfriend.

I've cast on to do the sleeves first, two-at-a-time (because I hate sleeves and want to get 'em both done first!). I've gone through the first skein and the sleeves now fit him from wrist to ALMOST his elbow so far.

I've had him try on the sleeves to make sure they're a good fit as I was knitting and we decided there were a few too many increases on the bottom. Even looking at the photos of FOs, the sleeves are too baggy. So I left off a bunch of increases and just knit straight once we got to a good width for the bottom half of the sleeves.

Of course, now I'm thinking ahead and wondering... how many stitches do I need to be able to connect the sleeves to the rest of the sweater? (I'm making the XL size.) It looks like 112. Right now I'm at 64 stitches per sleeve. I'm kinda worried that I'll either a) end up with sleeves that are too small to connect to the sweater or b) end up with sleeves that are triangular because I'm trying to squeeze in all the increases.

Any ideas on what I should do?