Fig decides to dump its second crop every year.

We have a fig tree that is about 7 feet tall (variety has been lost to time. Maybe Olympic?). It produces two crops, summer and fall. For the first crop it will produce maybe 20 ripe figs. Only a few "buds" will drop off at the early stages of fruit growth. We lose more to squirrels and birds (when ripe) than buds dropping off.

The problem is the second crop. After the first crop of a couple dozen, it will get almost twice as many buds for the second crop. Soon after, they just start falling off. We end up with just a couple ripe figs at the end.

What can I do to keep more of the figs the second time around?