Size has much more to do with diet and environment than genes. It's not uncommon for societies where food is scarce to feed boys more than girls. In some places fat wives are prized as an external indicator that her husband is wealthy. These relationships change with place and time, and faster than genetic selection could possibly act.