It's a complicated problem. The builder wants to make as much profit as possible, so they're going to build what they think they can sell at the current market rate. If high end housing isn't selling and a bunch of mansions are all for sale with nobody buying they're not going to build more mansions, instead they'll shoot for lower cost housing that there's demand for. However that's still going to leave a bunch of people unable to afford a house if there are still other people in the market able to afford more expensive houses. This then gets even more complicated by people who don't actually need a house, but are instead purchasing houses as investments who then end up buying many of those new houses and artificially inflating the demand and therefore the prices.

At some point though, demand will be entirely met, at the high end and builders will be forced to move down market to keep selling property. Municipalities can help address some of this by zoning and requiring a decent mix of low, mid, and high income housing to be built, but once again "investors" and even worse investment companies can screw this up by snapping up the low income housing and converting it into rentals, or simply sitting on it for some period of time.

As with most problems that haven't been solved yet, there's no easy solution (because if it was easy to solve, it would be already).