Housing became an investment so people bought more than one.
Second homes are far less of an issue than tenth homes or hundredth homes, what with the rise of rental property consolidation and AirBnB-ification.
Houses are twice the size they used to be. So they cost twice as much for that.
That's not how homes are priced, though. The real raw materials for construction are a fraction of the total cost. Which is why you'll see old ranch homes broken up into clusters of three-story townhomes the closer you get to growing downtown districts. Also, why you see increasingly shoddy construction jobs in new-builds, as compressed lumber and stucco replace concrete, steel, and hardwood. Nevermind the cost of labor, which is all over the map depending on whether ICE has decided to raid you recently.
Houses have a lot more regulations and codes they must comply with.
They've also become more cookie-cutter standard. Far fewer custom homes. Far more big developers who get a rubber stamp (or pay bribes to the right people) to roll out a thousand-unit development.
We build less houses now than we did in the 1970s.
We have less undeveloped real estate connected to highway access points than we did in the 1970s. And we have more Boomer-vacated housing re-entering the market.
The single biggest proximate cause to the housing price boom has been the ZIRP interest rate policy, flirted with after the Dot-Com bust of '01 and bought whole-hog after '08. We've been living in an era of low wages and cheap lending. This has resulted in a historic consolidation of private ownership under banks and hedge funds that have proximate access to the Fed credit window.
The end result is housing units being bought for a market rate and rented for a profit, until the market saturates and you have a sudden forced liquidation. These long drawn-out real estate famines punctuated with sudden windfalls reward people who just happen to be in the market at the right moment. But they otherwise stack the deck in favor of people with the most borrowing power.