I do have to agree with you. It's a demand issue, combined with the rate that it is being built.
I think it would be fully okay to have the cost of a house be multiplicative, based off how many you own after the second house. So, if you are looking to buy a third house, the price is actually double than what you would be expecting, and then your fourth house would be triple what you're expecting, etc.
The price of the house will be the cost the seller lists it as, and the seller gets the cost that they listed it as. However, anything over the cost it was listed, if you are affected by that multiplier, has that money go straight into the government.
The reason I say after the second house is because I do personally think it's fair for someone to want a vacation home somewhere, I just don't think it's fair for someone to have an interest into having a second, third, fourth, or sometimes even fifth vacation home. And there are many cases where you would want to own a second property such as if you run a self-employed job such as a computer repair specialist, you will probably own a shop, and having a office that's at twice the cost of what it normally would be is extremely cost prohibitive to someone wanting to get into the market.
I also firmly believe that there should be quite extortionate taxes for buildings that are labeled residential but have an owner who primarily rents it out for more than a quarter of the year. The whole Airbnb our property is technically zoned residential but are actually commercial loophole is absolutely ridiculous and a huge problem. If you aren't allowed to put a motel or a hotel there, you shouldn't be allowed to have an Airbnb there.