Hell, Japanese maps, up until maybe a few years back, used a similar symbol to denote temples. Pretty sure they might have changed the map symbol because of foreigners, but don't quote me on that because I could be wrong.

There was even a pokemon card that had to be changed for the international releases because it used the symbol. Cannot remember which, though.

卍 ( manji ) is the symbol in question. Easily accessible through just typing まんじ ( manji ) into my Japanese keyboard on my phone and selecting the first option.