Your example is fine. It's not an option for most people to be able to source everything they need where the supply chain doesn't rely on exploitation of labor and resource extraction somewhere along the process. Whether it's in the materials used by the producer, the logistics in goods transport(oil & gas, shipping, warehousing, etc), or manufacturing of goods (labor, production house, etc...), the chain relies on exploitation somewhere along the way.
Buying the apple from your local farmer still relies on the oil industry to run farm equipment or transit of the product. By and large this is minimize when buying local from independent producers. Those, however, are often more expensive.
The consumer is the last step of exploitation. Where the profit stems from. Again, your local farmer probably isn't extorting you, but your local grocery store probably is.
Sure regulate it then, effectively, and cut out any international trade which allows for the procurement of goods through a non-regulated system. Find a common agreement of what regulation is and find a way to ensure no one has an incentive to break that regulation. Find a way to ensure that a capitalist cannot accrue such wealth as to influence politics (or skirt whatever law you want against it).
The only people I've heard who want NO regulation are AnCaps and yeah lol no - (and also they still want property rights ig).
People just want a better system than what we have now. Which is laissez-faire capitalism. Beyond that there's the political philosophical aspect that "capitalism" can and does effectively mean rule by the wealthiest.
Lot's of people would be happy with a market system in a socialist society. Lots of people want Anarchism. Lots of people want Communism(the good or bad kind depending on who you ask).