I love to show that kind of shit to AI boosters. (In case you're wondering, the numbers were chosen randomly and the answer is incorrect).
They go waaa waaa its not a calculator, and then I can point out that it got the leading 6 digits and the last digit correct, which is a lot better than it did on the "softer" parts of the test.
Idk personally i kind of expect the ai makers to have at least had the sense to allow their bots to process math with a calculator and not guesswork.
You are in for all kinds of surprises if you think the people shoving AI into everything are doing anything logical. They want everything to fit their singular model.
Maybe some of the smaller ones that have to run locally might, because they care about resource usage.