ChatGPT can’t do basic math!
This is a bold statement – I know. But please bear with me.
I had a simple task to calculate a sum of some numbers. Usually, for something like this I would open Excell or Google Sheets, copy/past numbers in select them and click on SUM (Ʃ) button and I would get a result in a second. And this result was always correct. I never had any doubts and never had to double check it in any way.
Then, one day I had to do the same thing number of times. So I just used ChatGPT for this simple task. I would tell it to calculate the sum of the numbers and just copy/paste the nubers below and it would return me the result. I’ve done it number of times and I would take the result and use it as is. I never suspected that ChatGPT can’t just do this simple math calculation.
It was all fine (or at least I was under that impression) and I was happy to use ChatGPT to help me with this simple tasks. It felt so easy: just tossing a bunch of numbers to AI and telling him to return a sum. Until at one point I have noticed that ChatGPT sum result and the sum of these same numbers calculated by SQL SUM() function didn’t match! What in the World is going on!
This can’t be true!
How is this even possible??!?
I was in panic!
I have relied on these calculations and have used the results in projects not knowing these are not accurate!
This are the actual numbers I used when I noticed this:
3,861
4,750
3,719
4,768
6,478
7,597
8,553
2,442
10,515
1,507
1,926
3,731
4,966
3,133
6,819
8,903
11,935
3,838
5,935
4,471
1,929
1,462
8,482I asked ChatGPT to calculate me the sum of these numbers and it showed how it calculated it and the result it returned was 140,428
The sum calculated by SQL was 121,720.
ChatGPT was off! Way off! I still could not beleive it!
I have opened a Excell and have copy/pasted the numbers in it and the result was same as SQL:
I have opened Microsoft’s Copilot and have given it the same task (copy/pasted it from ChatGPT) and it has returned a different result: 112,918 – but to my supprise – it was also not accurate!
But at least it offerred links to two websites that can do online quick sum calculations to “verify the result” – indicating that the result may not be accurate something that ChatGPT didn’t tell me at all!
Basic Math Scoreboard:
Excel: 1
ChatGPT AI: 0
Microsof Copilot AI: 0
Okay, maybe my thousand separator is creating the confusion somehow. Let’s try another test this once again with AI using unformatted numbers:
3861
4750
3719
4768
6478
7597
8553
2442
10515
1507
1926
3731
4966
3133
6819
8903
11935
3838
5935
4471
1929
1462
8482So I have opened a new chat in ChatGPT and asked it to get me the sum of these numbers for me.
And once again I have got the invalid result: 153,185 – a number that’s even bigger than the previous result! Let me try this with Copilot and again I’ve got wrong number 119,035 – the closest one to the result but still invalid!
I have copy/pasted these numbers into Excell – and once again it didn’t had any trouble calculating the sum correctly.
Basic Math Scoreboard:
Excel: 2
ChatGPT AI: 0
Microsof Copilot AI: 0
This was getting out of hand! I’ve decided to make another test. This time I’ll go easy on the AI and I’m going to use less numbers (16 numbers this time) and all of them will be three-digit ones. Here are the actual numbers:
501
265
127
510
535
295
717
172
444
144
448
525
983
265
849
742Come on AI, you can do this!
Even I can accurately and reasonably quickly calculate this on my old Casio calculator.
Again there is some discrepancy in numbers between AI’s.
Excel, what do you have to say?
Once again Excel takes the win.
Basic Math Scoreboard:
Excel: 3
ChatGPT AI: 0
Microsof Copilot AI: 0
When this all took place I was using ChatGPT 3 (and the upgraded version was ChatGPT 3.5). As I was using it only ocasionally for simple things like this – I didn’t upgrade. I know it’s not the latest version but still it’s such a powerful tool that should be able to figure it out.
I have also tried asking it to calculate me averages of these numbers and all these were also not accurate: because in order to do so – it has to first calculate the sum – and then to devide that sum with the total count of numbers. The sum part was wrong every single time – but it returned the tight total count meaning that it didn’t leave any of the numbers out – it was just not capable of calculating the sum for some reason.
In the mean time, Sam Altman, the CEO of OpenAI a company behind ChatGPT said something like this “you’ll never have to use anything below version 3.5” and version 4 was released so the 3.5 became available to me.
It’s lot powerful and smarter than version 3 and I was eager to try it out but on the other hand I was affraid – what if – it fails again. Okay, let’s give it a try! Same 26 numbers, without thousand separator
So ChatGPT-3.5 says that the sum is 121.820
Pretty close – yes!
Accurate – no!
It has missed the result by 100!
Let me check these calculations one by one:
Lucky for me the error was made at 3rd operation so I didn’t have to check all of these!
For sure AI is amazing technology and it can do wonderful things.
But why it can’t do adding numbers right???
Why?
It’s not that difficult?
This simple test (that I didn’t plan or wanted to do in the first place) showed me that you need to be very careful when dealing with AI – as the results may not be accurate. Okay we kind of knew that already – but do we really need to question even simple things like this?!?! If that’s the case – what’s the use of it then? What is it good for if it’s not reliable?
At first, I wanted to record a Youtube video showing all these results, but since I’m no that good with video editing – I have decided to just made screenshots and add them to this post.
Do I want to upgrade to ChatGPT-4o and test it out? Well, yes and now. Yes because I want to see the technology advancing – and no as I’m affraid I’ll be dissapointed again.
Bottomline: if I have to check if AI has calculated correctly 12330+4768 – like it’s a third-grader – than just forget it! I’ll just use Excel / Google Sheets and I advise you the same!
The Final Basic Math Scoreboard:
Excel: 3.5
ChatGPT AI: 0
Microsof Copilot AI: 0