The Xai lies on Grook 3 Benchmarks?

Debacts above AI Benchmarks – and how are you signed by Abs – spread in public view.

This week, an opename employee accused ELON MUSK’S MUSK’S MAILY TAKE, TO PUBLING IN BENCHASK RESULTS FOR HERE PIECE AGO, GROOK 3. A BABUSBBING IGOR BABUSBBING, insist that the company was on the right.

The truth is found somewhere between.

In a Post on Xai BlogThe company has issued a graph showing Groking Grok 3 2025, a collection of math questions challenge by a recent mathematics examination of mathematics. Some experts have The validity of the self-quoted as a benchmark AI. I am However, AIME 2025 and older versions of the test are commonly used to survey a model capability of the model.

The chart of Xai demonstrated two variant 3, groof 3 reasoning Beta and Groot 3 mini reasoning the best professor available o3-mini-heighton the AIME 2025. Has employees to x are quick to the Xai graph of Xai has not included AIMI-High-High-High-High-High

What is consumption @ 64, you can ask? And well, it’s short to “consensus @ 64,” and become basically 64 templates to answer every problem in a benchmark and take the final answers. As you can imagine, COMPLETE @ 64 tends to the boent patterns of models, and the omentes it can make a methity that is in the reality, that is not random.

Grook 3 Reasoning Beta and Grok 3 Double For Aime For Aime 2025 to “@ 1” – The first punctuation has hit the score of the minimum. Grof 3 reasons blunt also the always-so-so-slightly behind the opening pattern o1 established in “medium” computation. However xai is Advertising groking 3 as the “you smarter of the world.”

Babuushkin Argue on x Let the Opena publish the AppleChmark Charts in the past – Albeit Charts who speak the performance of their models. A more neutral party in the debate puts a “accurate” graphics that shows almost any provision of the model in SS @ 64:

Hilarious as some of my plot as attacking attack and others as attacking groof while actually propaganda
(I really believed that Grook is good, and of the TTC of Attempting you behind O3-mini- * -Pass @ “” 1 “deserve more discovered.) https://t.co/djqljpcjh8 pic.twitter.com/3Wh8foun

– Teaturates ★ St (DeepSep Twitter polleds 2023 – ∞) (@TTertaxex) February 20 2025

But as ai searches Nathan Lambert indicated in a placeMaybe the most important metric remains a mystery: the computational cost (and monetary) took for each model to get their best score. That only go to show how much the most benchmarks more you communicate on models limitations – and their forces.