Intel’s new performance tool casts doubt on benchmark scores
Summary created by Smart Answers AI
In summary:
- PCWorld reports that Geekbench will warn users about Intel’s new IBOT technology in Arrow Lake Refresh processors, citing concerns about benchmark accuracy.
- IBOT optimizes application code for better performance, but lacks clear documentation, making it difficult for benchmark providers to detect when it is active or verify the results.
- Although PCWorld testing showed consistent gaming performance with IBOT enabled, synthetic benchmark scores can be unreliable and incomparable to other processors.
Geekbench, a popular benchmark tool, says it will issue a warning when Intel’s new “Arrow Lake Refresh” desktop chips enable Intel’s new IBOT feature. For what? Because the reference provider cannot be sure that the scores reported with it can be considered trustworthy.
The new Intel Core Ultra 200S Plus desktop processors (also known as the Arrow Lake Refresh generation) aren’t too different from their 2024 “Arrow Lake” predecessors. But one of the key additions is the Intel Binary Optimization Tool (IBOT), which reorganizes the code in certain applications to run more efficiently, thereby improving performance. IBOT can also run on Core Ultra Series 3 (aka “Panther Lake”) chips.
The problem is that IBOT is not well documented… and Geekbench developer Primate Labs posted a short warning on their blog.
Essentially, if you test your Core Ultra 200S Plus PC or CPU using the Geekbench tool, it will issue a warning as a “hopefully temporary” workaround: “This benchmark result may be invalid due to binary modification tools that may be running on this system.”
IBOT ‘black box’ needs to be opened, says Primate Labs
Geekbench’s warning touches on the big problem at the heart of all benchmarking efforts. When you’re using a laptop or desktop computer, you don’t really have a quantitative measure of how fast or slow it’s running, so you can’t exactly compare performance with other machines. That’s what a benchmark does: it delivers a specific number that you can use as a comparative reference.
So what Primate Labs is saying is two things. First, the reference lab doesn’t know exactly how IBOT works, but they’re pretty sure they modify the reference code itself. This means that a Core Ultra 200S Plus chip is essentially running different code than an AMD CPU would run, at least according to Primate Labs.
“Because the tool changes the benchmark and it is not clear to Primate Labs and the general public how these changes occur, results generated with the tool are not comparable to results generated without it,” Primate Labs said.
Second, the Geekbench tool cannot see whether IBOT has been enabled or not, which also adds an additional level of uncertainty. Could a Geekbench score be better than the one reported by the tool? Primate Labs, for now, has no way of knowing.
Geekbench, a respected benchmark, is on the right track: Intel shipped the Core Ultra 200S chips without really defining what IBOT is or what it does, which prompted us to sit down virtually with Intel executives and ask them about it. Unfortunately, we had this conversation before Primate Labs released this statement. We have contacted Intel for comment and will report back on what was said. As Primate Labs indicated, I hope this is just a temporary communication issue.
Benchmarking a game should be a bit simpler, though. Our benchmark tests of the Core Ultra 200S Plus recorded scores with IBOT enabled and IBOT disabled for the small number of games supported by IBOT technology, and you should see the same results. But for the small number of synthetic benchmarks that could benefit from IBOT, it is possible that, for the moment, some results remain doubtful.
Still, the story of the Core Ultra 200S remains relatively unchanged: it’s slightly better than the original Arrow Lake, still behind AMD’s Ryzen 9000X3D chips, but significantly cheaper.
