Discussion on Releasing a New Version of Unixbench

[wangxp006]

Hey everyone, I wanna start chatting about putting out a new version of Unixbench.
Our current 5.1.3 version has been around for more than ten years. In all that time, there've been big changes in hardware setups, operating systems, and the tools we use for compiling. To make sure Unixbench can show how modern systems really perform and keep being useful, we really need to release a new version.
Here's something to show how much it's improved: When we tested on an AMD 7763 system, the 5.1.3 version only got a score of 14,500, but the latest version on the master branch hit 85,068! That's a massive jump in performance, and it shows the old and new versions just aren't in the same league. If we keep using 5.1.3, we'll probably get results that are way out of date and won't tell us how a system truly performs.
I'm thinking we should call the new version 6.0.0. This version number isn't just some random thing; it's like a whole new beginning and a fresh standard for our benchmark tests.
Let's work together to make Unixbench even better. Everyone, feel free to share what you think about the new version and any ideas you've got!
@Zengkai163 @kdlucas

[gstrauss]

@wangxp006 Thank you for opening this discussion.

Back in 2016, I left this comment, which I think is still relevant:
#23 (comment)

Similar thoughts, again. This is not an exhaustive list:
#59 (comment)
#86 (comment)
#86 (comment)

I would support changing the version of Unixbench, along with some doc about what that means and what it does not mean.

I think it would be great to write some new benchmarks which demonstrate the power of modern systems. However, I caution against modifying existing benchmarks, which may be (improperly) used to compare results with older versions of the benchmark run against older machines. While I can not prevent marketing stooges from misrepresenting tool output numbers, I would like to better document that results of running Unixbench version x.y.z on Machine A are best compared to the same Unixbench x.y.z version results run on Machine B, both using the same operating system version and OS tunings. Comparing results from different versions of Unixbench are not comparing the same things ("comparing apples to apples").

[wangxp006]

Hi @gstrauss , I truly respect and understand your concerns.

Yeah, it's true that with the changes in the Unixbench code, there's really no point in comparing the scores across different versions.

The benchmark scores aren't just about the benchmark itself. They're heavily influenced by things like the operating system, the compiler version, and the optimization settings. Only when these aspects are the same can we actually have a proper basis for comparison.

I'll make sure to clearly mention these points in the documentation that's going to be released.

[wangxp006]

New version 6.0.0 has been released.