Ever notice how TV ads for new cars always show those vehicles on roads that are completely empty, even in the middle of a big city? There’s not a traffic jam, construction barrier or even another driver to be seen.
Same can be true for PC benchmarks. Just like those TV ads, they certainly look good. But also like those ads, what they show doesn’t always have much to do with reality.
Why does this matter? Because benchmarks, done right, measure how your customers’ PCs perform. And not in the lab, but in the real world.
But with so many benchmarks out there, how do you pick the right one? And how do you avoid the one that, like those TV ads, has little to do with reality?
First, you want a benchmark that’s based on executing real-world programs. Computer scientists who have looked into measuring PC performance say nothing else comes close.
What are these real-world applications? Everyday stuff. Browsing the web. Editing a Word document, Excel spreadsheet or PowerPoint deck. Collaborating with others over Teams, Zoom or Google Meet. Touching-up photos. Consuming media. Even playing games.
This mix has become even more important, with so many people now working and learning from home. Many people now rely on a single device for work, socializing, learning and relaxing.
Also, you want a benchmark that uses the software applications your customers are also likely to use. That means popular software such as Microsoft Office (with some 1.2 billion users), Chrome (approximately 5 billion users) and Adobe Photoshop (over 90% of the world’s creative pros).
You might think that’s obvious to benchmark providers, but it’s not. For example, one popular benchmarking tool, PCMark, tests photo editing not on Photoshop, but on an obscure program called ImageMagick.
Realistic vs. synthetic
Drilling down, you can divide common benchmarking tools into 2 groups: realistic and synthetic. You want a tool that’s realistic.
Here’s why: A synthetic benchmarking tool does not reflect real-world scenarios. Instead, these use averages that can vary over time, even day by day.
By contrast, realistic benchmarks show how a PC will perform in real-life scenarios. Typically, they do this by functioning as a “wrapper” for application tests. For example, the SYSmark benchmarking tool can tell Excel to calculate a formula, then measure how long it takes. That reflects pretty closely what an actual user will experience.
That said, some of these realistic benchmarks could be more relevant for your customers than others. For example, if a customer doesn’t use 3D image rendering, then a benchmark that does us it won’t be relevant for testing their PCs.
To help pick the most relevant benchmark, answer these 2 questions: What is my customer doing with their PC? And to do this work, which programs are they using?
What about convenience and speed, you ask? Sure, customers like these. But your job is to remind them of what really matters: accuracy and relevance.
There’s another way to get similar test results. You can run all the actual workloads yourself. That will give you accurate results, but it won’t be either quick or easy. In fact, it’s a lot of hard, precise work.
Reflecting reality may not be the best way to sell a car. But by helping your customers find benchmarking tools that reflect the way they actually use their PCs, you can help to keep them productive and happy.