Benchmark tool ``Speedometer 3.0'' developed jointly by Apple, Google, Microsoft, and Mozilla is now available

Speedometer 3.0 , a benchmark tool developed in collaboration with technology giants such as Apple, Google, Microsoft, and Mozilla, has been announced.

BrowserBench.org — Announcements
https://browserbench.org/announcements/speedometer3/

Improving Performance in Firefox and Across the Web with Speedometer 3 - Mozilla Hacks - the Web developer blog
https://hacks.mozilla.org/2024/03/improving-performance-in-firefox-and-across-the-web-with-speedometer-3/

Speedometer is a benchmarking tool that tests the responsiveness of browser web apps by measuring the timing of simulated user interactions. The first version of Speedometer was released by the WebKit team in 2014, and since then it has become a valuable tool for users and developers to create richer, smoother online experiences.

The latest version of this Speedometer, 'Speedometer 3.0', has been announced. Speedometer 3.0 is developed by three major browser engine developers: Google, which develops Blink , the browser engine for Google Chrome, Mozilla, which develops Gecko , the browser engine for Firefox, and Apple, which develops WebKit, the browser engine for Safari. It is born as a collaborative effort.

Speedometer 3
https://browserbench.org/Speedometer3.0/

Speedometer 3.0, like its predecessors, is designed to measure the responsiveness of web apps by simulating user interactions with real web pages. Speedometer 3.0 introduces a better way to measure performance and a more representative set of tests that reflect the modern web environment.

Speedometer 3.0 is supported by each major browser engine, including Blink/V8, Gecko/SpiderMonkey, and WebKit/JavaScriptCore, and this is the first such cross-industry collaboration. It is being developed based on a new consensus-based governance model, and you can also contribute to the development from a shared repository on GitHub.

GitHub - WebKit/Speedometer: An open source repository for the Speedometer benchmark
https://github.com/WebKit/Speedometer/

Speedometer 3.0 adds several new tests. To design the new benchmark, the developers started by identifying several key scenarios and user interactions that were considered important for optimizing the browser.

The new tests added this time are canvas and SVG chart rendering (React Stockcharts, Chart.js, Perf Dashboard, Observable Plot), code editing (CodeMirror), WYSIWYG editing (TipTap), and reading news sites (Next.js). etc. is simulated.

The TodoMVC tests have also been improved, with the code updated to adapt to the most popular versions of the most popular frameworks based on data from the HTTP archive. The frameworks and libraries that have been updated this time are as follows.

・Angular
・Backbone
・jQuery
・Lit
・Preact
・React
・React＋Redux
・Svelte
・Vue
・Vanilla JavaScript implementation and web component version targeting ES5 and ES6

More complex versions of these tests are also being introduced. It is embedded in a larger DOM tree with many complex CSS rules that more closely emulate the page weights and structure of popular web apps.

Together they run a broader and more representative cross-section of browser engines, providing new opportunities to optimize JavaScript, layout, CSS, graphics, and DOM APIs to improve the user experience on the web. Masu.

Additionally, the Speedometer 3.0 test runner itself has been improved to measure more work that the browser performs in response to user actions, such as painting and asynchronous tasks. Speedometer 2.0 measured the time the test script ran synchronously as 'sync' time, and the additional work before the 0 second timer fired as 'async' time. However, this seems to result in the loss of some work that the browser engine needs to do to update the rendering of the web page.

Speedometer 3.0 now measures rendering efforts that were previously missing, creating even more opportunities to optimize real-world content. For this I measure the test script inside the requestAnimationFrame callback as the 'sync' time and fire a 0 second timer scheduled in the second requestAnimationFrame as the 'async' time. This asynchronous time is guaranteed to include not only page rendering by the browser engine, but also work by the test's own timer. This will significantly improve benchmark accuracy and allow the browser engine to optimize tasks that were previously lacking, leading to significant improvements for users.

Other improvements include improved developer tools to help browser engineers better understand results, profile, and customize tests. The test runner architecture has been redesigned to make it easier to create and maintain complex test cases. There are also many code quality improvements and migrations to modern features that were not widely available when Speedometer 2.0 was released, such as native promises, async/await, classes, and modules.

The main goal of Speedometer 3.0 is to reflect the real-world web environment as much as possible, so users can benefit when browsers improve their benchmark scores. The Speedometer 3.0 development team said, 'Speedometer 3.0's goal has already achieved some success before its general release. Throughout 2023, core optimizations for each major browser engine will improve user responsiveness across the web. ”, emphasizing that the results of Speedometer are already being seen.