Basemark ES 3.1 User Guide

Basemark ES 3.1 User Guide
The 4th generation of the world’s most popular OpenGL ES benchmark for graphics
performance comparisons of OpenGL ES 3.1 enabled devices.
Follow us on social media
https://twitter.com/basemarks
http://www.facebook.com/Basemarks
http://www.youtube.com/rightwareoy
http://www.linkedin.com/company/Basemarks
1
Contents
The Product ....................................................................................................................................................... 3
Rightware’s Competence .................................................................................................................................. 3
System Requirements........................................................................................................................................ 4
Before Testing.................................................................................................................................................... 4
Technical Description ........................................................................................................................................ 4
Official Results & Scoring ................................................................................................................................... 6
Comparing the Scores – Power Board ........................................................................................................... 7
Custom Tests ..................................................................................................................................................... 8
Data Logging ...................................................................................................................................................... 9
About Rightware.............................................................................................................................................. 10
2
The Product
Basemark ES 3.1 measures graphics performance in rendering a 3D game like scene in OpenGL ES 3.1
enabled devices. One of the ground breaking new features in Basemark ES 3.1 is the ability to run custom
tests by toggling various graphics features on or off. This innovation enables users to measure how specific
graphical effects tax the device and to detect performance bottlenecks.
Basemark ES 3.1 has three different version: Free, Professional and Corporate. The following table describes
the main differences between the versions:
Business usage allowed
Available at Google Play
Overall score on UI
FPS information on UI
Configurations on UI
Results data file output
Forced results upload
Test automation
Source codes
Free
No
Yes
Yes
No
No
No
Yes
No
No
Pro
No
Yes
Yes
Yes
Yes
HTML
Yes
No
No
Corporate
Yes
No
Yes
Yes
Yes
HTML + XML
No
Yes
Yes
Rightware’s Competence
Rightware team has the industry’s longest and deepest experience of designing and implementing
trustworthy benchmarks for smartphones and other embedded devices. Rightware was born out of
Futuremark in a trade sale back in 2009 and the core team has been developing mobile phone benchmarks
already since 2004.
Basemark ES 3.1 has been developed under rigorous process and scrutiny by Rightware Benchmark
Development Program (BDP) members. The BDP members consists of virtually every relevant
semiconductor and OEM of the world. See full listing of Rightware BDP members pictured below. The list is
also posted online at http://www.rightware.com/benchmarks/corporate/
Picture 1: Rightware BDP Members
3
System Requirements
System requirements for Basemark ES 3.1 is as follows

Android:
o Minimum OS version: Android 5.0
o Minimum OpenGL ES version: 3.1
o Minimum Required Free Memory Space: 80 MB
Before Testing
In order to get accurate and consistent benchmark results, you should pay attention to the following
instructions:


Close all the other apps that may be running in the background, and disable notifications before
running the benchmark.
Allow your phone/tablet to cool down before running the benchmark. Some high-powered mobile
devices use thermal throttling to avoid overheating the CPU or GPU, which can lead to lower scores
on successive runs. To reduce this effect, it is better to wait 20 minutes before starting another
benchmark run.
Technical Description
Basemark ES 3.1 contains the following Features:

Deferred Lighting with Alpha Blending
o Four render targets are attached to shaders in geometry stage, two are used for storing
positions, normals and material properties while the other two are used for decal textures
and semi-transparent objects, respectively
o There are 80 Lights in the scene
o A Stencil Test is used to optimize the lighting, ensuring that only areas affected by the
lighting range of each light object will be shaded
Screenshots showing contents of Deferred Shader’s G-Buffer: Normal, Depth, Color and Semi-transparent objects.


Water physics simulation using compute shaders
o Based on Fast Fourier Transform for wave propagation.
Screen space Ambient occlusion using fragment shaders
4
Screen Space Ambient Occlusion in action in the screenshot on the right








Indirect Rendering
Particle Effects
Normal Mapping
Multi-pass water reflection
Cube map reflections
Blur
Depth of field
Bloom shading
Screenshot Displaying the scene with (right side) and without (left side) Bloom Shading + Depth of Field

Screenspace anti-aliasing
Effect of Screenspace anti-aliasing (enabled only on the right)
5
Official Results & Scoring
Basemark ES 3.1 is designed as a professional GPU performance analysis tool. Therefore, the score
computation is derived from FPS, which is calculated as an arithmetic mean with the following formula:
𝐹𝑃𝑆 =
𝑇𝑜𝑡𝑎𝑙 # 𝑜𝑓 𝑓𝑟𝑎𝑚𝑒𝑠
𝑇𝑜𝑡𝑎𝑙 𝑡𝑖𝑚𝑒 𝑠𝑝𝑒𝑛𝑡
Total # of frames rendered is constant for all target devices running the test with the same configuration.
For on-screen and off-screen test runs, the number of frames consists of the whole scene and for each
feature test the number of frames is approximately 1/3 of the whole scene.
To get official test results, make sure you have enabled Official Test Settings from the Configuration
menu!
The official Main Score is based on the off-screen test run, because it eliminates the differences in screen
pixel sizes and the v-sync limit of 60 FPS, by rendering the test frames 36 times to an auxiliary buffer before
refreshing the screen.
The Main Score is also normalized according to a constant (𝐶) generated by our baseline devices and then
scaled such that the score range for the current top performing GPUs is near 1000. The formula for
calculating the official Main Score is:
𝑀𝑎𝑖𝑛 𝑆𝑐𝑜𝑟𝑒 = (
1080𝑝 𝑜𝑓𝑓𝑠𝑐𝑟𝑒𝑒𝑛 𝐹𝑃𝑆
) × 1000
𝐶
Official Results Page
Additionally, the official test result page provides four sub-categories: Lighting, Compute, Instancing and
Post-Processing. These tests measure the delta between the baseline measurement of running the scene
without any effects on and the performance hit you get by enabling a particular effect. The score of these is
6
expressed as a percentage where bigger is better. The theoretical maximum score is 100, which would
imply that there was zero penalty for enabling the particular effect.
The following image describes how these sub-category tests are constructed from the Basemark ES 3.1
features:
Comparing the Scores – Power Board
Only free version of Basemark ES 3.1 comes with
integrated online results comparison service, Power
Board http://results.rightware.com/.
The results screen features a link to Power Board.
Upon clicking the link, the Power Board web site
opens and displays the user’s score as well as scores
of other users that Rightware has received in its
database.
Power Board is a large database of benchmark
results, enabling visualization of device performance
in several key areas, such as gaming performance,
browser performance, and GPU performance. It also
allows consumers to make easy and objective
comparisons between devices with different
operating systems, including Android, iOS, and
Windows Phone 8, as well as phones from different
operators. Power Board collects testing results from
over 7000 smart phones and tablets using
Rightware’s free benchmarks, including

Basemark ES 3.1 for Graphics performance

Basemark OS II for Productivity performance

Basemark X for Gaming performance

Browsermark for Web Browsing performance
7
Custom Tests
Configurations Menu
Basemark ES 3.1 Professional version allows user to customise test configurations via the Configuration
menu. Please note that if you change any of the settings, then the score you obtain will not be the official
score. Only test runs with Official Test Settings on will display the official results screen. For all other test
runs, Basemark ES 3.1 shows a custom score page as depicted in the screenshot below. Only official scores
are sent to Power Board.
8
Custom Results Page
Data Logging
Basemark ES 3.1 professional version exports HTML report. This can be located from SD Card external
memory at sdcard/Android/data/com.rightware.BasemarkEngine/files.
The HTML exported file is a user friendly summary of the test that displays general system information, the
test configuration chosen and the results in Score, FPS and Elapsed Time.
Fragment of a HTML Exported File
9
About Rightware
Rightware is the market leader in providing industry standard benchmarks for trustworthy performance
analysis of processors, smartphones, and tablets. Its renowned Basemark® product family has been widely
adopted by semiconductor companies, device manufacturers, operators, independent technology media,
as well as consumers for various evaluation and development purposes. Rightware also develops the Kanzi®
UI solution for rapid 3D and 2D UI design and deployment. Rightware is headquartered in Espoo, Finland
and has offices in Silicon Valley, Detroit, Taipei, Seoul, Tokyo, Shanghai and Munich.
Company Details
Rightware Oy
Niittymäentie 7, 02200, Espoo, Finland
Telephone: +358-9-855-4322
Website: www.basemark.com
Follow us on social media
https://twitter.com/basemarks
http://www.facebook.com/Basemarks
http://www.youtube.com/rightwareoy
http://www.linkedin.com/company/Basemarks
Contact Us:
http://www.rightware.com/contact/
10