Mobile Graphics Performance Benchmarking

Recent stuff

News, olds, and the like.

About the Software

MMark13 is a cross-platform benchmarking application for measuring mobile devices' CPU and graphics performance. It is built right on top of OpenGL ES 2.0 in native and portable C/C++, with additional thin layer for platform portability. In the design phase this was not just the software architectural goal but became the cornerstone of the whole software suite; to have the same exact code running the entire app on every single platform to provide results as comparable as possible. And it is free!

The test suite features four tests for measuring different things related to realtime 2D/3D graphics rendering. Once started, the tests are run consecutively and timing data is gathered. After the last test completes, the simplified score is shown on the device screen. After running the tests, you can submit your score to the MMark server; upon successful submission, a web page is opened in your device's browser to display further information about your device and your score.

Fractal: CPU Burn Test

This test measures solely the device's CPU performance. It allocates 4 concurrent threads which are used to calculate frames of the famous Mandelbrot fractal set. Each thread calculates every 4th scanline in the image; when all threads have completed the frame, the UI thread is notified and the new image frame is taken into use. The UI thread performs a 2D "zoom" into the rendered image while the next image is being calculated to create an illusion of seamless zooming at 60 FPS. Calculation of each pixel requires heavy complex number math. Naturally the devices with 4 or more cores will benefit most, while devices with a single core will have to do a lot of context switching.

The Mandelbrot set fractal was chosen because it is colorful and pretty and using it is a nod towards the demoscene of the 90s but most importantly because it is the perfect parallelization problem since each and every pixel can be calculated indenpendently from the others. The score from this stage is calculated based on the full Mandelbrot frames calculated, taking into consideration the amount of pixels calculated as well as the depth in the set. The CPU score is downscaled somewhat since the CPUs effect on 3D rendering is rather low.

Fillrate Tests

The fill rate tests are performed by rendering one frame a high number of repetition between glFinish() calls; the time spent is then measured and used as score for each test type. In other words, the textured quad is rendered several times on top of itself within a single frame to provide more accurate timing data than just doing it once.

First, the textured quad is rendered as is, with no lighting (Unlighted fillrate). Then it is rendered through a shader program that calculates the Gouraud-style lighting in the vertex shader (Vertexlighted fillrate); third test calculates the lighting in the fragment shader (Pixellighted fillrate). The final test calculates the lighting per-pixel in the fragment shader with additional specular component; normal/specular maps are also applied (Mapped fillrate).

Chess: GPU Basic Test

This test stage measures GPU performance with pretty standard techniques: multitexturing, per-pixel lighting, reflections and post-processing effects. The polygon count in the scene is not that high (around 1000-2000 per piece) but the post-processing depth-of-field effect applies a lot of load on the GPU. The effect is built with a 4-pass approach: first the scene is rendered into a texture, then gaussian blur is applied in both x/y directions and finally the blurred and original image are blended together using the depth value as the blending factor. The chessboard features planar reflections of the chess pieces; both the pieces and the board also reflect the environment. Everything is per-pixel lighted using the Phong illumination model.

As a curiosity, the game played out by the pieces is an actual game between chess grandmasters Garri Kasparov and Jan Timman in 1991 in Amsterdam. Visualized are the moves from 24th onwards. The game ended in agreed draw at move 29.

Mountains: Game Environment Test

The final test combines the aspects usually present in high-end games; large geometry, realtime shadows, physics, great viewing distances. The landscape mesh consists of about 120000 polygons, and the other models add up to another roughly 10000. The skybox-aligned sunlight is used as the source for lens-flares as well as shadow mapping. Lens-flare occlusion test is done every 100 milliseconds if the source is on the screen; shadow maps are calculated every frame even if no shadow caster movement occurs to keep the GPU stressed and the FPS steady. The shadow map size is 2k x 2k. The landscape is rendered by blending two separate ground textures together, blending factor based on the slope of the polygon. The trees/leaves are sorted and blended, and a alpha-based discard shader trick is used in their shadow map generation. The buggy model is tangent-space normalmapped and per-pixel lighted. The physics engine used is BulletPhysics 2.80, and physics calculations are done in the UI thread. The rigid bodies are the roman columns, the stone wall, the buggy and a planar "ground". Note that on purpose no landscape subdivision / frustum clipping is used to keep the GPU as strained as possible.

Scores

Scores / device information can be submitted to the backend database by tapping on the analyze button of the score screen, after the Full Test has completed running. An optional user name can be given. Upon submission, the backend uses GeoIP to detect the client's approximate location (roughly by city) and displays that information alongside the scores.

If the score submission fails, this could be (among other reasons) because:

Score page

Credits & Contact

All design & graphics & implementation by Matti Dahlbom. You can reach the author via e-mail: matti[at]777-team[dot]org.

Additional modeling (licenced under Creative Commons CC-BY or CC-0) credits as follows: Physics by Bullet Physics.

Download

See below for current availability on different platforms. For Android installation note that you will be prompted to install an utility app called Ministro unless you have already installed it; this will be used to deliver the Qt runtime libraries to the device. Also note that this software will not run on low-end devices as it requires a lot of RAM as well as resources from the display driver / hardware.

Future porting plans include Sailfish / Blackberry10 / Ubuntu Phone depending on their success. Windows Phone as soon as OpenGL support is available.

Android app on Google Play

The client software is and will be free of ad pollution. Neither will they cost anything, so be a good sport and visit a few Google Ads below (if any).