The mobile SoC that Intel wants to kill ARM performs well on benchmarks, but give it a pass until its rough edges are worked out.

Intel Bay Trail Benchmarking Workshop Matt Dunford 3 1024x682 Intels Bay Trail problem

Intel’s Matt Dunford talks about benchmarking Bay Trail while in Taipei.

Intel would very much like ARM to have a Bay Trail problem, but Chipzilla’s mobile platform seems to very much have an ARM problem.

Intel’s Bay Trail SoC is a promising platform, but it’s real-world performance does not yet match the grandiose promises Intel made when it was first announced. For a platform that is supposed to take on the mature and established ARM ecosystem anchored by Qualcomm’s Snapdragon, and keep off other mobile x86 platfroms like Kabini and Temash from AMD (which admittedly have a non-competitive power envelope),  Intel will need more than buzzwords and marketing slides in order win the trust of manufacturers and customers.

While benchmark testing proves the platform is potent in that regard, there is more to assessing a platform than relying on one style of quantitative measurement. There are a plethora of compatibility issues,  like webGL on Android and driver and codec incompatibility on Windows, that need to be addressed when building a processor a mobile environment. While Intel may list attractive marketing bullet points like “64-bit” or “improved gaming performance” while also promising compatibility with both Android and Windows, without fleshing out support for the requisite APIs and getting key drivers approved Intel’s great mobile hope will fail.

To be clear: Bay Trail and Intel’s mobile ambitions both show promise. At Intel’s traveling benchmarking roadshow that came through Taipei Thursday, Intel showed off the PC side-scroller Defense Grid, Torchlight, and Batman: The Dark Knight (this was on both Windows and Android). The gameplay was fluid, had the full roster of effects were present and overall it seemed to work well — for a demo.

Intel Bay Trail Benchmarking Workshop photo 1 1024x682 Intels Bay Trail problem

Putting it through its paces

During the benchmarking session in Taipei we were able to run some Windows benchmarks on a reference device and Asus (currently shipping in parts of Asia and Australia) Transformer T100. Running CPU and GPU intensive benchmarks on a tablet is only of limited use since they don’t reflect what an average person might do with the device; most users aren’t going to try and transcode 1080p video or crunch numbers on an Atom Z3740. Regardless, it’s an effective way to quantitatively measure speed in comparison to other processors and use that along with other factors — like screen quality, touch latency, connectivity and price — when assessing a mobile platform.

The  Asus T100 we were provided came with a mid-range Z3740 version of the chip with four cores clocked at 1.34GHz and 2GB of memory. It should be noted that benchmarks were run in 32-bit mode, as Intel claimed the 64-bit version of the SoC’s driver had yet to be certified with Microsoft (more on that later). Intel wanted to media to use the Principled Technologies’s WebXPRT benchmarking suite, which is known to be a benchmarking platform “friendly” to Intel’s processors. In light of AnTutu-gate, it’s always best to take any suggestions of benchmarking tools by manufacturers critically. Plus, WebXPRT is a browser-based benchmarking suite and there’s the possibility of wide variances between browsers.

The reference system tested had a Z3770 version of the chip with four cores clocked at 1.46 GHz and 2GB of RAM.

For the Windows test we ran Cinebench, Geekbench, x264 HD Benchmark 5.0.1, and 3DMark 11 in its performance mode:

ModelCinebench Open GLCinebench CPUGeekbench Single CoreGeekbench Multi Core1080p transcode x2643D Mark 11
ASUS Transformer T1006.020.7976220973.98P200
Reference Design (4 cores at 1.47) Valleyview5.841.4596630643.36P209

For the Android test we ran AnTuTu and GFXBench 2.7.2. Only a reference machine was available to use.

AnTuTuAnTuTu 3D rating benchmarkTRexHD C24Z162.5 Egypt HD C24Z16
Android Reference3606752024794 frames 14 fps3432 frames 30fps

Premature Birth 

Eventually Intel will have to ask themselves: what’s the point? As benchmarks show,  Bay Trail is a technically promising platform, but its execution thus far has almost been relatively lackluster.

Bay Trail’s problem lies in the fact that the tablet OS it works well on and timed to launch alongside of, Windows 8.1, is a terrible platform for touch-centric devices. Windows 8.1 is a desktop OS; it’s touch UI that was once known as Metro is latent and unintuitive compared to iOS or Android and using “desktop” mode is impossible without a mouse or keyboard. The tablet OS that it’s not ready for yet is the best platform for it — Android. It could very well be that Android never sees a proper birth on the platform because of lobbying from the Win side of the Wintel alliance, or lack of interest from OEMs in putting another Android tablet out in to the already crowded market.

There are, as Intel’s Matt Dunford put it, still some “quirks” in the implementation.

Next up is Bay Trail’s 64-bit strategy. While Intel representatives, when asked, maintain that Intel is waiting on Microsoft to certify the necessary 64-bit drivers, slides seen by VR-Zone show that this isn’t exactly the case. In February, likely at Mobile World Congress, Intel is planning on launching a 64-bit version of Bay Trail aimed at business users. But unless this 64-bit edition is paired with more than 4 GB of memory — which many Bay Trail tablets don’t have — the extra memory address space will be useless given the lack of RAM. 

So what does Intel do with Bay Trail? There’s definitely more room for competition in the mobile sector, and on paper Bay Trail can benchmark better than any of the leading ARM chips out there. Might it be that Bay Trail is simply an experiment to test the waters for the upcoming 14nm Cherry Trail? Gain experience and produce a few hardware wins, despite the venture not being overly profitable.

It’s all about implementation and earning design wins, and so far it’s been rough. Bay Trail’s victories might as well be pyrrhic, as Bay Trail leads in Windows compared to the non-existent market share of the Surface and AMD’s mobile chips while every Android tablet shipping this holiday season is ARM-based. But it’s not for lack of a gargantuan effort that only Intel could pull off.

“Our engagement with Google is very broad indeed,” The Register reported Intel executive Doug Fisher saying at IDF in September. “We have well over a thousand engineers working across Android and Chrome to bring these great devices to market.”