Tuesday, December 22, 2009

Windows 7 vs Ubuntu 9.10 - 3D Benchmarks

In the past I've done benchmarks regarding Wine software, today I am going to do something a bit different. Unigine is a cross-platform real-time 3D engine, I stumbled across awhile back on some message boards I am a part of. Since it runs natively cross-platform I have been curious to see exactly how drastic the performance difference of the engine is between the Windows and Linux platforms. Since I recently installed Windows 7 I decided to sit down and put the software through its paces.

The Tests: Unigine offers two free benchmarks - Tropics and Sanctuary. I ran both demos using OpenGL (because OpenGL runs on both platforms natively and DirectX does not).

The Hardware: While my hardware is not fastest in the world it is (as of posting this) relatively new and decently quick. Processor - Intel p9700 2.8ghz Dual Core, RAM - 4gigs of DDR3, Video Card: nVidia 260m with 1gig DDR3 dedicated memory (Running the latest stable nVidia driver on both Linux (190.42) and Windows (195.62)).

The Software: Windows 7 Ultimate 64bit, Ubuntu 9.10 (Karmic) 64bit, & Unigine Benchmarks

The Results:
Tropics -
  • Windows
  • - 1680x1050: 30.5 fps
  • - 1024x768: 44.8 fps
  • - 640x480: 59.8 fps
  • Ubuntu
  • - 1680x1050: 14.9 fps
  • - 1024x768: 23.6 fps
  • - 640x480: 30.9 fps
Sanctuary -
  • Windows
  • - 1680x1050: 35.2 fps
  • - 1024x768: 61.9 fps
  • - 640x480: 99.9 fps
  • Ubuntu
  • - 1680x1050: 17.3 fps
  • - 1024x768: 27.9 fps
  • - 640x480: 48.9 fps
Wrapping Up: The numbers speak for themselves. In the realm of 3D graphics it is clear that Ubuntu still has a long way to go if it ever wants to meet (or possibly beat) the performance Windows has. In all of the above tests Ubuntu scored between 48% and 55% lower frame rate than the same benchmark on Windows.

Cheers,
~Jeff Hoogland
Please note while these benchmark scores presented are accurate to the best of my abilities, they only represent my personal hardware and software configurations. Your results on your own system(s) may vary (and if they do, please share them!).

41 comments:

  1. Tell us a little bit about the details of the rendering stack you're using? On Karmic, it seems likely that you'd be using Xorg 1.6.4-2. What nvidia driver are you running with? (dmesg | grep NVIDIA)

    ReplyDelete
  2. Oh, sorry I'll add my driver version to the post. Currently using the 190.42 driver on Ubuntu (the 195.22 beta driver caused stability issues for me)

    ReplyDelete
  3. Did you have the composite extension enabled? I remember getting better performance with it disabled... The viewport performance of professional 3D programs has always been superior on Linux for me but if you are into gaming then Windows is the obvious choice.

    ReplyDelete
  4. @Last Poster - Yes I had desktop effects enabled in Ubuntu when I ran the tests - but I also had Aero running on Windows when I ran them.

    ReplyDelete
  5. are these with the closed (NVidia) drivers or the open (XOrg, Neauvou) video drivers?

    Also, what is a Tropics or a Sanctuary?

    ReplyDelete
  6. Closed source nVidia drivers - on Linux the 190.42 where used and on Windows the 195.62 where used.

    Tropics and Sanctuary are the names of the two benchmarks provided by the Unigine engine. Check the link under "the software" - its a free download.

    ReplyDelete
  7. For comparison, I'd like to see the DirectX numbers on Windows as well.

    ReplyDelete
  8. @Last Poster - I plan to do an OpenGL vs DirectX comparison on Windows later this week in another posting, check back later for it :)

    ReplyDelete
  9. I'd really like to see these benchmarks with desktop effects turned off on Ubuntu. Also, a 32-bit test (Ubuntu). I think it would make a difference.

    ReplyDelete
  10. First, "Great Sales Today" is obviously spreading Spam Link, so you might consider deleting at the least the link to the site.

    It's nice to see someone actually capture some numbers and present them logically.

    I don't think these particular numbers should discourage someone from trying Ubuntu. On the contrary, the posting suggests it is at least as easy to set up as Windows, and you get adequate 3D throughput for any practical task.

    However, if gaming is very important to you, probably sticking with Windows is a good idea (though I'm not sure why you would pay to upgrade to Win 7 to play games, but still).

    ReplyDelete
  11. I tried to do the benchmark in my computer and the results were the following:

    LINUX with Compiz:
    Unigine
    Tropics Demo v1.2

    FPS: 17.3
    Scores: 436

    Hardware--
    Binary: Linux 32bit GCC 4.3.3 Release May 26 2009
    Operating system: Linux 2.6.31-17-generic x86_64
    CPU model: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz
    CPU flags: 166MHz MMX SSE SSE2 SSE3 SSSE3 SSE41 HTT
    GPU model: GeForce 9700M GT PCI Express 190.53 512Mb

    Settings--
    Render: opengl
    Mode: 1280x1024 fullscreen
    Shaders: high
    Textures: high
    Filter: trilinear
    Anisotropy: 4x
    Occlusion: disabled
    Reflection: enabled
    Refraction: enabled
    Volumetric: enabled

    Windows 7 With Aero
    Unigine
    Tropics Demo v1.2

    FPS: 18.2
    Scores: 459

    Hardware--
    Binary: Windows 32bit Visual C++ 1500 Release May 26 2009
    Operating system: Windows 7 (build 7600) 64bit
    CPU model: Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz
    CPU flags: 65MHz MMX SSE SSE2 SSE3 SSSE3 SSE41 HTT
    GPU model: NVIDIA GeForce 9700M GT 8.17.11.9562 512Mb

    Settings--
    Render: opengl
    Mode: 1280x1024 fullscreen
    Shaders: high
    Textures: high
    Filter: trilinear
    Anisotropy: 4x
    Occlusion: disabled
    Reflection: enabled
    Refraction: enabled
    Volumetric: enabled

    As you can see the diference is not that much and linux has compiz enabled that is way better than aero in my opinion.

    I did the tests also in linux without compiz and in windows without aero.The results in windows were the same and in linux it increased 1 fps.

    As you i have 64 bit version and the updated video drivers.

    ReplyDelete
  12. @Stt@L Thats really interesting, I wonder if my bottle neck on Linux is a limitation in the nVidia drivers for my particular graphics card then (mine is a bit newer hardware than what you are running). I was not aware the 190.53 drivers had been released for Linux - I'll load them up on my system and see if they yield any difference for me.

    Also regarding Compiz - I also agree that compiz crushes Aero but at the same time toggling it off for a single fps or two is kinda goofy.

    ReplyDelete
  13. Yeah i agree i just turned it of to test it cause compiz its realtime it could make the diference.

    If you are using ubuntu just add nvidia repos to your system because they aren't in the ubuntu or canonical repositories.

    ReplyDelete
  14. I manually check the downloads site now and again because I use the beta drivers off and on. At any rate I just got a new harddrive for my Sager (640gb w00t!) and I will have a fresh install of Ubuntu with the latest drivers to try out later this evening or sometime tomorrow :)

    ReplyDelete
  15. Regarding compiz & aero, it should make very little difference. Was this run in full screen or windowed mode? If full screen, it should have 0 impact on perf, if in window mode, you should only have extra 1 window sized copy per frame in both cases. Considering all the game is doing, the extra copy shouldn't be a big portion of the cost.

    ReplyDelete
  16. All the tests where done full screen.

    ReplyDelete
  17. Offtopic: Nice blog last night i've been looking around and i found some realy interesting stuff in here.
    Realy good job congrats.

    ReplyDelete
  18. Thanks Stt@L :)

    Also I just got around to running the tests with the 190.53 drivers on Linux - I saw a slight increase but nothing sizable (for instance my tropics FPS went from 14.9 to 15.8). I'm curious as to why I see such a large difference between the two platforms and you do not. I'm going to see if I can get a hold of another dual booted system with a decent gfx card so I can get a third set of data to see which is the odd one out.

    ReplyDelete
  19. Just i thing i've noticed Since your card is 260M im guessing that you are using a notebook as mine. Did you checked you power setting in windows and ubuntu??
    In my test's they are all made in high performance settings.
    I dont know just in case.

    ReplyDelete
  20. Yep. Everything was on full force on both operating systems. Hopefully I'll get test results within the next week or some from another system to compare with.

    ReplyDelete
  21. I somehow had the "feeling" that Linux was slower than Windows. I tried to confirm your observation, alas...

    Sanctuary Benchmark, 1024 x 768, Win 7 64bit, Ubuntu 9.10 64bit, proprietary driver, Compiz and Aero enabled, NVidia 9600GT, Phenom 9950 Quad @ 2.9Ghz yields
    Win 7: 55,3FPS, score 2346
    Karmic: 54,3FPS, score 2303
    i.e. more or less identical figures.

    Gregor

    ReplyDelete
  22. Staying within the Linux realm:

    As above - windowed + Compiz on:
    54.3FPS/2303
    fullscreen + Compiz on:
    55.9FPS/2372
    fullscreen + Compiz off:
    60.5FPS/2566

    Seems as if Compiz comes at a cost.

    Gregor

    ReplyDelete
  23. Thats really odd, seeing as how you are the second one to post scores that show about the same between Windows & Linux I'm slightly at a loss as to why my setup produces such skewed results.

    Also for instance when I toggled compiz on/off on my system the difference was less than 2 fps.

    ReplyDelete
  24. I just repeated the procedure on Win 7.
    Aero off, OpenGL:
    58.3FPS/2471 windowed
    58.7FPS/2489 fullscreen
    (BTW fullscreen turns Aero automatically off)
    Comparing the "best cases" (fullscreen, compositing off) means that Karmic is even slightly *faster* (60,5 vs. 58.7FPS) than Win 7.

    However, DX9 in turn is about 10% faster than OpenGL.

    Have you tried other benchmarks from the Phoronix Suite? Plenty of other results to compare with:
    http://global.phoronix-test-suite.com/?k=category&u=unigine

    Gregor

    ReplyDelete
  25. Lol guy… Why did you compare results on miscelleaneous drivers? You shall do test on the same driver. Its normal that, the newest drivers are quiet faster than olders.

    ReplyDelete
  26. @Mariom As of the test date those are the latest drivers for both Windows and Linux - obviously the different operating systems have different driver version numbers...

    ~Jeff

    ReplyDelete
  27. Mariom... Not necessarily.

    The Windows driver was a 195.xx (it is a newly set up and updated system) whereas the Linux driver was a 190.53.

    Anyway, we are talking about some minuscle differences. The overall finding is, that OpenGL on Linux is just as fast as on Windows on my machine (when using the proprietary drivers).

    Gregor

    ReplyDelete
  28. Something is wrong with the combination of Ubuntu+NVidia.
    http://www.phoronix.com/scan.php?page=article&item=fedora12_ubuntu910&num=1

    ReplyDelete
  29. Phoronix shows rather different results though...
    http://www.phoronix.com/scan.php?page=article&item=linux_windows_part1&num=1

    ReplyDelete
  30. Yes, and several of other posters on here showed results similar to Phoronix, however as I said my results where triple checked on my hardware - and where the same each time.

    Perhaps I will re-run these benchmarks now that Ubuntu 10.04 is out of beta and installed on my main system.

    ReplyDelete
  31. Update - Phoronix also depicts Windows beating Ubuntu by a sizeable degree here - http://www.phoronix.com/scan.php?page=article&item=linux_windows_part3&num=3

    ReplyDelete
  32. @Jeff: open your eyes, dude!
    Ubuntu runs on INTEL-gpu
    The Windows on NVidia GPU

    It depicts nothing.
    There's by NO means a FPS-loss with similar hardware running OpenGL, whatever OS you're running. Provided you've installed the correct drivers, properly. Which, on Ubuntu, might not be as monkey-proof as it is on Windows.

    ReplyDelete
  33. @Oscar did you even read my article? The tests where done on the exact same system (dual booted) both with the latest nVidia drivers installed.

    Anyone who tells you drivers and operating system are meaningless regarding OpenGL performance is plain wrong.

    ReplyDelete
  34. Hi Jeff is it possible for you to please run the benchmark again fresh with updated drivers and more recent Unigine demos and upload the result for both the OS benchmarking and the OpenGL and Direct3D benchmarking. I am currently writing a report which I'm referencing your site in, it will look bad if the conclusion for the article does not correlate with the ones in the comment... I would do this myself but I don't have the right graphics card to run the benchmarks... I'll really appreciate it if you can do this.. thanking you in anticipation.

    ReplyDelete
  35. @Anonymous Sure. Will be at least a week or so though - the only hard drive in my laptop is currently has a GPT partitioning layout - meaning I need to do a clean install of everything to get Win 7 reinstalled

    ReplyDelete
  36. Thank you very much I look forward to it

    ReplyDelete
  37. hi Jeff,

    I would like to know roughly when the new data will be published. thanks

    ReplyDelete
  38. Just ran the tests yesterday. Should have it formatted nicely and ready to post by Tuesday. Sorry for the delay.

    ReplyDelete
  39. Hi jeff is the new data going to be released as a new post?... thanks for your help

    ReplyDelete
  40. The new Windows data can be found - http://jeffhoogland.blogspot.com/2010/09/does-windows-7-areo-slow-down-your-3d.html - just compare it to the Linux benchmarks in my other recent posts.

    ReplyDelete
  41. The use of Wine would be the largest bottleneck in this case. It slows everything down greatly, because it has to go through so much more code to do the same thing. Running natively, both tests should run at about the same speed.

    I can run Steam, and the games I have on it, easily on a five-year-old Pentium 4 with an NVIDIA FX 5500. Steam just barely chugged along on a new dual-core Celeron, and the games were unplayable with a newer GPU (an Intel 915, if I'm not mistaken). Wine is terribly inefficient as a wrapper, and would produce incredibly skewed results.

    ReplyDelete