Ubuntu is a Dog with 3D performance...

[beastrace91]

I recently multi-booted the crap out of my main gaming rig with 9 different Linux distros so I could run 3D stress tests between them all and see how they perform. They all scored fairly close, except for Ubuntu, which was 10% lower than my top scoring distro.

The full statistics can be found on my website here (http://jeffhoogland.blogspot.com/2010/09/best-linux-distro-for-3d-performance.html).

Why do you think Ubuntu performs so poorly?

~Jeff

[snowpine]

Ubuntu has an extra "layer" of user-friendliness compared with its parent distro (Debian). This makes it easier to use at the expense of performance.

For example check out this comparision of Xubuntu and Debian Xfce to see how the two distributions' goals differ: http://distrowatch.com/weekly.php?issue=20090427#feature

The incredible success of Ubuntu indicates to me that many users will choose ease of use over maximum performance.

ps I appreciate the time you took to run the tests and share the results with others, thank you! :)

[Legendary_Bibo]

Ubuntu has an extra "layer" of user-friendliness compared with its parent distro (Debian). This makes it easier to use at the expense of performance.

For example check out this comparision of Xubuntu and Debian Xfce to see how the two distributions' goals differ: http://distrowatch.com/weekly.php?issue=20090427#feature

The incredible success of Ubuntu indicates to me that many users will choose ease of use over maximum performance.

ps I appreciate the time you took to run the tests and share the results with others, thank you! :)

That entirely did not make any sense.

[beastrace91]

I appreciate the time you took to run the tests and share the results with others, thank you! :)

:) Much to be girlfriend's dismay most of my spare time this past week has gone to configuring all those distros and running the benchmarks. XD

~Jeff

[cj.surrusco]

Good info, but did you take the Desktop Environment/Window Manager into account? That's a variable you would probably want to take into consideration.

[foxxxy]

Could it be the desktop effects? My computer runs slower when running multiple OpenGL programs at the same time, Desktop effects are OpenGL.

[fillintheblanks]

Hello chakra, goodbye ubuntu ):P

[beastrace91]

Could it be the desktop effects? My computer runs slower when running multiple OpenGL programs at the same time, Desktop effects are OpenGL.

Desktop effects where disabled.

~Jeff

[Anduu]

Desktop effects where disabled.

~Jeff

Have you tried with effects enabled?

Just an example in my case...QuakeLive performs way better if I leave compositing enabled.

[beastrace91]

Have you tried with effects enabled?



Desktop effects where disabled for all distributions. So whether or not they would give a boost across the board is kinda moot IMO

~Jeff

[EnGorDiaz]

Ubuntu has an extra "layer" of user-friendliness compared with its parent distro (Debian). This makes it easier to use at the expense of performance.

For example check out this comparision of Xubuntu and Debian Xfce to see how the two distributions' goals differ: http://distrowatch.com/weekly.php?issue=20090427#feature

The incredible success of Ubuntu indicates to me that many users will choose ease of use over maximum performance.

ps I appreciate the time you took to run the tests and share the results with others, thank you! :)

that did not make any bloody sense

[Legendary_Bibo]

that did not make any bloody sense

Haha exactly. UI changes does not mean more of your resources are used.

[cascade9]

EnGorDiaz and Legendary_Bibo-


Ubuntu has an extra "layer" of user-friendliness compared with its parent distro (Debian). This makes it easier to use at the expense of performance.

For example check out this comparision of Xubuntu and Debian Xfce to see how the two distributions' goals differ: http://distrowatch.com/weekly.php?issue=20090427#feature

The incredible success of Ubuntu indicates to me that many users will choose ease of use over maximum performance.

ps I appreciate the time you took to run the tests and share the results with others, thank you! :)

What on earth do you mean "that did not make sense"?

Its debateable how much difference it would make to 3d testing (I would liked to have seen that in the distowatch test, but lets face it, debian or ubuntu are not really seen as 'gaming OSes' so I understand why there isnt any 3D benchmarks in the test) but it sure does show major perfromance differences.

@beastrace91- I'll agree with snowpine (again!) and say " I appreciate the time you took to run the tests and share the results with others, thank you!". Just so you know I'm not kicking pointlessly. ;)

That said, you've done something that I find....welll......reprehensible. Your graphs lie. I'll direct you to this article-

http://www.dansdata.com/goop.htm

You dont need to read the whole thing, just "Lying with statistics".

Its something I really hate in graphs and benchmarking, in general.

BTW, I would have liked to have seen Debian unstable (and/or aptosid) in your test as well, and maybe Debian stable (though squeeze is now in a freeze, so maybe that not 'lenny'). It would have been interesting.

[beastrace91]

You dont need to read the whole thing, just "Lying with statistics".

Its something I really hate in graphs and benchmarking, in general.

/sigh. The graphs are default ones Open Office calc created for me from the data I gave it. I realized just glancing at the graphs may give a slightly misleading idea - but you know what? If someone is too lazy to actually look at the y-axis values (which are clearly labeled) and/or read the article (which clearly states Ubuntu only lags by 10ish% not 50%) Plus I gave the actual numerical values. So you can skip the graphs all together if you really do not like them.

~Jeff

[del_diablo]

Tsk, the graphs are still flawed.
Just for fun: Compare them with a graph where you start on zero insted, or give % difference from top to bottom somewhere.
But yes: This is quite interisting.

[snowpine]

That entirely did not make any sense.

In plain English, Ubuntu is not "tuned" for maximum performance. It is a comfortable, easy-driving Cadillac, not a Formula One racecar. :)

[kamaboko]

I'll just blink a few times to make up for the time difference.

[cj.surrusco]

The graphs weren't incorrect at all, they were just misleading. The increments on the side tell you exactly where each graph is, you just need to interpret it correctly. Many companies will use this technique in advertising to fool somebody that doesn't interpret it very well.

[forrestcupp]

What I want to know is why is Mint that much higher than Ubuntu? That's what doesn't make sense.

[cj.surrusco]

What I want to know is why is Mint that much higher than Ubuntu? That's what doesn't make sense.

I'll bet you it has to do with the desktop environment. Mint uses xfce by default, correct? Xfce is supposed to be a little bit lighter.

[NightwishFan]

To be honest, you have to look into the real issues such as driver, Xorg version, Mesa, etc.. That is far more important than any kind of desktop environment... :/

[dragos240]

Ubuntu, a dog? No wonder I'm using a different distro now. ;)

[Dustin2128]

I don't use it much, but I hear wonderful things about gentoo's performance; makes sense as everything down to the kernel is tweaked for your particular system. As soon as I get my new desktop (this weekend!) and get my wired network setup properly, I'm going to wipe hardy and do a gentoo install on my old one, dual booting with arch. It (my desktop) feels far too slow for a linux system with a 3GHz CPU. Then again, it was my first, and I've got a lot of stuff I don't need on it.

[NightwishFan]

I can guarantee that your system with Gentoo/Arch with the same software will perform.. the same. The Mythical 'high performance' configuration does not exist. To be honest you are better off with something more generic that will not be weak at a variety of tasks.

[Dustin2128]

I can guarantee that your system with Gentoo/Arch with the same software will perform.. the same. The Mythical 'high performance' configuration does not exist. To be honest you are better off with something more generic that will not be weak at a variety of tasks.
well I wanted to test them out anyway, and my opinions will be totally biased since I won't be installing useless software that just becomes bloat after a few months. I am not using genkernel this time though, sorry :)

[PC_load_letter]

I'll bet you it has to do with the desktop environment. Mint uses xfce by default, correct? Xfce is supposed to be a little bit lighter.

Nope, it uses Gnome as default DE, all the others are labeled Mint KDE, XFCE ...etc. and some of these extra editions are based on community efforts, and they are usually released a few weeks after the Gnome version.

It is very interesting if someone could explain the boosted performance of Mint (an Ubuntu based distro) here on these tests.

[cascade9]

/sigh. The graphs are default ones Open Office calc created for me from the data I gave it. I realized just glancing at the graphs may give a slightly misleading idea - but you know what? If someone is too lazy to actually look at the y-axis values (which are clearly labeled) and/or read the article (which clearly states Ubuntu only lags by 10ish% not 50%) Plus I gave the actual numerical values. So you can skip the graphs all together if you really do not like them.

~Jeff

Yes, well, not everybody actually READS these sorts of articles. You get lots of peopel who just skim, check a graph or two, and never look at the values.

I'm actually annoyed that OO would even create graphs like that without being told to lie, but it goes to show how widespread lying with staticics is..... :(


I can guarantee that your system with Gentoo/Arch with the same software will perform.. the same. The Mythical 'high performance' configuration does not exist. To be honest you are better off with something more generic that will not be weak at a variety of tasks.

Personally, I have never felt that gentoo was worth it. It is fast in my expereince, but not fast enough to make up for all the compling time, but gentoo wasnt even touched in this test so we can forget it, lets just look at Arch (well, chakra).

Got any data to back that up? I see people say this all the time, even when (like in this case) its flying in the face of the data provided. Its debable how much it is worth running one distro over another just for a few % points of speed, but to deny that some distros are faster, and some slower, is blind IMO.

According to these tests Chakra sure does show the highest perfromance. On some things, yeah, you would never notice a difference, but once you start looking at at 10% difference, that is major.

I'd expect that 3D would be one of the areas where preformance would be closest between distros (different versions of the nVidia drivers, the game version and to a lesser extent xorg, would make the biggest difference).


What I want to know is why is Mint that much higher than Ubuntu? That's what doesn't make sense.

Linux Mint Debian Edition. I'm not overly suprised that LMDE did better than ubuntu. Ubuntu has always felt slower, and benchmarked slower than debian in any test I've ever seen. Not that LMDE is debian persay, but its another push for me to try it.

[del_diablo]

What I want to know is why is Mint that much higher than Ubuntu? That's what doesn't make sense.

Its Mint running on the rolling distro Debian Testing.
IT DOES NOT USE UBUNTU AS A BASE!
How many people missed that?

[NightwishFan]

Not that a well configured system will not run well. However compilation optimizations, other than something like SSE for ffmpeg, will not make much difference. I believe most 64-bit distributions contain most processor features by default.

As for 3d performance, Ubuntu does indeed blacklist cards with issues, which is a good thing, obviously vesa is better for them than the native driver with whatever issue it has. 3d performance I assume depends on Driver and Xorg version. Ubuntu 10.04 (for example) was out performed by Fedora 13 which had a newer version of Xorg. Though in some benchmarks it itself was still comparable to Windows 7.

Obviously I do not put much stock in benchmarks. Unless someone is a computer scientist it is hard to do them right. Even though I will agree a trend is a trend across the board.

I will let it be known I use the server kernel on my laptop. I get no audio stutter when running stress with 8 cpu etc, so I assume the latency and scheduling are good enough. I like to take any opportunity to improve battery life and raw throughput. Though note for 'responsiveness', which most folks mistake for 'performance' you will probably want a more preemptive kernel.

[luceerose]

How did Debian score ?

[forrestcupp]

Linux Mint Debian Edition. I'm not overly suprised that LMDE did better than ubuntu.


Its Mint running on the rolling distro Debian Testing.
IT DOES NOT USE UBUNTU AS A BASE!
How many people missed that?

I missed it. Thanks guys. I just scrolled down to the charts and saw "Mint". Serves me right.

[cascade9]

Not that a well configured system will not run well. However compilation optimizations, other than something like SSE for ffmpeg, will not make much difference. I believe most 64-bit distributions contain most processor features by default.

As for 3d performance, Ubuntu does indeed blacklist cards with issues, which is a good thing, obviously vesa is better for them than the native driver with whatever issue it has. 3d performance I assume depends on Driver and Xorg version. Ubuntu 10.04 (for example) was out performed by Fedora 13 which had a newer version of Xorg. Though in some benchmarks it itself was still comparable to Windows 7.

Considering that you said "I can guarantee that your system with Gentoo/Arch with the same software will perform.. the same" when there was data, right in frount of you that went totally against what you said, frogive me for not believing you on "compilation optimizations *sinp* will not make much difference". I doubt it would make much different, but without any data at all, thats just a guess.

I'd forget all about blaming xorg versions for any difference. Madriva still ahs xorg 1.7.7, sabayon has 1.7.6, I *think* chakara has 1.8.x (hasnt checked that) and I have no idea what Mint Debian is using, but probably 1.7.7.

I should go through and check the nVidia vesions on those distros, but I have this feeling that they will all be smilar versions as well (probably 195.36.24).


Obviously I do not put much stock in benchmarks. Unless someone is a computer scientist it is hard to do them right. Even though I will agree a trend is a trend across the board.

What exactly do you mean by "computer scientist'?

Benchmarking isnt hard. Sure, I've seen people screw it up both by mistake and deliberately, but its not rocket science.

[beastrace91]

I should go through and check the nVidia vesions on those distros, but I have this feeling that they will all be smilar versions as well (probably 195.36.24).

I manually installed the same 256.53 driver on all of the distros to ensure the results where as accurate as possible.

~Jeff

[FuturePilot]

.

[Half-Left]

Always take benchmarks with a pinch of salt because you will get different results on different hardware configurations.

This benchmark proves nothing.

[NightwishFan]

Benchmarks are indeed hard and are very difficult to do in a default and neutral way. I do not mind if you do not believe me I am not here to convince you, just advise you.

[Simian Man]

I'd be willing to bet that the biggest reason is the amount of stuff running in the background by default. Ubuntu has the most cruft out of the box which means it is the slowest. If you did a minimal install, I'd be willing to bet that gap would narrow quite a bit.


Considering that you said "I can guarantee that your system with Gentoo/Arch with the same software will perform.. the same" when there was data, right in frount of you that went totally against what you said, frogive me for not believing you on "compilation optimizations *sinp* will not make much difference". I doubt it would make much different, but without any data at all, thats just a guess.
He is right, compiling for your hardware doesn't make much of a difference at all - especially if you're using 64-bit anyway. The reason Gentoo gives better performance is it doesn't come with the extra crap Ubuntu has out of the box.



What exactly do you mean by "computer scientist'?
LOL. One who practices computer science (http://en.wikipedia.org/wiki/Computer_Science).


Benchmarking isnt hard. Sure, I've seen people screw it up both by mistake and deliberately, but its not rocket science.
You could not be more wrong. Benchmarking is extremely hard. Any idiot (cough Phoronix cough) can run some programs and plot the results on their blog. It takes a lot of effort to ensure that you control the variables you aren't interested in, choose benchmarks that actually test what you are interested in and even harder to ensure that the conclusions you draw from the results are valid.

[forrestcupp]

Always take benchmarks with a pinch of salt because you will get different results on different hardware configurations.

This benchmark proves nothing.

Did you even look at it? He multibooted all of the distros on the same PC and used the same nvidia driver in each of them. So every distro had the exact same hardware configuration with the same video driver.

I agree that some benchmarks prove nothing, but I just don't get how you can use that argument to say that this benchmark proves nothing.

[Half-Left]

Did you even look at it? He multibooted all of the distros on the same PC and used the same nvidia driver in each of them. So every distro had the exact same hardware configuration with the same video driver.

I agree that some benchmarks prove nothing, but I just don't get how you can use that argument to say that this benchmark proves nothing.

Well it doesn't. it says that the hardware he benchmarked runs worse with Ubuntu. Most people will get mixed results and some might find that Ubuntu performs better or equal.

[NightwishFan]

Ubuntu may have had cron jobs running at the time, allowing disk IO. A download may have been left up mistakenly. Ubuntu 10.04 DOES have a known issue with performance, at least theoretically compared to Fedora. This is because Fedora 13 uses a newer Xorg as it was released later. Perhaps the drivers had Sync To Vblank Enabled or similar. Benchmarks are quite unreliable. Generally the are an unproven way to help confirm what is already expected.

I know you were trying to help, or confirm an issue. Ubuntu is not a dog at 3d performance. There is a REASON, even if the reason escapes us all. It is the devs job to find it. Did you report the issue and show them the data on Launchpad?

As far as 3d performance goes, I never do any benchmarks, however real life performance seems quite good with Nvidia on 10.04. I have a friends machine (used to be mine) with a Nvidia 6150SE which is a terrible card, however it manages to get a solid framerate on Urban Terror and Battlefield 2 in Wine, and is capable of playing Oblivion in Wine. (Its not the fastest but it runs ok). Oblivion did not run on Windows Vista on this same machine.

[realzippy]

All of the tests where run on clean, fully updated install of each distribution. They use the stock kernel each of the distributions provides

...doesn't that mean you ran the benchmarks with same nvidia driver but different kernel???

So the test says nothing about the distro but the kernel...

[beastrace91]

All of the tests where run on clean, fully updated install of each distribution. They use the stock kernel each of the distributions provides

...doesn't that mean you ran the benchmarks with same nvidia driver but different kernel???

So the test says nothing about the distro but the kernel...

Correct. But each distribution heavily customizes it's own kernel, so the kernel is very much part of the distribution itself.

~Jeff

[realzippy]

Btw,what exactly is the "stock kernel"?



Edit:
Btw2:
Don't get me wrong.I enjoy your blog and cherish the effort made for this,just asking
if you compared e.g.:
"Ubuntu"2.6.32-25-generic
to
"Chakra"2.6.32-25-generic
to
"Fedora"2.6.32-25-generic

aso ?

[Half-Left]

Well, if people don't configure the kernel it's pretty useless when you download it from kernel.org.

it could be some driver regression or bug and I can't say I've noticed any worse 3D performance when switching distros. I play EVE Online through WINE and it performs very well in Lucid.

[forrestcupp]

Well it doesn't. it says that the hardware he benchmarked runs worse with Ubuntu. Most people will get mixed results and some might find that Ubuntu performs better or equal.

I get what you're saying, now. But according to that argument, no benchmark can ever be meaningful. You'd have to do the exact same thing this guy did on 20 different computers.

[beastrace91]

Btw,what exactly is the "stock kernel"?



Edit:
Btw2:
Don't get me wrong.I enjoy your blog and cherish the effort made for this,just asking
if you compared e.g.:
"Ubuntu"2.6.32-25-generic
to
"Chakra"2.6.32-25-generic
to
"Fedora"2.6.32-25-generic

aso ?

By stock I mean default kernel the distro ships with. Many users do not have any reason to tinker with such things so I felt this was the best thing to run the benchmarks on.

~Jeff

[Dustin2128]

... But each distribution heavily customizes it's own kernel...
Nope. Slackware runs a vanilla linux kernel.

[baddog144]

The number of people saying "omg these graphs lie" astonishes me. They're not lying, you're just not bothering to look at them closely enough.

[CraigPaleo]

How did Debian score ?

Debian is represented by Linux Mint Rolling Release. I know the regular Mint is based on Ubuntu but the rollling release is Debian-based.

There are obviously people who haven't heard yet. It was announced here soon after its release but was thrown into "Recurring Discussions." Linux Mint Debian Edition (http://ubuntuforums.org/showthread.php?t=1569906)

[JDShu]

You could not be more wrong. Benchmarking is extremely hard. Any idiot (cough Phoronix cough) can run some programs and plot the results on their blog. It takes a lot of effort to ensure that you control the variables you aren't interested in, choose benchmarks that actually test what you are interested in and even harder to ensure that the conclusions you draw from the results are valid.

I lol'd at the Phoronix reference.


The number of people saying "omg these graphs lie" astonishes me. They're not lying, you're just not bothering to look at them closely enough.

That is called being misleading. Why would you post a graph if it doesn't make it easier to understand the data?

I agree that Ubuntu probably has more background processes than the rest of them.

[cascade9]

I manually installed the same 256.53 driver on all of the distros to ensure the results where as accurate as possible.

~Jeff

Which sort of rules out the nvidia driver version, but then you can satrt wondering if some distros 'like' manually installed drivers more than others LOL.


I'd be willing to bet that the biggest reason is the amount of stuff running in the background by default. Ubuntu has the most cruft out of the box which means it is the slowest. If you did a minimal install, I'd be willing to bet that gap would narrow quite a bit.

I think you are probably right on that.


He is right, compiling for your hardware doesn't make much of a difference at all - especially if you're using 64-bit anyway. The reason Gentoo gives better performance is it doesn't come with the extra crap Ubuntu has out of the box.

Yeah, point, and your probably right there as well. I was questioning NightwishFan because 'no data, dont trust the data gathering methods, but still Arch/gentoo would be the same' doesnt stack up....or fit with the benchmarks here, or that I've seen anywhere else.


ILOL. One who practices computer science (http://en.wikipedia.org/wiki/Computer_Science).

LOL, yeah, I know what its defined as. I was wondering what exactly would fit under 'computer scientist'? Do you need a degree/post grad/masters/doctorate/professors chair to do benchmarks? :lolflag:

BTW, if anyone thinks thats being silly, you should see what some of the people I've had to deal with have said should be the 'minimum' to discuss some subjects. I've had people in academia (and this is no lie) come out and state that ALL books on subject 'x' that were not written by 'qualified professionals' (by which they mean minimum of post-grad, preferable masters or higher) should be taken down and destroyed.


IYou could not be more wrong. Benchmarking is extremely hard. Any idiot (cough Phoronix cough) can run some programs and plot the results on their blog. It takes a lot of effort to ensure that you control the variables you aren't interested in, choose benchmarks that actually test what you are interested in and even harder to ensure that the conclusions you draw from the results are valid.

I'll disagree. Its not hard at all. Well, the actual benchmarking isnt hard, but there are a lot of possible pitfalls.

[Simian Man]

LOL, yeah, I know what its defined as. I was wondering what exactly would fit under 'computer scientist'? Do you need a degree/post grad/masters/doctorate/professors chair to do benchmarks? :lolfalg:

BTW, if anyone thinks thats being silly, you should see what some of the people I've had to deal with have said should be the 'minimum' to discuss some subjects. I've had people in academia (and this is no lie) come out and state that ALL books on subject 'x' that were not written by 'qualified professionals' (by which they mean minimum of post-grad, preferable masters or higher) should be taken down and destroyed.

No I didn't mean you need any certain level of education to be qualified to do valid benchmarks. Just that you need to understand the problems involved. I know people with nothing but a high school degree who I consider computer scientists :).