Firefox 3.6: And it gets FASTER!

[phrostbyte]

Tests were done on a 64-bit version of Ubuntu 9.10 with minimal non-interactive processes.

Firefox 3.5.6 Sunspider Results - 2030.4ms

Firefox 3.6 Beta 5 Sunspider Results - 891.4ms

Looks like Firefox 3.6 is ~2.25 times faster with JavaScript then Firefox 3.5!

Explanation: TraceMonkey got a 64-bit just-in-time compiler in Firefox 3.6!

Compared to Chromium (latest build):
chromium-browser 4.0.283.0~svn20091226r35283-0ubuntu1~ucd1~karmic Sunspider Results - 368.8ms

Chromium is still the king of JS performance on Ubuntu.

Another interesting comparison:

Firefox 3.5.6 (32-bit) running on Wine - 880.4ms
Firefox 3.6 Beta 5 (32-bit) running on Wine - 769.0ms

Looks like the Windows version (Firefox 3.5) beats the Linux version by a large amount! Firefox 3.6 is less of difference, but it still beats the Linux version. I couldn't test it with a 64-bit build, only 32-bit version is available. One explanation is the 32-bit version of Firefox is more optimised (ie, the JIT'er).

Here is a nice chart (X axis is "Execution time in milliseconds"):



Another benchmark: IE 8 on Windows 7 - 3923.0ms (Just to put things in perspective)

Now onto Peacekeeper. Peacekeeper is a benchmark that tests a number of different things, including JavaScript performance, DOM performance, and rendering.

Here is the Peacekeeper results (Ubuntu 64-bit) - Compiz is enabled:



Here is Peacekeeper result on Windows 7 (64-bit) - Aero is enabled:


Firefox 3.6 has also improved in this benchmark, but it's not as drastic (most of 3.6's improvements are to it's JS engine). This is a benchmark where Linux totally defeats Windows, especially in rendering. Seem's Xorg's hardware acceleration is destroying Windows's lack thereof.

[pwnst*r]

Chrome 4.0.249.43

406.2 ms with 10 tabs open (dunno if any of that makes a difference)

*edit* make that thirteen tabs. i have 3 tabs that are incognito so i can have my other gmail address open.

[gnuvistawouldbecool]

Well, since this will be rather dependant on hardware, this probably means little, not knowing what you were running, but using:
20091226 Minefield/3.7a1pre, on a AMD Athlon 64 3000, 2.0GHz, 512MB DDR computer,

1661.0ms

[pwnst*r]

Firefox 3.5.6

912.6 ms one tab open

[phrostbyte]

Well, since this will be rather dependant on hardware, this probably means little, not knowing what you were running, but using:
20091226 Minefield/3.7a1pre, on a AMD Athlon 64 3000, 2.0GHz, 512MB DDR computer,

1661.0ms

Run a test on same hardware with Firefox 3.5.6

[phrostbyte]

dbl post

[pwnst*r]

lol

[Pogeymanz]

Firefox 3.6Beta5:
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 1323.2ms +/- 2.9%
--------------------------------------------

3d: 192.6ms +/- 2.5%
cube: 62.6ms +/- 5.0%
morph: 39.4ms +/- 1.7%
raytrace: 90.6ms +/- 2.5%

access: 152.4ms +/- 3.6%
binary-trees: 46.2ms +/- 1.2%
fannkuch: 60.4ms +/- 3.1%
nbody: 28.0ms +/- 17.5%
nsieve: 17.8ms +/- 5.8%

bitops: 42.2ms +/- 2.5%
3bit-bits-in-byte: 1.8ms +/- 30.9%
bits-in-byte: 12.6ms +/- 5.4%
bitwise-and: 3.0ms +/- 0.0%
nsieve-bits: 24.8ms +/- 4.2%

controlflow: 40.0ms +/- 2.2%
recursive: 40.0ms +/- 2.2%

crypto: 67.2ms +/- 5.8%
aes: 39.0ms +/- 9.8%
md5: 18.4ms +/- 3.7%
sha1: 9.8ms +/- 13.9%

date: 202.4ms +/- 2.1%
format-tofte: 126.6ms +/- 2.7%
format-xparb: 75.8ms +/- 4.8%

math: 51.0ms +/- 19.3%
cordic: 16.8ms +/- 6.2%
partial-sums: 25.8ms +/- 40.9%
spectral-norm: 8.4ms +/- 8.1%

regexp: 104.0ms +/- 4.5%
dna: 104.0ms +/- 4.5%

string: 471.4ms +/- 5.4%
base64: 11.4ms +/- 6.0%
fasta: 79.0ms +/- 2.2%
tagcloud: 133.6ms +/- 7.3%
unpack-code: 207.2ms +/- 9.1%
validate-input: 40.2ms +/- 10.4%


Firefox3.7Alpha1:
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 1352.8ms +/- 14.6%
--------------------------------------------

3d: 198.4ms +/- 3.5%
cube: 64.4ms +/- 1.7%
morph: 42.2ms +/- 2.5%
raytrace: 91.8ms +/- 6.9%

access: 137.0ms +/- 2.1%
binary-trees: 28.8ms +/- 4.7%
fannkuch: 65.4ms +/- 1.0%
nbody: 26.6ms +/- 6.3%
nsieve: 16.2ms +/- 3.4%

bitops: 43.2ms +/- 2.4%
3bit-bits-in-byte: 1.6ms +/- 42.6%
bits-in-byte: 12.4ms +/- 5.5%
bitwise-and: 2.2ms +/- 25.3%
nsieve-bits: 27.0ms +/- 3.3%

controlflow: 122.2ms +/- 166.6%
recursive: 122.2ms +/- 166.6%

crypto: 68.4ms +/- 1.0%
aes: 40.0ms +/- 5.8%
md5: 18.4ms +/- 3.7%
sha1: 10.0ms +/- 21.5%

date: 182.4ms +/- 0.8%
format-tofte: 105.2ms +/- 1.3%
format-xparb: 77.2ms +/- 1.8%

math: 53.2ms +/- 2.0%
cordic: 22.6ms +/- 3.0%
partial-sums: 22.0ms +/- 4.0%
spectral-norm: 8.6ms +/- 7.9%

regexp: 104.6ms +/- 7.7%
dna: 104.6ms +/- 7.7%

string: 443.4ms +/- 4.2%
base64: 11.4ms +/- 6.0%
fasta: 79.4ms +/- 2.1%
tagcloud: 127.4ms +/- 2.5%
unpack-code: 185.0ms +/- 12.5%
validate-input: 40.2ms +/- 12.3%


So far the alpha is doing a little worse on my machine. Oh well. I don't have FF3.5 installed at all, so I can't compare that.

[phrostbyte]

Firefox 3.6Beta5:
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 1323.2ms +/- 2.9%
--------------------------------------------

3d: 192.6ms +/- 2.5%
cube: 62.6ms +/- 5.0%
morph: 39.4ms +/- 1.7%
raytrace: 90.6ms +/- 2.5%

access: 152.4ms +/- 3.6%
binary-trees: 46.2ms +/- 1.2%
fannkuch: 60.4ms +/- 3.1%
nbody: 28.0ms +/- 17.5%
nsieve: 17.8ms +/- 5.8%

bitops: 42.2ms +/- 2.5%
3bit-bits-in-byte: 1.8ms +/- 30.9%
bits-in-byte: 12.6ms +/- 5.4%
bitwise-and: 3.0ms +/- 0.0%
nsieve-bits: 24.8ms +/- 4.2%

controlflow: 40.0ms +/- 2.2%
recursive: 40.0ms +/- 2.2%

crypto: 67.2ms +/- 5.8%
aes: 39.0ms +/- 9.8%
md5: 18.4ms +/- 3.7%
sha1: 9.8ms +/- 13.9%

date: 202.4ms +/- 2.1%
format-tofte: 126.6ms +/- 2.7%
format-xparb: 75.8ms +/- 4.8%

math: 51.0ms +/- 19.3%
cordic: 16.8ms +/- 6.2%
partial-sums: 25.8ms +/- 40.9%
spectral-norm: 8.4ms +/- 8.1%

regexp: 104.0ms +/- 4.5%
dna: 104.0ms +/- 4.5%

string: 471.4ms +/- 5.4%
base64: 11.4ms +/- 6.0%
fasta: 79.0ms +/- 2.2%
tagcloud: 133.6ms +/- 7.3%
unpack-code: 207.2ms +/- 9.1%
validate-input: 40.2ms +/- 10.4%


Firefox3.7Alpha1:
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 1352.8ms +/- 14.6%
--------------------------------------------

3d: 198.4ms +/- 3.5%
cube: 64.4ms +/- 1.7%
morph: 42.2ms +/- 2.5%
raytrace: 91.8ms +/- 6.9%

access: 137.0ms +/- 2.1%
binary-trees: 28.8ms +/- 4.7%
fannkuch: 65.4ms +/- 1.0%
nbody: 26.6ms +/- 6.3%
nsieve: 16.2ms +/- 3.4%

bitops: 43.2ms +/- 2.4%
3bit-bits-in-byte: 1.6ms +/- 42.6%
bits-in-byte: 12.4ms +/- 5.5%
bitwise-and: 2.2ms +/- 25.3%
nsieve-bits: 27.0ms +/- 3.3%

controlflow: 122.2ms +/- 166.6%
recursive: 122.2ms +/- 166.6%

crypto: 68.4ms +/- 1.0%
aes: 40.0ms +/- 5.8%
md5: 18.4ms +/- 3.7%
sha1: 10.0ms +/- 21.5%

date: 182.4ms +/- 0.8%
format-tofte: 105.2ms +/- 1.3%
format-xparb: 77.2ms +/- 1.8%

math: 53.2ms +/- 2.0%
cordic: 22.6ms +/- 3.0%
partial-sums: 22.0ms +/- 4.0%
spectral-norm: 8.6ms +/- 7.9%

regexp: 104.6ms +/- 7.7%
dna: 104.6ms +/- 7.7%

string: 443.4ms +/- 4.2%
base64: 11.4ms +/- 6.0%
fasta: 79.4ms +/- 2.1%
tagcloud: 127.4ms +/- 2.5%
unpack-code: 185.0ms +/- 12.5%
validate-input: 40.2ms +/- 12.3%


So far the alpha is doing a little worse on my machine. Oh well. I don't have FF3.5 installed at all, so I can't compare that.

I'm surprised the alpha is doing so well, I would assume they built it with debug symbols.

[crimesaucer]

Yeah, I've been using the TraceMonkey for 64-bit patch for the last couple of months now, I think it gives me issues on certain flash sites, like pages like this: http://www.surfline.com/surf-news/gift-wrapped_39664/1/



Before the patch, my version of Firefox 3.5 had a score of about 4800 for the sunspider benchmarks, and the first versions of 3.6b1 and 3.6b2 (without the patch) were around 4400.



Then when the "TraceMonkey for 64-bit" patch came out it dropped my results down to 1660 - 1600 for versions 3.6b2, 3.63, and 3.6b4.



When building Firefox 3.7a1pre (with native TraceMonkey 64-bit included and no patch needed) my score dropped down to around 1400.



Sadly, I've tested every version number that I've built for Linux64, and I've compared them all to the windows versions on my Vista64 partition, and the windows versions are always faster. The 32-bit windows version (on my Vista64) of 3.7a1pre has a score near 1200.



Another nice thing about the newer versions of Firefox is that they have the PGO working better.