|
As XML and XSLT move from the cutting edge into the mainstream,
they are used in more real applications, like the rendering of web
sites. That transition now means that XSLT users need to pay
attention to the performance of their stylesheets, especially if the
XSLT engine is run for every web page served.
Resin's XSLT engine was about three times faster than
Xalan and about 80% faster than Saxon.
DataPower Technology has created a comprehensive XSLT benchmark,
called XSLTMark to
evaluate the performance of XSLT engines. Like all synthetic
benchmarks, it's not as good as testing your own application, but it
is comprehensive and gives a good indication of XSLT engine
performance.
Although Resin has included an XSLT engine since 1998, Resin's
reputation as a fast servlet and JSP engine has overshadowed its
XSLT implementation. This comparison reintroduces Resin's XSLT
engine and compares its performance with the other popular Java
XSLT engines.
For this benchmark, Resin's XSLT engine was about three times faster than
Xalan and about 80% faster than Saxon.
The following Java XSLT engines were tested:
The XSLT benchmark drivers was taken from the XSLTMark website and all
comparisons were done on the same RedHat 7.2 system (dual 800Mhz
Pentium III).
XSLTMark summarizes the results from the individual tests by
calculating a number of means. For benchmarks of this type, the
geometric mean is the most valid, since the geometric mean weighs all
test equally. In other words, even though queens
is slow and identity is fast for every engine, the geometric
mean treats them both identically.
Using the geometric mean, Resin is:
- 10% faster than XT
- 80% faster than Saxon
- 3 times faster than Xalan
| | Mean
|
|---|
| Engine | Arithmetic | Geometric | Harmonic
|
|---|
| Resin | 548.64 | 383.67 | 196.55
|
| XT | 465.58 | 323.85 | 162.83
|
| Saxon | 331.35 | 209.25 | 94.94
|
| Xalan | 204.16 | 121.12 | 53.87
|
The following graph shows the performance for each test. The
units are KB/s.
XSLTMark 2.0 (Feb 28 2001) running on Linux 2.4.14 x86
Java Runtime Environment is IBM Corporation 1.3.0
Current time is 15:22:11 PST, 11 Jan 2002
Results for driver ResinDriver
prep exec
Test Component iter ms ms KB In KB Out KB/s status
alphabetize 10 2021 1292 196 156 136.66 OK
attsets 100 1846 578 121 254 324.9 OK
avts 100 1897 1398 1966 1155 1116.34 OK
axis 100 1994 297 35 57 155.36 OK
backwards 100 1965 486 260 260 535.6 OK
bottles 100 1923 568 4 1130 999 OK
breadth 100 1823 558 415 199 551.11 OK
brutal 100 2504 1076 130 451 270.64 OK
chart 100 2168 748 121 467 393.76 OK
creation 50 1891 1191 983 1520 1050.81 OK
current 100 1904 165 18 27 141.45 OK
dbonerow 10 2096 10677 19621 9 919.3 OK
dbtail 100 1909 1109 1966 502 1112.83 OK
decoy 100 2133 3500 1966 1972 562.67 OK
depth 100 4 394 588 371 1219.46 OK
encrypt 10 1973 286 196 196 687.38 OK
functions 100 1957 4429 1966 616 291.5 OK
game 100 2087 420 260 53 373.54 OK
html 100 1913 149 29 53 275.6 CHK OUTPUT
identity 10 1741 1421 1962 1962 1381.06 OK
inventory 100 2129 378 184 200 509.6 OK
metric 100 1903 371 88 61 202.42 CHK OUTPUT
number 100 1956 1163 18 79 42.32 NO REF
oddtemplate 100 1855 279 16 13 54.95 OK
patterns 100 1920 2552 1966 1972 771.68 OK
prettyprint 10 1958 1342 196 1405 596.75 OK
priority 100 2174 328 91 55 223.6 OK
products 100 2013 442 196 43 271.54 OK
queens 10 2023 440 0 17 19.73 OK
reverser 100 1937 260 129 125 491.29 OK
stringsort 10 2001 1613 1962 1933 1207.59 OK
summarize 100 1936 417 712 27 887.57 OK
total 100 1915 126 121 36 625.47 OK
tower 20 2037 1000 1 1381 691.05 OK
trend 100 1867 3147 176 804 155.92 OK
union 100 1931 73 16 10 185.28 OK
xpath 100 1858 97 28 18 244.64 OK
xslbench1 100 2385 295 36 732 1303.13 CHK OUTPUT
xslbench2 20 2191 1487 2920 2109 1691.3 CHK OUTPUT
xslbench3 20 19 1101 2920 25 1337.72 CHK OUTPUT
Aggr. Results 2740 67346 44250 38601 19490 656.41
Arithmetic Mean 1924.17 548.64
Geometric Mean 383.67
Harmonic Mean 196.55
Conformance Score (OK+NOREF+SKIPPED divided by TOTAL) 35/40 = 87.5%
XSLTMark 2.0 (Feb 28 2001) running on Linux 2.4.14 x86
Java Runtime Environment is IBM Corporation 1.3.0
Current time is 15:24:46 PST, 11 Jan 2002
Results for driver XTDriver
prep exec
Test Component iter ms ms KB In KB Out KB/s status
alphabetize 10 247 992 196 156 178.07 OK
attsets 100 7 724 121 255 260.53 OK
avts 100 6 1788 1966 1156 873.31 OK
axis 100 10 314 35 60 153.02 OK
backwards 100 5 1301 260 261 200.72 CHK OUTPUT
bottles 100 9 1044 4 1130 543.52 OK
breadth 100 3 767 415 200 402.02 OK
brutal 100 8 678 130 394 386.88 OK
chart 100 8 596 121 359 403.32 OK
creation 50 5 1610 983 1520 777.6 OK
current 100 10 343 18 29 71.18 OK
dbonerow 10 7 14146 19621 8 693.85 OK
dbtail 100 3 1682 1966 503 734.22 OK
decoy 100 12 3959 1966 1974 497.68 OK
depth 100 2 850 588 373 566.23 OK
encrypt 10 3 326 196 196 603.3 OK
functions 100 7 7437 1966 618 173.76 OK
game 100 8 292 260 43 520.22 OK
html 100 3 174 29 40 199.52 CHK OUTPUT
identity 10 2 1735 1962 1962 1131.17 OK
inventory 100 3 425 184 202 455.19 OK
metric 0 0 0 0 0 ? ERROR
number 100 9 1060 18 79 46.43 NO REF
oddtemplate 100 3 147 16 15 109.95 OK
patterns 100 3 3172 1966 1974 621.16 OK
prettyprint 10 11 1681 196 1405 476.41 OK
priority 100 6 209 91 57 354.88 OK
products 100 3 951 196 40 124.41 OK
queens 10 13 669 0 17 13.1 OK
reverser 100 2 1151 129 129 112.63 OK
stringsort 10 101 2828 1962 1933 688.8 OK
summarize 100 7 605 712 29 613.14 OK
total 100 2 221 121 33 350.63 OK
tower 20 147 2168 1 1381 318.83 OK
trend 100 2 5103 176 806 96.34 OK
union 100 3 127 16 11 113.04 OK
xpath 100 2 125 28 20 196.48 OK
xslbench1 100 7 287 36 663 1220.7 OK
xslbench2 20 3 1838 2920 2129 1373.68 OK
xslbench3 20 2 1517 2920 25 970.86 OK
Aggr. Results 2780 686 63567 44218 21904 520.11
Arithmetic Mean 18.54 465.58
Geometric Mean 323.85
Harmonic Mean 162.83
Conformance Score (OK+NOREF+SKIPPED divided by TOTAL) 37/40 = 92.5%
XSLTMark 2.0 (Feb 28 2001) running on Linux 2.4.14 x86
Java Runtime Environment is IBM Corporation 1.3.0
Current time is 15:38:35 PST, 11 Jan 2002
Results for driver SaxonDriver
prep exec
Test Component iter ms ms KB In KB Out KB/s status
alphabetize 10 32 1472 196 156 120 OK
attsets 100 17 1244 121 255 151.59 OK
avts 100 12 2247 1966 1156 694.89 OK
axis 100 8 694 35 60 69.16 OK
backwards 100 5 1704 260 261 153.22 OK
bottles 100 22 1626 4 1130 348.97 OK
breadth 100 6 1574 415 200 195.87 OK
brutal 100 19 1323 130 623 285 OK
chart 100 8 1295 121 726 327.39 OK
creation 50 12 2134 983 1520 586.65 OK
current 100 5 786 18 29 31 OK
dbonerow 10 8 12633 19621 12 777.11 OK
dbtail 100 7 2049 1966 503 602.69 OK
decoy 100 10 4898 1966 1974 402.26 OK
depth 100 3 1266 588 373 380.13 OK
encrypt 10 170 599 196 196 328.33 OK
functions 100 9 5321 1966 618 242.85 OK
game 100 6 518 260 79 327.94 OK
html 100 10 421 29 77 126.88 OK
identity 10 4 2531 1962 1962 775.41 OK
inventory 100 5 797 184 201 242.67 OK
metric 100 5 735 88 65 104.83 OK
number 100 10 815 18 79 60.39 NO REF
oddtemplate 100 5 859 16 15 18.76 OK
patterns 100 5 3900 1966 1974 505.2 OK
prettyprint 10 12 3779 196 1405 211.91 OK
priority 100 7 422 91 57 175.64 OK
products 100 4 638 196 62 202.43 OK
queens 10 17 1064 0 17 8.23 OK
reverser 100 4 3613 129 129 35.87 OK
stringsort 10 12 2850 1962 1933 683.48 OK
summarize 100 4 800 712 29 463.62 OK
total 100 4 385 121 51 224.23 OK
tower 20 10 2367 1 1381 292.02 OK
trend 100 4 3913 176 806 125.62 OK
union 100 4 309 16 11 46.3 OK
xpath 100 4 330 28 20 74.28 OK
xslbench1 100 198 583 36 877 784.01 OK
xslbench2 20 6 2225 2920 2556 1230.76 OK
xslbench3 20 5 1765 2920 32 836.45 OK
Aggr. Results 3080 698 78484 44596 23631 434.66
Arithmetic Mean 17.45 331.35
Geometric Mean 209.25
Harmonic Mean 94.94
Conformance Score (OK+NOREF+SKIPPED divided by TOTAL) 40/40 = 100%
XSLTMark 2.0 (Feb 28 2001) running on Linux 2.4.14 x86
Java Runtime Environment is IBM Corporation 1.3.0
Current time is 15:09:09 PST, 11 Jan 2002
Results for driver Xalan2JDriver
prep exec
Test Component iter ms ms KB In KB Out KB/s status
alphabetize 10 64 3221 196 156 54.84 OK
attsets 100 50 1792 121 255 105.26 OK
avts 100 22 4993 1966 1156 312.73 OK
axis 100 29 1042 35 60 46.11 OK
backwards 100 11 1977 260 261 132.09 OK
bottles 100 31 2597 4 1130 218.5 OK
breadth 100 17 2341 415 200 131.72 OK
brutal 100 72 1768 130 394 148.36 OK
chart 100 43 2111 121 359 113.87 OK
creation 50 11 3684 983 1520 339.83 OK
current 100 15 909 18 29 26.86 OK
dbonerow 10 17 29714 19621 8 330.32 OK
dbtail 100 11 2920 1966 503 422.93 OK
decoy 100 227 11657 1966 1974 169.02 OK
depth 100 12 2036 588 373 236.39 OK
encrypt 10 11 845 196 196 232.75 OK
functions 100 25 22354 1966 618 57.81 OK
game 100 14 911 260 43 166.74 OK
html 100 14 828 29 47 45.94 CHK OUTPUT
identity 10 17 5043 1962 1962 389.17 OK
inventory 100 11 1305 184 202 148.24 OK
metric 100 14 1523 88 65 50.66 OK
number 100 16 1872 18 79 26.29 NO REF
oddtemplate 100 15 1268 16 15 12.75 OK
patterns 100 14 8132 1966 1974 242.29 OK
prettyprint 10 39 2134 196 1405 375.28 OK
priority 100 19 881 91 57 84.19 OK
products 100 16 2532 196 40 46.73 OK
queens 10 74 1799 0 17 4.87 OK
reverser 100 23 970 129 129 133.65 OK
stringsort 10 16 5591 1962 1933 348.4 OK
summarize 100 12 1339 712 29 277.03 OK
total 100 10 639 121 33 121.27 OK
tower 20 27 2796 1 1381 247.22 OK
trend 100 9 32396 176 806 15.17 OK
union 100 13 574 16 11 25.01 OK
xpath 100 112 611 28 20 40.2 OK
xslbench1 100 95 870 36 665 403.65 OK
xslbench2 20 75 3735 2920 2139 677.33 OK
xslbench3 20 16 1407 2920 25 1046.76 OK
Aggr. Results 2980 1325 174289 44567 22244 191.67
Arithmetic Mean 33.97 204.16
Geometric Mean 121.12
Harmonic Mean 53.87
Conformance Score (OK+NOREF+SKIPPED divided by TOTAL) 39/40 = 97.5%
Resin's three times performance gain over Xalan and 80% gain
over Saxon is a significant advantage for web sites using Resin to
render their pages from XML. Besides the obvious improved response
times, the better performance means that more web sites can take
advantage of XSLT instead of avoiding it because of performance
worries.
As XML becomes more important for web services, the core engine
performance will also become more important. Resin's fast XML and
XSLT support strengthens Resin's position as a leading Java XML
application server.
Copyright (c) 1998-2009 Caucho Technology, Inc. All rights reserved.
caucho® ,
resin® and
quercus®
are registered trademarks of Caucho Technology, Inc.
| 
