1 2011-07-02 Maciej Stachowiak <mjs@apple.com>
3 Update SunSpider versioning to 1.0
4 https://bugs.webkit.org/show_bug.cgi?id=63870
6 Reviewed by Dan Bernstein.
8 * hosted/sunspider.html:
9 * hosted/versions.html:
10 * resources/driver-TEMPLATE.html:
11 * resources/results-TEMPLATE.html:
14 2011-07-02 Maciej Stachowiak <mjs@apple.com>
16 Sunspider 0.9.1 harness doesn't actually close() its test documents
17 https://bugs.webkit.org/show_bug.cgi?id=47045
19 Reviewed by Daniel Bates.
21 * hosted/versions.html:
22 * resources/driver-TEMPLATE.html:
24 2011-07-02 Maciej Stachowiak <mjs@apple.com>
28 Fix obviou typo in previous commit.
30 * tests/sunspider-1.0/bitops-bitwise-and.js:
32 2011-07-02 Maciej Stachowiak <mjs@apple.com>
34 Reviewed by Anders Carlsson.
36 SunSpider: all four bitops benchmarks can be replaced with NOP
37 https://bugs.webkit.org/show_bug.cgi?id=38446
39 * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
40 (TimeFunc): Save the result.
41 * tests/sunspider-1.0/bitops-bits-in-byte.js:
42 (TimeFunc): Save the result.
43 * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
45 * tests/sunspider-1.0/bitops-nsieve-bits.js:
46 (sieve): Save the result.
48 2011-07-02 Maciej Stachowiak <mjs@apple.com>
50 SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
51 https://bugs.webkit.org/show_bug.cgi?id=60937
53 Reviewed by Dan Bernstein.
55 * tests/sunspider-1.0/string-validate-input.js:
56 (doTest): Rename the variable to username
58 2011-07-02 Maciej Stachowiak <mjs@apple.com>
60 Reviewed by Darin Adler.
62 SunSpider: The main function of math-cordic is dead code
63 https://bugs.webkit.org/show_bug.cgi?id=63863
65 Try to prevent math-cordic from being eliminated as dead code or
66 compiled down to a constant.
68 * tests/sunspider-1.0/math-cordic.js:
72 2011-06-30 Maciej Stachowiak <mjs@apple.com>
74 Reviewed by Adam Barth.
76 Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
77 https://bugs.webkit.org/show_bug.cgi?id=63782
79 * make-hosted: Learn about the new directory.
80 * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
82 2010-11-29 Geoffrey Garen <ggaren@apple.com>
84 Reviewed by Gavin Barraclough.
86 Improved accuracy of command-line SunSpider.
88 * resources/sunspider-standalone-driver.js: Use the returned number of
89 elapsed milliseconds from "run" and "checkSyntax" instead of measuring
90 ourselves, for slightly more accurate numbers.
92 2010-09-16 Gavin Barraclough <barraclough@apple.com>
94 Reviewed by Mark Rowe.
95 https://bugs.webkit.org/show_bug.cgi?id=45924
97 Modify the SunSpider harness to allow a path to be passed to the --suite
98 argument, supress the normal prepend-"tests/" behaviour for suite names
101 Also wrap the code in an anonymous function to move variables out of global
102 scope (currently the harness cannot handle running tests the overwrite the
105 * resources/sunspider-standalone-driver.js:
108 2010-08-27 Michael Saboff <msaboff@apple.com>
110 Reviewed by Stephanie Lewis.
112 Added version 5 and version 6 of V8 tests with modifications similar
113 to the version 4 tests so they'll work with the SunSpider test harness.
114 Note that these files are external source and therefore don't conform
115 to WebKit coding standards.
116 https://bugs.webkit.org/show_bug.cgi?id=44638
118 * tests/v8-v5: Added.
119 * tests/v8-v5/LIST: Added.
120 * tests/v8-v5/v8-crypto.js: Added.
121 * tests/v8-v5/v8-deltablue.js: Added.
122 * tests/v8-v5/v8-earley-boyer.js: Added.
123 * tests/v8-v5/v8-raytrace.js: Added.
124 * tests/v8-v5/v8-regexp.js: Added.
125 * tests/v8-v5/v8-richards.js: Added.
126 * tests/v8-v5/v8-splay.js: Added.
127 * tests/v8-v6: Added.
128 * tests/v8-v6/LIST: Added.
129 * tests/v8-v6/v8-crypto.js: Added.
130 * tests/v8-v6/v8-deltablue.js: Added.
131 * tests/v8-v6/v8-earley-boyer.js: Added.
132 * tests/v8-v6/v8-raytrace.js: Added.
133 * tests/v8-v6/v8-regexp.js: Added.
134 * tests/v8-v6/v8-richards.js: Added.
135 * tests/v8-v6/v8-splay.js: Added.
137 2010-07-15 Stephanie Lewis <slewis@apple.com>
139 Reviewed by Geoff Garen.
141 http://bugs.webkit.org/show_bug.cgi?id=42406
142 Output location of results file.
146 2010-05-17 Maciej Stachowiak <mjs@apple.com>
148 Reviewed by Mark Rowe.
150 Browser-hosted version of SunSpider runs tests twice as many times as advertised
151 https://bugs.webkit.org/show_bug.cgi?id=39201
153 * make-hosted: Make sure to reset test list after every test suite.
155 2010-05-07 Adam Roben <aroben@apple.com>
157 Fix typo in sunspider-compare-results.js that caused incorrect
158 two-sample t statistics to be computed
160 Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
161 two-sample t statistics
163 Reviewed by Darin Adler.
165 * resources/sunspider-compare-results.js:
166 (sunspiderCompareResults.resultLine): Fixed the calculation of the t
167 variable to match the definition of a two-sample t statistic. Looks
168 like this was just a simple typo.
170 2010-04-05 Darin Adler <darin@apple.com>
172 Updated the ignore property for changes to the contents here.
174 * .: Modified property svn:ignore.
176 2010-02-20 Maciej Stachowiak <mjs@apple.com>
178 Reviewed by Adam Barth.
180 Add sunspider-0.9.1 to the site in preparation for announcing it.
181 https://bugs.webkit.org/show_bug.cgi?id=35206
183 * hosted/sunspider.css: Remove tabs, which I noticed where present while
184 copying to the WebKitSite directory.
186 2010-01-11 Oliver Hunt <oliver@apple.com>
188 Reviewed by Darin Adler.
190 Sunspider parse-only tests don't run
191 https://bugs.webkit.org/show_bug.cgi?id=33489
193 Fix parse-only test list
195 * tests/parse-only/LIST:
197 2009-12-15 Maciej Stachowiak <mjs@apple.com>
199 Reviewed by Adele Peterson.
201 Make SunSpider version more prominent in the title
202 https://bugs.webkit.org/show_bug.cgi?id=32574
204 * hosted/sunspider.html:
205 * hosted/versions.html:
206 * resources/driver-TEMPLATE.html:
207 * resources/results-TEMPLATE.html:
209 2009-12-15 Maciej Stachowiak <mjs@apple.com>
211 Reviewed by Darin Adler.
213 Make SunSpider refuse to compare results across content versions
214 https://bugs.webkit.org/show_bug.cgi?id=32573
216 The results URL now includes a version indicator, and compare mode
217 will refuse to compare between versions.
219 * resources/driver-TEMPLATE.html:
220 * resources/results-TEMPLATE.html:
222 2009-12-14 Maciej Stachowiak <mjs@apple.com>
224 Reviewed by Darin Adler.
226 Some Browser-hosted SunSpider files are not valid HTML5
227 https://bugs.webkit.org/show_bug.cgi?id=32536
229 Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
230 close tags and adding a meta charset declaration).
232 * hosted/sunspider.html:
233 * hosted/versions.html:
234 * resources/TEMPLATE.html:
235 * resources/driver-TEMPLATE.html:
236 * resources/results-TEMPLATE.html:
238 2009-12-14 Maciej Stachowiak <mjs@apple.com>
240 Reviewed by Eric Seidel.
242 Make sunspider-0.9.1 the default content set (both command-line and hosted)
243 https://bugs.webkit.org/show_bug.cgi?id=32537
245 * hosted/sunspider.html: Change default suite link.
246 * sunspider: Change default suite.
247 * sunspider-compare-results: Likewise.
249 2009-12-14 Maciej Stachowiak <mjs@apple.com>
251 Reviewed by Sam Weinig.
253 in-browser SunSpider suffers excessive penalty under power management
254 https://bugs.webkit.org/show_bug.cgi?id=32505
256 I have made a few changes to address this:
258 1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
259 document.write to reduce triggering of progress UI.
260 2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
261 stepped all the way down the whole time.
262 3) Run the test cycle an extra time for warmup before the runs that count.
264 * hosted/sunspider-record-result.js: Removed.
266 * resources/TEMPLATE.html:
267 * resources/driver-TEMPLATE.html:
269 2009-12-13 Maciej Stachowiak <mjs@apple.com>
271 Reviewed by Gavin Barraclough.
273 SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
274 https://bugs.webkit.org/show_bug.cgi?id=16806
276 Based on a patch by Eric Seidel.
278 Fix the base64 computation to actually compute correct results. The impact on runtime of
279 the test is pretty small, but noticeable for some browsers. But at least it's not
280 doing a wrong and meaningless computation any more.
282 * tests/sunspider-0.9.1/string-base64.js:
286 2009-12-13 Maciej Stachowiak <mjs@apple.com>
288 Fixing commit error...
290 I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
292 * resources/driver-TEMPLATE.html:
294 2009-12-13 Maciej Stachowiak <mjs@apple.com>
296 Reviewed by Sam Weinig.
298 Use JSON.parse instead of eval for Web-hosted SunSpider results processing
299 https://bugs.webkit.org/show_bug.cgi?id=32490
301 * hosted/json2.js: Added. Incorporated from json.org
302 * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
303 * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
306 2009-12-12 Maciej Stachowiak <mjs@apple.com>
308 Reviewed by Oliver Hunt.
310 Enable Web-hosted version of SunSpider to handle multiple versions
311 https://bugs.webkit.org/show_bug.cgi?id=32478
313 * make-hosted: Now generate the hosted version to be able to run both the
314 0.9 and the 0.9.1 test suites.
315 * hosted: Modified property svn:ignore.
316 * hosted/sunspider-driver.html: Removed.
317 * hosted/sunspider-results.html: Removed.
318 * hosted/sunspider.html:
319 * hosted/versions.html: Added.
320 * resources/TEMPLATE.html:
321 * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
322 * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
324 2009-12-12 Maciej Stachowiak <mjs@apple.com>
326 Reviewed by Oliver Hunt.
328 Give command-line sunspider the ability to handle multiple suites and versions
329 https://bugs.webkit.org/show_bug.cgi?id=32477
331 Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
332 enumerating the tests. Current suites are:
334 sunspider-0.9 - riginal SunSpider content
335 sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
336 ubench - "ubench" microbenchmark suite
337 v8-v4 - version 4 of the v8 benchmark
338 parse-only - some parser tests
340 Sunspider can now be run with the --suite= parameter to select a
341 suite. The default is --suite=sunspider-0.9.
343 * .: Modified property svn:ignore to ignore new results directories.
344 * make-hosted: Generate based on sunspider-0.9 suite.
345 * resources/TEMPLATE.html: Fix an HTML compliance problem.
346 * resources/sunspider-standalone-driver.js: Become aware of suite names.
347 * sunspider: Modified to handle multiple suites as described above.
348 * sunspider-compare-results: ditto
349 * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
350 * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
351 * tests/LIST-UBENCH: Removed.
352 * tests/LIST-V8: Removed.
353 * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
354 * tests/3d-morph.js: ditto
355 * tests/3d-raytrace.js: ditto
356 * tests/access-binary-trees.js: ditto
357 * tests/access-fannkuch.js: ditto
358 * tests/access-nbody.js: ditto
359 * tests/access-nsieve.js: ditto
360 * tests/bitops-3bit-bits-in-byte.js: ditto
361 * tests/bitops-bits-in-byte.js: ditto
362 * tests/bitops-bitwise-and.js: ditto
363 * tests/bitops-nsieve-bits.js: ditto
364 * tests/controlflow-recursive.js: ditto
365 * tests/crypto-aes.js: ditto
366 * tests/crypto-md5.js: ditto
367 * tests/crypto-sha1.js: ditto
368 * tests/date-format-tofte.js: ditto
369 * tests/date-format-xparb.js: ditto
370 * tests/math-cordic.js: ditto
371 * tests/math-partial-sums.js: ditto
372 * tests/math-spectral-norm.js: ditto
373 * tests/regexp-dna.js: ditto
374 * tests/string-base64.js: ditto
375 * tests/string-fasta.js: ditto
376 * tests/string-tagcloud.js: ditto
377 * tests/string-unpack-code.js: ditto
378 * tests/string-validate-input.js: ditto
379 * tests/sunspider-0.9/: Added
380 * tests/sunspider-0.9/*.js: Moved from one level up
381 * tests/sunspider-0.9.1: Added.
382 * tests/sunspider-0.9.1/*.js: Moved from one level up.
383 * tests/v8-crypto.js: Moved to v8-v4/ directory.
384 * tests/v8-deltablue.js: ditto
385 * tests/v8-earley-boyer.js: ditto
386 * tests/v8-raytrace.js: ditto
387 * tests/v8-regexp.js: ditto
388 * tests/v8-richards.js: ditto
389 * tests/v8-splay.js: ditto
390 * tests/v8-v4: Added.
392 2009-09-30 Csaba Osztrogonac <oszi@inf.u-szeged.hu>
394 Reviewed by Darin Adler.
396 Platform specific null device replaced with a platform independent.
397 https://bugs.webkit.org/show_bug.cgi?id=29544
399 * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
401 2009-09-21 Csaba Osztrogonac <oszi@inf.u-szeged.hu>
403 Reviewed by Maciej Stachowiak.
405 [Qt] Make sunspider script work on Windows platform.
406 https://bugs.webkit.org/show_bug.cgi?id=29544
409 Using platform specific null device instead of /dev/null.
411 2009-06-19 Adam Treat <adam.treat@torchmobile.com>
413 Reviewed by Oliver Hunt.
415 https://bugs.webkit.org/show_bug.cgi?id=26540
416 Currently the SunSpider test driver lacks an option to run a test suite that
417 will test JavaScriptCore parsing performance only. This patch adds just such
418 a test suite and option to SunSpider as well as the jsc test shell. I've included
419 three large javascript source files found in the wild: jquery, mootools and prototype.
420 Combined with the concatenation of all three, these form a new testsuite to measure
421 and test pure JavaScriptCore parsing performance.
423 * resources/sunspider-standalone-driver.js:
424 Load the js with parse-only flag if it is part of the parse-only suite.
426 Add the parse-only suite to the list of options.
427 * sunspider-compare-results:
428 Add the parse-only suite to the list of options.
429 * tests/LIST-PARSE-ONLY: Added.
430 * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
431 A very large concatenation of the scripts found below. This js takes ~30ms to parse on my machine.
432 * tests/parse-only/jquery-1.3.2.js: Added.
433 Latest version from jquery.com.
434 * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
435 Latest version from mootools.net.
436 * tests/parse-only/prototype-1.6.0.3.js: Added.
437 Latest version from prototypejs.org.
439 2009-06-13 Steve Falkenburg <sfalken@apple.com>
441 Update V8 benchmark to version 4.
443 Reviewed by Cameron Zwarich.
446 * tests/v8-crypto.js:
447 * tests/v8-deltablue.js:
448 * tests/v8-earley-boyer.js:
449 * tests/v8-raytrace.js:
450 * tests/v8-regexp.js: Added.
451 * tests/v8-richards.js:
452 * tests/v8-splay.js: Added.
454 2009-05-26 Olivier DOLE <odole@pleyo.com>
458 Store sunspider test suite results in a specific directory for each
459 different type of benchmark (sunspider, V8 or ubench).
462 * sunspider-compare-results:
464 2009-05-12 Steve Falkenburg <sfalken@apple.com>
470 2009-05-11 Steve Falkenburg <sfalken@apple.com>
472 Include SunSpider tests in production builds.
474 Reviewed by Adam Roben.
476 * SunSpider.make: Added.
477 * xcopy.excludes: Added.
479 2009-03-04 Oliver Hunt <oliver@apple.com>
481 Added a few more tests to ubench
485 Adding a couple of test to cover too few and too many arguments. This
486 is needed to prevent regressions in all call paths from my current refactoring.
489 * tests/ubench/function-correct-args.js: Added.
491 * tests/ubench/function-excess-args.js: Added.
494 2008-10-30 Zoltan Horvath <Horvath.Zoltan.6@stud.u-szeged.hu>
496 Fix a bug where when V8's or UBENCH's test suite list file is missing
497 the error message is suggesting that SunSpider's list file is missing.
499 Reviewed by Sam Weinig.
502 (loadTestsList): Fix the error message to use the right list name.
504 2008-10-07 David Hyatt <hyatt@apple.com>
506 https://bugs.webkit.org/show_bug.cgi?id=18989
508 Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
509 we want all browsers to be doing the same thing on the test.
513 * tests/regexp-dna.js:
515 2008-09-09 Cameron Zwarich <cwzwarich@uwaterloo.ca>
517 Rubber-stamped by Maciej Stachowiak.
519 The bleeding edge branch of V8 now supports passing fileanmes after
520 '-f', so the SunSpider script should be changed to do this again.
521 This allows all 3 major JavaScript engines to be used with the
522 unmodified SunSpider script.
526 2008-09-04 Cameron Zwarich <cwzwarich@uwaterloo.ca>
528 Reviewed by Maciej Stachowiak.
530 Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
531 and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
532 correct the punctuation in its documentation.
536 2008-09-04 Cameron Zwarich <cwzwarich@uwaterloo.ca>
538 Reviewed by Oliver Hunt.
540 Bug 20616: Incorporate V8 benchmarks in testing
541 <https://bugs.webkit.org/show_bug.cgi?id=20616>
543 This is a first attempt at running the V8 benchmarks within SunSpider,
544 with the --v8 option. The number of iterations is fixed for each
545 benchmark, and was chosen to make each of the benchmarks run between
546 500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
549 * tests/LIST-V8: Added.
550 * tests/v8-crypto.js: Added.
551 * tests/v8-deltablue.js: Added.
552 * tests/v8-earley-boyer.js: Added.
553 * tests/v8-raytrace.js: Added.
554 * tests/v8-richards.js: Added.
556 2008-09-02 Cameron Zwarich <cwzwarich@uwaterloo.ca>
560 Remove a local change that was accidentally committed in r36034.
562 * resources/sunspider-standalone-driver.js:
564 2008-09-02 Cameron Zwarich <cwzwarich@uwaterloo.ca>
566 Rubber-stamped by Oliver Hunt.
568 Make SunSpider work with JavaScript shells that don't take filenames
569 after a '-f' argument.
573 2008-06-07 Cameron Zwarich <cwzwarich@uwaterloo.ca>
577 Remove the --squirrelfish option from SunSpider, as it is no longer needed.
580 * tests/LIST-SQUIRRELFISH: Removed.
582 2008-06-03 Alexey Proskuryakov <ap@webkit.org>
584 Rubber-stamped by Mark Rowe.
586 Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
590 2008-04-15 Maciej Stachowiak <mjs@apple.com>
592 Rubber stamped by Oliver.
594 - add newly working tests to squirrelfish list
596 * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
598 2008-04-14 Maciej Stachowiak <mjs@apple.com>
600 Rubber stamped by Oliver.
602 - move ubench tests properly.
604 * tests/ubench: Added.
605 * tests/ubench/function-closure.js: Added.
606 * tests/ubench/function-empty.js: Added.
607 * tests/ubench/function-missing-args.js: Added.
608 * tests/ubench/function-sum.js: Added.
609 * tests/ubench/loop-empty-resolve.js: Added.
610 * tests/ubench/loop-empty.js: Added.
611 * tests/ubench/loop-sum.js: Added.
613 2008-04-14 Maciej Stachowiak <mjs@apple.com>
615 Rubber stamped by Oliver.
617 * sunspider: Don't force --runs=1 in squirrelfish mode any more.
619 2008-04-14 Maciej Stachowiak <mjs@apple.com>
623 - make --squirrelfish mode use the real harness, but a reduced set of tests
625 Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
628 * tests/LIST-SQUIRRELFISH:
629 * tests/LIST-UBENCH: Added.
630 * tests/squirrelfish: Moved to tests/ubench
632 2008-04-14 Geoffrey Garen <ggaren@apple.com>
634 Reviewed by Maciej Stachowiak.
636 Removed an unused test.
638 * tests/squirrelfish/loop-resolve.js: Removed.
640 2008-04-10 Maciej Stachowiak <mjs@apple.com>
644 Add newly runnable tests to --squirrelfish mode.
646 * tests/LIST-SQUIRRELFISH:
648 2008-04-06 Geoffrey Garen <ggaren@apple.com>
650 Reviewed by Maciej Stachowiak.
652 A little more love for --squirrelfish mode:
654 Fixed a misplaced sort. Results properly sort now.
656 Added a test list just for squirrelfish, and updated it to include
657 all currently passing SunSpider tests.
659 Fixed the pruning regexp to match 3d-morph.js.
663 2008-03-26 Geoffrey Garen <ggaren@apple.com>
665 Reviewed by Oliver Hunt.
667 --squirrelfish mode: pared down tests for squirrelfish to chew on.
671 * tests/squirrelfish: Added.
672 * tests/squirrelfish/function-closure.js: Added.
673 * tests/squirrelfish/function-empty.js: Added.
674 * tests/squirrelfish/function-missing-args.js: Added.
675 * tests/squirrelfish/function-sum.js: Added.
676 * tests/squirrelfish/loop-empty-resolve.js: Added.
677 * tests/squirrelfish/loop-empty.js: Added.
678 * tests/squirrelfish/loop-resolve.js: Added.
679 * tests/squirrelfish/loop-sum.js: Added.
681 2008-03-21 Alexey Proskuryakov <ap@webkit.org>
685 * sunspider: Pause update daemon on Darwin to make results reliable
686 again (workaround for <rdar://problem/5811127>).
688 2008-02-07 Eric Seidel <eric@webkit.org>
692 * sunspider: add an error message when ./tests/LIST is not found
694 2007-12-29 Alp Toker <alp@atoker.com>
698 * hosted/sunspider.html:
700 2007-12-29 Alp Toker <alp@atoker.com>
704 Provide the current URL in the SunSpider results page. Useful for
705 copying benchmark results out of browsers without a location bar or
706 where the location bar can't handle long URLs.
708 * hosted/sunspider-results.html:
710 2007-12-20 Eric Seidel <eric@webkit.org>
714 * sunspider: increase default number of runs for more accurate numbers
716 2007-12-18 Mark Rowe <mrowe@apple.com>
720 * hosted/sunspider.html:
722 2007-12-18 Maciej Stachowiak <mjs@apple.com>
726 - give an extra digit of precision for small differences.
728 * resources/sunspider-compare-results.js:
730 2007-12-17 Maciej Stachowiak <mjs@apple.com>
734 - work around IE layout bugs
736 * hosted/sunspider.css:
738 2007-12-17 Maciej Stachowiak <mjs@apple.com>
742 - fix compare feature
744 * hosted/sunspider-results.html:
746 2007-12-17 Maciej Stachowiak <mjs@apple.com>
750 - Change the format of the output URL to be shorter, to meet IE constraints
752 * hosted/sunspider-driver.html:
753 * hosted/sunspider-results.html:
755 2007-12-13 Maciej Stachowiak <mjs@apple.com>
759 - Add some explanatory test and a dumb logo to SunSpider web hosted version
761 * hosted/sunspider-driver.html:
762 * hosted/sunspider-results.html:
763 * hosted/sunspider.css:
764 * hosted/sunspider.html:
766 2007-12-13 Maciej Stachowiak <mjs@apple.com>
770 - Make the web-hosted version of SunSpider look a little bit less drab.
772 * hosted/sunspider-driver.html:
773 * hosted/sunspider-results.html:
774 * hosted/sunspider.css: Added.
775 * hosted/sunspider.html:
776 * resources/TEMPLATE.html:
778 2007-12-12 Maciej Stachowiak <mjs@apple.com>
782 - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
784 A sample of what new-style results would look like:
786 TEST COMPARISON FROM TO DETAILS
788 ** TOTAL **: 1.41x as fast 6613.6ms +/- 0.1% 4703.0ms +/- 0.1% significant
790 3bit-bits-in-byte: *1.36x as slow* 88.8ms +/- 0.6% 121.1ms +/- 0.6% significant
792 * resources/sunspider-compare-results.js:
794 2007-11-14 Eric Seidel <eric@webkit.org>
798 * sunspider: add --shark-cache for L2 Cache Miss profiling
800 2007-11-09 Alexey Proskuryakov <ap@webkit.org>
802 Reviewed by Darin Adler.
804 http://bugs.webkit.org/show_bug.cgi?id=15925
805 SunSpider should check for Shark being installed
807 * sunspider: Explicitly test that shark command line tool is installed.
809 2007-11-04 Darin Adler <darin@apple.com>
811 * hosted/sunspider.html: Tweaked formatting a bit.
813 2007-11-03 Maciej Stachowiak <mjs@apple.com>
815 Reviewed by Darin Adler.
817 - Add compare mode and a description of the benchmark to the hosted version.
819 * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
820 * hosted/sunspider.html: Added intro text.
821 * make-hosted: Copy new file.
822 * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
824 * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
826 * sunspider-compare-results: Use sunspider-standalone-compare.js.
828 2007-11-02 Darin Adler <darin@apple.com>
830 * sunspider: Changed "--base" to "--set-baseline".
832 2007-11-02 Darin Adler <darin@apple.com>
836 * sunspider: Added a "--base" option that records the current test as a baseline.
837 Moved the code to find the newest .mshark file into a function (since I needed to use
838 it in the other script and I prefer to copy/paste a function).
840 * sunspider-compare-results: Added logic to compare the baseline with the last
841 run when you don't pass any parameters.
843 2007-10-29 Maciej Stachowiak <mjs@apple.com>
847 - Added list of ECMAScript3 features that SunSpider does not cover currently.
851 2007-10-29 Maciej Stachowiak <mjs@apple.com>
855 - Give better error messages when sunspider-compare-results fails
857 * sunspider-compare-results:
859 2007-10-29 Maciej Stachowiak <mjs@apple.com>
861 Reviewed by Darin Adler.
863 - fix Date tests to work in command-line SpiderMonkey
865 * tests/date-format-tofte.js:
866 * tests/date-format-xparb.js:
868 2007-10-29 Darin Adler <darin@apple.com>
872 * sunspider: Added --shark20 option, to run Shark at its highest sample
873 resolution instead of its default.
875 2007-10-28 Adam Roben <aroben@apple.com>
877 Fix SunSpider on Windows
881 * sunspider: Don't use colons in filenames.
883 2007-10-28 Darin Adler <darin@apple.com>
887 * resources/sunspider-analyze-results.js: Tweak the output format a little.
888 Change so that when there's only one run we don't write out confidence
889 intervals at all rather than writing "NaN" over and over again.
891 * sunspider: Use the time and date as a suffix on the results file, that way
892 all the results are left behind in the tmp directory. This will make it easy
893 for us to add features that compare past results in the future. Also rename
894 the .mshark files using the same scheme.
896 * sunspider-compare-results: Relax the parsing rules so we can parse the
897 results file exactly as it's emitted from sunspider.
899 2007-10-25 Adam Roben <aroben@apple.com>
901 Strip carriage returns from results
903 Carriage returns were screwing up sunspider-compare-results on
906 Rubberstamped by Oliver.
910 2007-10-24 Darin Adler <darin@apple.com>
914 * sunspider: Open the profile in shark after completing a run.
915 * sunspider-compare-results: Tiny perl tweaks.
917 2007-10-23 Maciej Stachowiak <mjs@apple.com>
919 Reviewed by Darin Adler.
921 - Don't hardcode my path to testkjs
923 * sunspider-compare-results:
925 2007-10-23 Maciej Stachowiak <mjs@apple.com>
927 Reviewed by Darin Adler.
929 - Add a compare script to compare results, and improve formatting of standard results a bit.
931 * resources/sunspider-analyze-results.js:
932 * resources/sunspider-compare-results.js: Added.
933 * sunspider-compare-results: Added.
935 2007-10-23 Maciej Stachowiak <mjs@apple.com>
937 Reviewed by Darin Adler.
939 * resources/sunspider-analyze-results.js: Do the error range
940 properly, using the t-distribution instead of 1.96 (which was
941 based on the normal distribution).
943 * sunspider: Print results in a way that will be friendlier to the
944 soon-to-come compare mode.
946 2007-10-22 Maciej Stachowiak <mjs@apple.com>
950 - add more tests (probably the final set for now)
952 Already balanced. Added date, regexp, control flow, and a few
953 more string and object/array access tests.
956 * tests/access-fannkuch.js: Added.
957 * tests/access-nbody.js: Added.
958 * tests/controlflow-recursive.js: Added.
959 * tests/date-format-tofte.js: Added.
960 * tests/date-format-xparb.js: Added.
961 * tests/regexp-dna.js: Added.
962 * tests/string-validate-input.js: Added.
964 2007-10-21 Eric Seidel <eric@webkit.org>
968 * sunspider: make --shark only sample from testkjs
970 2007-10-21 Maciej Stachowiak <mjs@apple.com>
972 Rubber stamped by Eric.
974 - added crypto tests (already balanced)
977 * tests/crypto-aes.js: Added.
978 * tests/crypto-md5.js: Added.
979 * tests/crypto-sha1.js: Added.
981 2007-10-21 Maciej Stachowiak <mjs@apple.com>
985 - rebalance new tests
987 Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
989 Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
991 This balances them with the existing tests.
993 * tests/access-binary-trees.js:
994 * tests/access-nsieve.js:
995 * tests/bitops-nsieve-bits.js:
996 * tests/math-partial-sums.js:
997 * tests/math-spectral-norm.js:
998 * tests/string-fasta.js:
1000 2007-10-20 Eric Seidel <eric@webkit.org>
1004 Add improved argument handling to sunspider, including
1005 --runs=<number>, --tests=<pattern, --shark, and --help
1006 Also re-factor code into subroutines
1008 * sunspider: improved argument handling
1010 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1014 - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1018 * tests/access-binary-trees.js: Added.
1019 * tests/access-nsieve.js: Added.
1020 * tests/bitops-nsieve-bits.js: Added.
1021 * tests/math-partial-sums.js: Added.
1022 * tests/math-spectral-norm.js: Added.
1023 * tests/string-fasta.js: Added.
1025 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1027 Reviewed by Darin Adler.
1029 * TODO: Updated to note areas that are now well-covered.
1031 - Change the tests and the driver to avoid leaving large object
1032 graphs hanging around, since that throws off the subsequent tests.
1034 * resources/sunspider-standalone-driver.js:
1036 * tests/3d-morph.js:
1037 * tests/3d-raytrace.js:
1038 * tests/bitops-bitwise-and.js:
1039 * tests/string-base64.js:
1040 * tests/string-tagcloud.js:
1042 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1044 Reviewed by Nikolas.
1046 - Rebalance test complexity.
1048 Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1050 Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1052 However, for some tests, IE7 was a huge outlier compared to other
1053 major browsers. For those tests, I used Firefox 2.0.0.8 on the
1054 same OS and hardware as a normalization basis.
1056 * tests/3d-morph.js:
1057 * tests/3d-raytrace.js:
1058 * tests/bitops-3bit-bits-in-byte.js:
1059 * tests/bitops-bits-in-byte.js:
1060 * tests/bitops-bitwise-and.js:
1061 * tests/math-cordic.js:
1062 * tests/string-base64.js:
1063 * tests/string-tagcloud.js:
1064 * tests/string-unpack-code.js:
1066 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1070 - Fix the browser-hosted driver to be compatible with Safari 2
1072 * hosted/sunspider-driver.html:
1074 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1076 Rubber stamped by Hyatt.
1078 - Add BSD license notice to cordic test, the original author said to do so.
1080 * tests/math-cordic.js:
1082 2007-10-18 Maciej Stachowiak <mjs@apple.com>
1086 - Support Windows and Windows browsers.
1088 * hosted/sunspider-results.html: Display results correctly in IE.
1089 * sunspider: tweaks to work on Windows, and with spaces in the path.
1091 2007-10-18 Maciej Stachowiak <mjs@apple.com>
1093 Rubber stamped by Oliver.
1095 * tests/string-tagcloud.js: Remove stray print()
1097 2007-10-17 Maciej Stachowiak <mjs@apple.com>
1101 Add some string test cases.
1103 * tests/string-base64.js: Added. Does base64 encoding and decoding.
1104 * tests/string-tagcloud.js: Added. Parses a JSON string and
1105 generates tagcloud HTML markup based on the data.
1106 * tests/string-unpack-code.js: Added. Unpack compressed versions of
1107 four of the most popular JavaScript libraries.
1108 * tests/LIST: Added the new tests.
1110 2007-10-16 Maciej Stachowiak <mjs@apple.com>
1114 * TODO: Updated with info on how to do the stats properly.
1116 * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1117 avoid overly slowing down the benchmark.
1119 Added some 3D-related test cases.
1121 * tests/3d-cube.js: Added.
1122 * tests/3d-morph.js: Added.
1123 * tests/3d-raytrace.js: Added.
1126 2007-10-14 Maciej Stachowiak <mjs@apple.com>
1128 Reviewed by Darin Adler.
1130 - New JavaScript benchmark
1131 http://bugs.webkit.org/show_bug.cgi?id=15515
1133 A start on a new JavaScript benchmark, with standalone and
1134 browser-hosted drivers.
1136 The test content is still incomplete and the drivers could use
1137 some more features but there is enough here to make a good start.
1139 The drivers compute a 95% confidence interval on the mean for the
1140 whole test, each category, and each individual test to make it
1141 easier to tell whether differences are statistically
1142 significant. The confidence interval can be narrowed by running
1143 the test more times.
1145 * TODO: Added. Remaining things that need to be done.
1146 * sunspider: Added. Perl script that acts as the standalone test driver.
1147 * resources/sunspider-standalone-driver.js: Added. JavaScript part
1148 of standalone test driver.
1149 * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1151 * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1152 * make-hosted: Added. Script to generate browser-hosted tests.
1153 * hosted/sunspider.html: Added. Start page for browser-hosted test.
1154 * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1155 * hosted/sunspider-results.html: Added. Results page for browser-hosted
1157 * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1159 * tests/LIST: Added. List of tests to use.
1160 * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1161 * tests/bitops-bits-in-byte.js: Added. Ditto.
1162 * tests/bitops-bitwise-and.js: Added. Ditto.
1163 * tests/math-cordic.js: Added. Ditto.