1 2013-05-03 Cosmin Truta <ctruta@blackberry.com>
3 SunSpider 1.0: 3d-morph: use epsilon to check result
4 https://bugs.webkit.org/show_bug.cgi?id=115553
6 Reviewed by George Staikos.
8 SunSpider/3d-morph should be more lenient in validating its results, since they depend on Math.sin, which isn't formally specified.
10 * tests/sunspider-1.0/3d-morph.js:
12 2013-04-19 Filip Pizlo <fpizlo@apple.com>
14 SunSpider/string-tagcloud should be more lenient in validating its results, since they depend on Math.log, which isn't formally specified
15 https://bugs.webkit.org/show_bug.cgi?id=114895
17 Reviewed by Michael Saboff.
19 * tests/sunspider-1.0/string-tagcloud.js:
21 2013-04-18 Filip Pizlo <fpizlo@apple.com>
23 Whenever it is cheap and non-invasive, SunSpider tests should validate their results to ensure that the browser runs them correctly
24 https://bugs.webkit.org/show_bug.cgi?id=114852
26 Reviewed by Geoffrey Garen.
28 This adds low-overhead checking of the results of each SunSpider tests. This is not
29 meant to be an exhaustive test that everything was executed correctly, but rather a
30 simple sanity check that will catch glaring mistakes. The philosophy here is that we're
31 not trying to prevent all forms of cheating, or that we're trying to prove the
32 browser's correctness. Moreover, these checks are meant to incur minimal overhead;
33 they currently clock in at <2% slow-down in SunSpider 1.0.
35 The test expectations were generated using the LLInt on command-line, and I've verified
36 that Firefox 20, Chrome 26, IE 10, Safari 6, and ToT with all of the JITs agree. Note
37 that some tests cannot be precisely validated because they use Math functions, which
38 ECMAScript chooses to not formally specify - those functions like sin() and friends are
39 allowed to return implementation-dependent results. Also some tests cannot be validated
40 at all because their behavior is either intentionally random or is timezone-dependent.
41 But 23 out of 26 tests now have some kind of validation.
43 I've updated the harnesses to show erroneous runs when displaying results.
45 * resources/TEMPLATE.html:
46 * resources/driver-TEMPLATE.html:
47 * resources/sunspider-analyze-results.js:
51 * resources/sunspider-compare-results.js:
55 (sunspiderCompareResults):
56 * resources/sunspider-standalone-driver.js:
57 * tests/sunspider-1.0/3d-cube.js:
59 * tests/sunspider-1.0/3d-morph.js:
60 * tests/sunspider-1.0/3d-raytrace.js:
61 * tests/sunspider-1.0/access-binary-trees.js:
62 * tests/sunspider-1.0/access-fannkuch.js:
63 * tests/sunspider-1.0/access-nbody.js:
64 * tests/sunspider-1.0/access-nsieve.js:
66 * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
67 * tests/sunspider-1.0/bitops-bits-in-byte.js:
68 * tests/sunspider-1.0/bitops-bitwise-and.js:
69 * tests/sunspider-1.0/bitops-nsieve-bits.js:
70 * tests/sunspider-1.0/controlflow-recursive.js:
71 * tests/sunspider-1.0/crypto-aes.js:
72 * tests/sunspider-1.0/crypto-md5.js:
73 * tests/sunspider-1.0/crypto-sha1.js:
74 * tests/sunspider-1.0/date-format-tofte.js:
75 * tests/sunspider-1.0/date-format-xparb.js:
76 * tests/sunspider-1.0/math-cordic.js:
77 * tests/sunspider-1.0/math-partial-sums.js:
79 * tests/sunspider-1.0/math-spectral-norm.js:
80 * tests/sunspider-1.0/regexp-dna.js:
81 * tests/sunspider-1.0/string-base64.js:
82 * tests/sunspider-1.0/string-fasta.js:
85 * tests/sunspider-1.0/string-tagcloud.js:
86 * tests/sunspider-1.0/string-unpack-code.js:
87 * tests/sunspider-1.0/string-validate-input.js:
89 2013-04-11 Filip Pizlo <fpizlo@apple.com>
91 Get rid of the 10ms delay between test executions
92 https://bugs.webkit.org/show_bug.cgi?id=114458
93 <rdar://problem/13631112>
95 Reviewed by Darin Adler and Geoffrey Garen.
97 More details on this in the bug. Long story short, I don't know why the 10ms delay is
98 there, and it appears to be the root of some noisyness and unreliability in the results.
99 Removing it makes results more consistent, which is a good thing.
101 * resources/driver-TEMPLATE.html:
103 2013-01-16 Eric Seidel <eric@webkit.org>
105 Remove --shark* support from sunspider/run-sunspider now that Shark is dead
106 https://bugs.webkit.org/show_bug.cgi?id=99512
108 Reviewed by Darin Adler.
110 I tried to use --instruments, but that also seems broken (in the same way before/after this patch).
115 2012-08-02 Filip Pizlo <fpizlo@apple.com>
117 Unreviewed, revert debugging stuff I landed in http://trac.webkit.org/changeset/124557
120 * tests/v8-v6/v8-crypto.js:
129 2012-03-11 Maciej Stachowiak <mjs@apple.com>
131 Add --instruments option to sunspider to profile with Instruments
132 https://bugs.webkit.org/show_bug.cgi?id=80783
134 Reviewed by Filip Pizlo.
136 * resources/TimeProfile20us.tracetemplate: Added.
140 2011-11-15 Andy Wingo <wingo@igalia.com>
142 Extend sunspider driver to be able to run kraken
143 https://bugs.webkit.org/show_bug.cgi?id=71799
145 Reviewed by Filip Pizlo.
147 * resources/sunspider-standalone-driver.js: Try to load a -data
148 file. If that succeeds, we have a kraken-like test, so we time
149 the test using `load'. Otherwise fall back to using `run'.
151 2011-08-11 Mark Rowe <mrowe@apple.com>
153 Fix sunspider-compare-results to default to the same suite version as the sunspider script.
155 Rubber-stamped by Maciej Stachowiak.
157 * sunspider-compare-results:
159 2011-07-02 Maciej Stachowiak <mjs@apple.com>
161 Update SunSpider versioning to 1.0
162 https://bugs.webkit.org/show_bug.cgi?id=63870
164 Reviewed by Dan Bernstein.
166 * hosted/sunspider.html:
167 * hosted/versions.html:
168 * resources/driver-TEMPLATE.html:
169 * resources/results-TEMPLATE.html:
172 2011-07-02 Maciej Stachowiak <mjs@apple.com>
174 Sunspider 0.9.1 harness doesn't actually close() its test documents
175 https://bugs.webkit.org/show_bug.cgi?id=47045
177 Reviewed by Daniel Bates.
179 * hosted/versions.html:
180 * resources/driver-TEMPLATE.html:
182 2011-07-02 Maciej Stachowiak <mjs@apple.com>
186 Fix obviou typo in previous commit.
188 * tests/sunspider-1.0/bitops-bitwise-and.js:
190 2011-07-02 Maciej Stachowiak <mjs@apple.com>
192 Reviewed by Anders Carlsson.
194 SunSpider: all four bitops benchmarks can be replaced with NOP
195 https://bugs.webkit.org/show_bug.cgi?id=38446
197 * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
198 (TimeFunc): Save the result.
199 * tests/sunspider-1.0/bitops-bits-in-byte.js:
200 (TimeFunc): Save the result.
201 * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
203 * tests/sunspider-1.0/bitops-nsieve-bits.js:
204 (sieve): Save the result.
206 2011-07-02 Maciej Stachowiak <mjs@apple.com>
208 SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
209 https://bugs.webkit.org/show_bug.cgi?id=60937
211 Reviewed by Dan Bernstein.
213 * tests/sunspider-1.0/string-validate-input.js:
214 (doTest): Rename the variable to username
216 2011-07-02 Maciej Stachowiak <mjs@apple.com>
218 Reviewed by Darin Adler.
220 SunSpider: The main function of math-cordic is dead code
221 https://bugs.webkit.org/show_bug.cgi?id=63863
223 Try to prevent math-cordic from being eliminated as dead code or
224 compiled down to a constant.
226 * tests/sunspider-1.0/math-cordic.js:
230 2011-06-30 Maciej Stachowiak <mjs@apple.com>
232 Reviewed by Adam Barth.
234 Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
235 https://bugs.webkit.org/show_bug.cgi?id=63782
237 * make-hosted: Learn about the new directory.
238 * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
240 2010-11-29 Geoffrey Garen <ggaren@apple.com>
242 Reviewed by Gavin Barraclough.
244 Improved accuracy of command-line SunSpider.
246 * resources/sunspider-standalone-driver.js: Use the returned number of
247 elapsed milliseconds from "run" and "checkSyntax" instead of measuring
248 ourselves, for slightly more accurate numbers.
250 2010-09-16 Gavin Barraclough <barraclough@apple.com>
252 Reviewed by Mark Rowe.
253 https://bugs.webkit.org/show_bug.cgi?id=45924
255 Modify the SunSpider harness to allow a path to be passed to the --suite
256 argument, supress the normal prepend-"tests/" behaviour for suite names
259 Also wrap the code in an anonymous function to move variables out of global
260 scope (currently the harness cannot handle running tests the overwrite the
263 * resources/sunspider-standalone-driver.js:
266 2010-08-27 Michael Saboff <msaboff@apple.com>
268 Reviewed by Stephanie Lewis.
270 Added version 5 and version 6 of V8 tests with modifications similar
271 to the version 4 tests so they'll work with the SunSpider test harness.
272 Note that these files are external source and therefore don't conform
273 to WebKit coding standards.
274 https://bugs.webkit.org/show_bug.cgi?id=44638
276 * tests/v8-v5: Added.
277 * tests/v8-v5/LIST: Added.
278 * tests/v8-v5/v8-crypto.js: Added.
279 * tests/v8-v5/v8-deltablue.js: Added.
280 * tests/v8-v5/v8-earley-boyer.js: Added.
281 * tests/v8-v5/v8-raytrace.js: Added.
282 * tests/v8-v5/v8-regexp.js: Added.
283 * tests/v8-v5/v8-richards.js: Added.
284 * tests/v8-v5/v8-splay.js: Added.
285 * tests/v8-v6: Added.
286 * tests/v8-v6/LIST: Added.
287 * tests/v8-v6/v8-crypto.js: Added.
288 * tests/v8-v6/v8-deltablue.js: Added.
289 * tests/v8-v6/v8-earley-boyer.js: Added.
290 * tests/v8-v6/v8-raytrace.js: Added.
291 * tests/v8-v6/v8-regexp.js: Added.
292 * tests/v8-v6/v8-richards.js: Added.
293 * tests/v8-v6/v8-splay.js: Added.
295 2010-07-15 Stephanie Lewis <slewis@apple.com>
297 Reviewed by Geoff Garen.
299 http://bugs.webkit.org/show_bug.cgi?id=42406
300 Output location of results file.
304 2010-05-17 Maciej Stachowiak <mjs@apple.com>
306 Reviewed by Mark Rowe.
308 Browser-hosted version of SunSpider runs tests twice as many times as advertised
309 https://bugs.webkit.org/show_bug.cgi?id=39201
311 * make-hosted: Make sure to reset test list after every test suite.
313 2010-05-07 Adam Roben <aroben@apple.com>
315 Fix typo in sunspider-compare-results.js that caused incorrect
316 two-sample t statistics to be computed
318 Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
319 two-sample t statistics
321 Reviewed by Darin Adler.
323 * resources/sunspider-compare-results.js:
324 (sunspiderCompareResults.resultLine): Fixed the calculation of the t
325 variable to match the definition of a two-sample t statistic. Looks
326 like this was just a simple typo.
328 2010-04-05 Darin Adler <darin@apple.com>
330 Updated the ignore property for changes to the contents here.
332 * .: Modified property svn:ignore.
334 2010-02-20 Maciej Stachowiak <mjs@apple.com>
336 Reviewed by Adam Barth.
338 Add sunspider-0.9.1 to the site in preparation for announcing it.
339 https://bugs.webkit.org/show_bug.cgi?id=35206
341 * hosted/sunspider.css: Remove tabs, which I noticed where present while
342 copying to the WebKitSite directory.
344 2010-01-11 Oliver Hunt <oliver@apple.com>
346 Reviewed by Darin Adler.
348 Sunspider parse-only tests don't run
349 https://bugs.webkit.org/show_bug.cgi?id=33489
351 Fix parse-only test list
353 * tests/parse-only/LIST:
355 2009-12-15 Maciej Stachowiak <mjs@apple.com>
357 Reviewed by Adele Peterson.
359 Make SunSpider version more prominent in the title
360 https://bugs.webkit.org/show_bug.cgi?id=32574
362 * hosted/sunspider.html:
363 * hosted/versions.html:
364 * resources/driver-TEMPLATE.html:
365 * resources/results-TEMPLATE.html:
367 2009-12-15 Maciej Stachowiak <mjs@apple.com>
369 Reviewed by Darin Adler.
371 Make SunSpider refuse to compare results across content versions
372 https://bugs.webkit.org/show_bug.cgi?id=32573
374 The results URL now includes a version indicator, and compare mode
375 will refuse to compare between versions.
377 * resources/driver-TEMPLATE.html:
378 * resources/results-TEMPLATE.html:
380 2009-12-14 Maciej Stachowiak <mjs@apple.com>
382 Reviewed by Darin Adler.
384 Some Browser-hosted SunSpider files are not valid HTML5
385 https://bugs.webkit.org/show_bug.cgi?id=32536
387 Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
388 close tags and adding a meta charset declaration).
390 * hosted/sunspider.html:
391 * hosted/versions.html:
392 * resources/TEMPLATE.html:
393 * resources/driver-TEMPLATE.html:
394 * resources/results-TEMPLATE.html:
396 2009-12-14 Maciej Stachowiak <mjs@apple.com>
398 Reviewed by Eric Seidel.
400 Make sunspider-0.9.1 the default content set (both command-line and hosted)
401 https://bugs.webkit.org/show_bug.cgi?id=32537
403 * hosted/sunspider.html: Change default suite link.
404 * sunspider: Change default suite.
405 * sunspider-compare-results: Likewise.
407 2009-12-14 Maciej Stachowiak <mjs@apple.com>
409 Reviewed by Sam Weinig.
411 in-browser SunSpider suffers excessive penalty under power management
412 https://bugs.webkit.org/show_bug.cgi?id=32505
414 I have made a few changes to address this:
416 1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
417 document.write to reduce triggering of progress UI.
418 2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
419 stepped all the way down the whole time.
420 3) Run the test cycle an extra time for warmup before the runs that count.
422 * hosted/sunspider-record-result.js: Removed.
424 * resources/TEMPLATE.html:
425 * resources/driver-TEMPLATE.html:
427 2009-12-13 Maciej Stachowiak <mjs@apple.com>
429 Reviewed by Gavin Barraclough.
431 SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
432 https://bugs.webkit.org/show_bug.cgi?id=16806
434 Based on a patch by Eric Seidel.
436 Fix the base64 computation to actually compute correct results. The impact on runtime of
437 the test is pretty small, but noticeable for some browsers. But at least it's not
438 doing a wrong and meaningless computation any more.
440 * tests/sunspider-0.9.1/string-base64.js:
444 2009-12-13 Maciej Stachowiak <mjs@apple.com>
446 Fixing commit error...
448 I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
450 * resources/driver-TEMPLATE.html:
452 2009-12-13 Maciej Stachowiak <mjs@apple.com>
454 Reviewed by Sam Weinig.
456 Use JSON.parse instead of eval for Web-hosted SunSpider results processing
457 https://bugs.webkit.org/show_bug.cgi?id=32490
459 * hosted/json2.js: Added. Incorporated from json.org
460 * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
461 * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
464 2009-12-12 Maciej Stachowiak <mjs@apple.com>
466 Reviewed by Oliver Hunt.
468 Enable Web-hosted version of SunSpider to handle multiple versions
469 https://bugs.webkit.org/show_bug.cgi?id=32478
471 * make-hosted: Now generate the hosted version to be able to run both the
472 0.9 and the 0.9.1 test suites.
473 * hosted: Modified property svn:ignore.
474 * hosted/sunspider-driver.html: Removed.
475 * hosted/sunspider-results.html: Removed.
476 * hosted/sunspider.html:
477 * hosted/versions.html: Added.
478 * resources/TEMPLATE.html:
479 * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
480 * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
482 2009-12-12 Maciej Stachowiak <mjs@apple.com>
484 Reviewed by Oliver Hunt.
486 Give command-line sunspider the ability to handle multiple suites and versions
487 https://bugs.webkit.org/show_bug.cgi?id=32477
489 Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
490 enumerating the tests. Current suites are:
492 sunspider-0.9 - riginal SunSpider content
493 sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
494 ubench - "ubench" microbenchmark suite
495 v8-v4 - version 4 of the v8 benchmark
496 parse-only - some parser tests
498 Sunspider can now be run with the --suite= parameter to select a
499 suite. The default is --suite=sunspider-0.9.
501 * .: Modified property svn:ignore to ignore new results directories.
502 * make-hosted: Generate based on sunspider-0.9 suite.
503 * resources/TEMPLATE.html: Fix an HTML compliance problem.
504 * resources/sunspider-standalone-driver.js: Become aware of suite names.
505 * sunspider: Modified to handle multiple suites as described above.
506 * sunspider-compare-results: ditto
507 * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
508 * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
509 * tests/LIST-UBENCH: Removed.
510 * tests/LIST-V8: Removed.
511 * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
512 * tests/3d-morph.js: ditto
513 * tests/3d-raytrace.js: ditto
514 * tests/access-binary-trees.js: ditto
515 * tests/access-fannkuch.js: ditto
516 * tests/access-nbody.js: ditto
517 * tests/access-nsieve.js: ditto
518 * tests/bitops-3bit-bits-in-byte.js: ditto
519 * tests/bitops-bits-in-byte.js: ditto
520 * tests/bitops-bitwise-and.js: ditto
521 * tests/bitops-nsieve-bits.js: ditto
522 * tests/controlflow-recursive.js: ditto
523 * tests/crypto-aes.js: ditto
524 * tests/crypto-md5.js: ditto
525 * tests/crypto-sha1.js: ditto
526 * tests/date-format-tofte.js: ditto
527 * tests/date-format-xparb.js: ditto
528 * tests/math-cordic.js: ditto
529 * tests/math-partial-sums.js: ditto
530 * tests/math-spectral-norm.js: ditto
531 * tests/regexp-dna.js: ditto
532 * tests/string-base64.js: ditto
533 * tests/string-fasta.js: ditto
534 * tests/string-tagcloud.js: ditto
535 * tests/string-unpack-code.js: ditto
536 * tests/string-validate-input.js: ditto
537 * tests/sunspider-0.9/: Added
538 * tests/sunspider-0.9/*.js: Moved from one level up
539 * tests/sunspider-0.9.1: Added.
540 * tests/sunspider-0.9.1/*.js: Moved from one level up.
541 * tests/v8-crypto.js: Moved to v8-v4/ directory.
542 * tests/v8-deltablue.js: ditto
543 * tests/v8-earley-boyer.js: ditto
544 * tests/v8-raytrace.js: ditto
545 * tests/v8-regexp.js: ditto
546 * tests/v8-richards.js: ditto
547 * tests/v8-splay.js: ditto
548 * tests/v8-v4: Added.
550 2009-09-30 Csaba Osztrogonac <oszi@inf.u-szeged.hu>
552 Reviewed by Darin Adler.
554 Platform specific null device replaced with a platform independent.
555 https://bugs.webkit.org/show_bug.cgi?id=29544
557 * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
559 2009-09-21 Csaba Osztrogonac <oszi@inf.u-szeged.hu>
561 Reviewed by Maciej Stachowiak.
563 [Qt] Make sunspider script work on Windows platform.
564 https://bugs.webkit.org/show_bug.cgi?id=29544
567 Using platform specific null device instead of /dev/null.
569 2009-06-19 Adam Treat <adam.treat@torchmobile.com>
571 Reviewed by Oliver Hunt.
573 https://bugs.webkit.org/show_bug.cgi?id=26540
574 Currently the SunSpider test driver lacks an option to run a test suite that
575 will test JavaScriptCore parsing performance only. This patch adds just such
576 a test suite and option to SunSpider as well as the jsc test shell. I've included
577 three large javascript source files found in the wild: jquery, mootools and prototype.
578 Combined with the concatenation of all three, these form a new testsuite to measure
579 and test pure JavaScriptCore parsing performance.
581 * resources/sunspider-standalone-driver.js:
582 Load the js with parse-only flag if it is part of the parse-only suite.
584 Add the parse-only suite to the list of options.
585 * sunspider-compare-results:
586 Add the parse-only suite to the list of options.
587 * tests/LIST-PARSE-ONLY: Added.
588 * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
589 A very large concatenation of the scripts found below. This js takes ~30ms to parse on my machine.
590 * tests/parse-only/jquery-1.3.2.js: Added.
591 Latest version from jquery.com.
592 * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
593 Latest version from mootools.net.
594 * tests/parse-only/prototype-1.6.0.3.js: Added.
595 Latest version from prototypejs.org.
597 2009-06-13 Steve Falkenburg <sfalken@apple.com>
599 Update V8 benchmark to version 4.
601 Reviewed by Cameron Zwarich.
604 * tests/v8-crypto.js:
605 * tests/v8-deltablue.js:
606 * tests/v8-earley-boyer.js:
607 * tests/v8-raytrace.js:
608 * tests/v8-regexp.js: Added.
609 * tests/v8-richards.js:
610 * tests/v8-splay.js: Added.
612 2009-05-26 Olivier DOLE <odole@pleyo.com>
616 Store sunspider test suite results in a specific directory for each
617 different type of benchmark (sunspider, V8 or ubench).
620 * sunspider-compare-results:
622 2009-05-12 Steve Falkenburg <sfalken@apple.com>
628 2009-05-11 Steve Falkenburg <sfalken@apple.com>
630 Include SunSpider tests in production builds.
632 Reviewed by Adam Roben.
634 * SunSpider.make: Added.
635 * xcopy.excludes: Added.
637 2009-03-04 Oliver Hunt <oliver@apple.com>
639 Added a few more tests to ubench
643 Adding a couple of test to cover too few and too many arguments. This
644 is needed to prevent regressions in all call paths from my current refactoring.
647 * tests/ubench/function-correct-args.js: Added.
649 * tests/ubench/function-excess-args.js: Added.
652 2008-10-30 Zoltan Horvath <Horvath.Zoltan.6@stud.u-szeged.hu>
654 Fix a bug where when V8's or UBENCH's test suite list file is missing
655 the error message is suggesting that SunSpider's list file is missing.
657 Reviewed by Sam Weinig.
660 (loadTestsList): Fix the error message to use the right list name.
662 2008-10-07 David Hyatt <hyatt@apple.com>
664 https://bugs.webkit.org/show_bug.cgi?id=18989
666 Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
667 we want all browsers to be doing the same thing on the test.
671 * tests/regexp-dna.js:
673 2008-09-09 Cameron Zwarich <cwzwarich@uwaterloo.ca>
675 Rubber-stamped by Maciej Stachowiak.
677 The bleeding edge branch of V8 now supports passing fileanmes after
678 '-f', so the SunSpider script should be changed to do this again.
679 This allows all 3 major JavaScript engines to be used with the
680 unmodified SunSpider script.
684 2008-09-04 Cameron Zwarich <cwzwarich@uwaterloo.ca>
686 Reviewed by Maciej Stachowiak.
688 Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
689 and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
690 correct the punctuation in its documentation.
694 2008-09-04 Cameron Zwarich <cwzwarich@uwaterloo.ca>
696 Reviewed by Oliver Hunt.
698 Bug 20616: Incorporate V8 benchmarks in testing
699 <https://bugs.webkit.org/show_bug.cgi?id=20616>
701 This is a first attempt at running the V8 benchmarks within SunSpider,
702 with the --v8 option. The number of iterations is fixed for each
703 benchmark, and was chosen to make each of the benchmarks run between
704 500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
707 * tests/LIST-V8: Added.
708 * tests/v8-crypto.js: Added.
709 * tests/v8-deltablue.js: Added.
710 * tests/v8-earley-boyer.js: Added.
711 * tests/v8-raytrace.js: Added.
712 * tests/v8-richards.js: Added.
714 2008-09-02 Cameron Zwarich <cwzwarich@uwaterloo.ca>
718 Remove a local change that was accidentally committed in r36034.
720 * resources/sunspider-standalone-driver.js:
722 2008-09-02 Cameron Zwarich <cwzwarich@uwaterloo.ca>
724 Rubber-stamped by Oliver Hunt.
726 Make SunSpider work with JavaScript shells that don't take filenames
727 after a '-f' argument.
731 2008-06-07 Cameron Zwarich <cwzwarich@uwaterloo.ca>
735 Remove the --squirrelfish option from SunSpider, as it is no longer needed.
738 * tests/LIST-SQUIRRELFISH: Removed.
740 2008-06-03 Alexey Proskuryakov <ap@webkit.org>
742 Rubber-stamped by Mark Rowe.
744 Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
748 2008-04-15 Maciej Stachowiak <mjs@apple.com>
750 Rubber stamped by Oliver.
752 - add newly working tests to squirrelfish list
754 * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
756 2008-04-14 Maciej Stachowiak <mjs@apple.com>
758 Rubber stamped by Oliver.
760 - move ubench tests properly.
762 * tests/ubench: Added.
763 * tests/ubench/function-closure.js: Added.
764 * tests/ubench/function-empty.js: Added.
765 * tests/ubench/function-missing-args.js: Added.
766 * tests/ubench/function-sum.js: Added.
767 * tests/ubench/loop-empty-resolve.js: Added.
768 * tests/ubench/loop-empty.js: Added.
769 * tests/ubench/loop-sum.js: Added.
771 2008-04-14 Maciej Stachowiak <mjs@apple.com>
773 Rubber stamped by Oliver.
775 * sunspider: Don't force --runs=1 in squirrelfish mode any more.
777 2008-04-14 Maciej Stachowiak <mjs@apple.com>
781 - make --squirrelfish mode use the real harness, but a reduced set of tests
783 Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
786 * tests/LIST-SQUIRRELFISH:
787 * tests/LIST-UBENCH: Added.
788 * tests/squirrelfish: Moved to tests/ubench
790 2008-04-14 Geoffrey Garen <ggaren@apple.com>
792 Reviewed by Maciej Stachowiak.
794 Removed an unused test.
796 * tests/squirrelfish/loop-resolve.js: Removed.
798 2008-04-10 Maciej Stachowiak <mjs@apple.com>
802 Add newly runnable tests to --squirrelfish mode.
804 * tests/LIST-SQUIRRELFISH:
806 2008-04-06 Geoffrey Garen <ggaren@apple.com>
808 Reviewed by Maciej Stachowiak.
810 A little more love for --squirrelfish mode:
812 Fixed a misplaced sort. Results properly sort now.
814 Added a test list just for squirrelfish, and updated it to include
815 all currently passing SunSpider tests.
817 Fixed the pruning regexp to match 3d-morph.js.
821 2008-03-26 Geoffrey Garen <ggaren@apple.com>
823 Reviewed by Oliver Hunt.
825 --squirrelfish mode: pared down tests for squirrelfish to chew on.
829 * tests/squirrelfish: Added.
830 * tests/squirrelfish/function-closure.js: Added.
831 * tests/squirrelfish/function-empty.js: Added.
832 * tests/squirrelfish/function-missing-args.js: Added.
833 * tests/squirrelfish/function-sum.js: Added.
834 * tests/squirrelfish/loop-empty-resolve.js: Added.
835 * tests/squirrelfish/loop-empty.js: Added.
836 * tests/squirrelfish/loop-resolve.js: Added.
837 * tests/squirrelfish/loop-sum.js: Added.
839 2008-03-21 Alexey Proskuryakov <ap@webkit.org>
843 * sunspider: Pause update daemon on Darwin to make results reliable
844 again (workaround for <rdar://problem/5811127>).
846 2008-02-07 Eric Seidel <eric@webkit.org>
850 * sunspider: add an error message when ./tests/LIST is not found
852 2007-12-29 Alp Toker <alp@atoker.com>
856 * hosted/sunspider.html:
858 2007-12-29 Alp Toker <alp@atoker.com>
862 Provide the current URL in the SunSpider results page. Useful for
863 copying benchmark results out of browsers without a location bar or
864 where the location bar can't handle long URLs.
866 * hosted/sunspider-results.html:
868 2007-12-20 Eric Seidel <eric@webkit.org>
872 * sunspider: increase default number of runs for more accurate numbers
874 2007-12-18 Mark Rowe <mrowe@apple.com>
878 * hosted/sunspider.html:
880 2007-12-18 Maciej Stachowiak <mjs@apple.com>
884 - give an extra digit of precision for small differences.
886 * resources/sunspider-compare-results.js:
888 2007-12-17 Maciej Stachowiak <mjs@apple.com>
892 - work around IE layout bugs
894 * hosted/sunspider.css:
896 2007-12-17 Maciej Stachowiak <mjs@apple.com>
900 - fix compare feature
902 * hosted/sunspider-results.html:
904 2007-12-17 Maciej Stachowiak <mjs@apple.com>
908 - Change the format of the output URL to be shorter, to meet IE constraints
910 * hosted/sunspider-driver.html:
911 * hosted/sunspider-results.html:
913 2007-12-13 Maciej Stachowiak <mjs@apple.com>
917 - Add some explanatory test and a dumb logo to SunSpider web hosted version
919 * hosted/sunspider-driver.html:
920 * hosted/sunspider-results.html:
921 * hosted/sunspider.css:
922 * hosted/sunspider.html:
924 2007-12-13 Maciej Stachowiak <mjs@apple.com>
928 - Make the web-hosted version of SunSpider look a little bit less drab.
930 * hosted/sunspider-driver.html:
931 * hosted/sunspider-results.html:
932 * hosted/sunspider.css: Added.
933 * hosted/sunspider.html:
934 * resources/TEMPLATE.html:
936 2007-12-12 Maciej Stachowiak <mjs@apple.com>
940 - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
942 A sample of what new-style results would look like:
944 TEST COMPARISON FROM TO DETAILS
946 ** TOTAL **: 1.41x as fast 6613.6ms +/- 0.1% 4703.0ms +/- 0.1% significant
948 3bit-bits-in-byte: *1.36x as slow* 88.8ms +/- 0.6% 121.1ms +/- 0.6% significant
950 * resources/sunspider-compare-results.js:
952 2007-11-14 Eric Seidel <eric@webkit.org>
956 * sunspider: add --shark-cache for L2 Cache Miss profiling
958 2007-11-09 Alexey Proskuryakov <ap@webkit.org>
960 Reviewed by Darin Adler.
962 http://bugs.webkit.org/show_bug.cgi?id=15925
963 SunSpider should check for Shark being installed
965 * sunspider: Explicitly test that shark command line tool is installed.
967 2007-11-04 Darin Adler <darin@apple.com>
969 * hosted/sunspider.html: Tweaked formatting a bit.
971 2007-11-03 Maciej Stachowiak <mjs@apple.com>
973 Reviewed by Darin Adler.
975 - Add compare mode and a description of the benchmark to the hosted version.
977 * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
978 * hosted/sunspider.html: Added intro text.
979 * make-hosted: Copy new file.
980 * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
982 * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
984 * sunspider-compare-results: Use sunspider-standalone-compare.js.
986 2007-11-02 Darin Adler <darin@apple.com>
988 * sunspider: Changed "--base" to "--set-baseline".
990 2007-11-02 Darin Adler <darin@apple.com>
994 * sunspider: Added a "--base" option that records the current test as a baseline.
995 Moved the code to find the newest .mshark file into a function (since I needed to use
996 it in the other script and I prefer to copy/paste a function).
998 * sunspider-compare-results: Added logic to compare the baseline with the last
999 run when you don't pass any parameters.
1001 2007-10-29 Maciej Stachowiak <mjs@apple.com>
1005 - Added list of ECMAScript3 features that SunSpider does not cover currently.
1009 2007-10-29 Maciej Stachowiak <mjs@apple.com>
1013 - Give better error messages when sunspider-compare-results fails
1015 * sunspider-compare-results:
1017 2007-10-29 Maciej Stachowiak <mjs@apple.com>
1019 Reviewed by Darin Adler.
1021 - fix Date tests to work in command-line SpiderMonkey
1023 * tests/date-format-tofte.js:
1024 * tests/date-format-xparb.js:
1026 2007-10-29 Darin Adler <darin@apple.com>
1030 * sunspider: Added --shark20 option, to run Shark at its highest sample
1031 resolution instead of its default.
1033 2007-10-28 Adam Roben <aroben@apple.com>
1035 Fix SunSpider on Windows
1039 * sunspider: Don't use colons in filenames.
1041 2007-10-28 Darin Adler <darin@apple.com>
1045 * resources/sunspider-analyze-results.js: Tweak the output format a little.
1046 Change so that when there's only one run we don't write out confidence
1047 intervals at all rather than writing "NaN" over and over again.
1049 * sunspider: Use the time and date as a suffix on the results file, that way
1050 all the results are left behind in the tmp directory. This will make it easy
1051 for us to add features that compare past results in the future. Also rename
1052 the .mshark files using the same scheme.
1054 * sunspider-compare-results: Relax the parsing rules so we can parse the
1055 results file exactly as it's emitted from sunspider.
1057 2007-10-25 Adam Roben <aroben@apple.com>
1059 Strip carriage returns from results
1061 Carriage returns were screwing up sunspider-compare-results on
1064 Rubberstamped by Oliver.
1068 2007-10-24 Darin Adler <darin@apple.com>
1072 * sunspider: Open the profile in shark after completing a run.
1073 * sunspider-compare-results: Tiny perl tweaks.
1075 2007-10-23 Maciej Stachowiak <mjs@apple.com>
1077 Reviewed by Darin Adler.
1079 - Don't hardcode my path to testkjs
1081 * sunspider-compare-results:
1083 2007-10-23 Maciej Stachowiak <mjs@apple.com>
1085 Reviewed by Darin Adler.
1087 - Add a compare script to compare results, and improve formatting of standard results a bit.
1089 * resources/sunspider-analyze-results.js:
1090 * resources/sunspider-compare-results.js: Added.
1091 * sunspider-compare-results: Added.
1093 2007-10-23 Maciej Stachowiak <mjs@apple.com>
1095 Reviewed by Darin Adler.
1097 * resources/sunspider-analyze-results.js: Do the error range
1098 properly, using the t-distribution instead of 1.96 (which was
1099 based on the normal distribution).
1101 * sunspider: Print results in a way that will be friendlier to the
1102 soon-to-come compare mode.
1104 2007-10-22 Maciej Stachowiak <mjs@apple.com>
1108 - add more tests (probably the final set for now)
1110 Already balanced. Added date, regexp, control flow, and a few
1111 more string and object/array access tests.
1114 * tests/access-fannkuch.js: Added.
1115 * tests/access-nbody.js: Added.
1116 * tests/controlflow-recursive.js: Added.
1117 * tests/date-format-tofte.js: Added.
1118 * tests/date-format-xparb.js: Added.
1119 * tests/regexp-dna.js: Added.
1120 * tests/string-validate-input.js: Added.
1122 2007-10-21 Eric Seidel <eric@webkit.org>
1126 * sunspider: make --shark only sample from testkjs
1128 2007-10-21 Maciej Stachowiak <mjs@apple.com>
1130 Rubber stamped by Eric.
1132 - added crypto tests (already balanced)
1135 * tests/crypto-aes.js: Added.
1136 * tests/crypto-md5.js: Added.
1137 * tests/crypto-sha1.js: Added.
1139 2007-10-21 Maciej Stachowiak <mjs@apple.com>
1143 - rebalance new tests
1145 Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1147 Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1149 This balances them with the existing tests.
1151 * tests/access-binary-trees.js:
1152 * tests/access-nsieve.js:
1153 * tests/bitops-nsieve-bits.js:
1154 * tests/math-partial-sums.js:
1155 * tests/math-spectral-norm.js:
1156 * tests/string-fasta.js:
1158 2007-10-20 Eric Seidel <eric@webkit.org>
1162 Add improved argument handling to sunspider, including
1163 --runs=<number>, --tests=<pattern, --shark, and --help
1164 Also re-factor code into subroutines
1166 * sunspider: improved argument handling
1168 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1172 - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1176 * tests/access-binary-trees.js: Added.
1177 * tests/access-nsieve.js: Added.
1178 * tests/bitops-nsieve-bits.js: Added.
1179 * tests/math-partial-sums.js: Added.
1180 * tests/math-spectral-norm.js: Added.
1181 * tests/string-fasta.js: Added.
1183 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1185 Reviewed by Darin Adler.
1187 * TODO: Updated to note areas that are now well-covered.
1189 - Change the tests and the driver to avoid leaving large object
1190 graphs hanging around, since that throws off the subsequent tests.
1192 * resources/sunspider-standalone-driver.js:
1194 * tests/3d-morph.js:
1195 * tests/3d-raytrace.js:
1196 * tests/bitops-bitwise-and.js:
1197 * tests/string-base64.js:
1198 * tests/string-tagcloud.js:
1200 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1202 Reviewed by Nikolas.
1204 - Rebalance test complexity.
1206 Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1208 Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1210 However, for some tests, IE7 was a huge outlier compared to other
1211 major browsers. For those tests, I used Firefox 2.0.0.8 on the
1212 same OS and hardware as a normalization basis.
1214 * tests/3d-morph.js:
1215 * tests/3d-raytrace.js:
1216 * tests/bitops-3bit-bits-in-byte.js:
1217 * tests/bitops-bits-in-byte.js:
1218 * tests/bitops-bitwise-and.js:
1219 * tests/math-cordic.js:
1220 * tests/string-base64.js:
1221 * tests/string-tagcloud.js:
1222 * tests/string-unpack-code.js:
1224 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1228 - Fix the browser-hosted driver to be compatible with Safari 2
1230 * hosted/sunspider-driver.html:
1232 2007-10-20 Maciej Stachowiak <mjs@apple.com>
1234 Rubber stamped by Hyatt.
1236 - Add BSD license notice to cordic test, the original author said to do so.
1238 * tests/math-cordic.js:
1240 2007-10-18 Maciej Stachowiak <mjs@apple.com>
1244 - Support Windows and Windows browsers.
1246 * hosted/sunspider-results.html: Display results correctly in IE.
1247 * sunspider: tweaks to work on Windows, and with spaces in the path.
1249 2007-10-18 Maciej Stachowiak <mjs@apple.com>
1251 Rubber stamped by Oliver.
1253 * tests/string-tagcloud.js: Remove stray print()
1255 2007-10-17 Maciej Stachowiak <mjs@apple.com>
1259 Add some string test cases.
1261 * tests/string-base64.js: Added. Does base64 encoding and decoding.
1262 * tests/string-tagcloud.js: Added. Parses a JSON string and
1263 generates tagcloud HTML markup based on the data.
1264 * tests/string-unpack-code.js: Added. Unpack compressed versions of
1265 four of the most popular JavaScript libraries.
1266 * tests/LIST: Added the new tests.
1268 2007-10-16 Maciej Stachowiak <mjs@apple.com>
1272 * TODO: Updated with info on how to do the stats properly.
1274 * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1275 avoid overly slowing down the benchmark.
1277 Added some 3D-related test cases.
1279 * tests/3d-cube.js: Added.
1280 * tests/3d-morph.js: Added.
1281 * tests/3d-raytrace.js: Added.
1284 2007-10-14 Maciej Stachowiak <mjs@apple.com>
1286 Reviewed by Darin Adler.
1288 - New JavaScript benchmark
1289 http://bugs.webkit.org/show_bug.cgi?id=15515
1291 A start on a new JavaScript benchmark, with standalone and
1292 browser-hosted drivers.
1294 The test content is still incomplete and the drivers could use
1295 some more features but there is enough here to make a good start.
1297 The drivers compute a 95% confidence interval on the mean for the
1298 whole test, each category, and each individual test to make it
1299 easier to tell whether differences are statistically
1300 significant. The confidence interval can be narrowed by running
1301 the test more times.
1303 * TODO: Added. Remaining things that need to be done.
1304 * sunspider: Added. Perl script that acts as the standalone test driver.
1305 * resources/sunspider-standalone-driver.js: Added. JavaScript part
1306 of standalone test driver.
1307 * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1309 * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1310 * make-hosted: Added. Script to generate browser-hosted tests.
1311 * hosted/sunspider.html: Added. Start page for browser-hosted test.
1312 * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1313 * hosted/sunspider-results.html: Added. Results page for browser-hosted
1315 * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1317 * tests/LIST: Added. List of tests to use.
1318 * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1319 * tests/bitops-bits-in-byte.js: Added. Ditto.
1320 * tests/bitops-bitwise-and.js: Added. Ditto.
1321 * tests/math-cordic.js: Added. Ditto.