Move SunSpider into PerformanceTests
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
2
3         Reviewed by Gavin Barraclough.
4
5         Improved accuracy of command-line SunSpider.
6
7         * resources/sunspider-standalone-driver.js: Use the returned number of
8         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
9         ourselves, for slightly more accurate numbers.
10
11 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
12
13         Reviewed by Mark Rowe.
14         https://bugs.webkit.org/show_bug.cgi?id=45924
15
16         Modify the SunSpider harness to allow a path to be passed to the --suite
17         argument, supress the normal prepend-"tests/" behaviour for suite names
18         containing a '/'.
19
20         Also wrap the code in an anonymous function to move variables out of global
21         scope (currently the harness cannot handle running tests the overwrite the
22         variable 'j').
23
24         * resources/sunspider-standalone-driver.js:
25         * sunspider:
26
27 2010-08-27  Michael Saboff  <msaboff@apple.com>
28
29         Reviewed by Stephanie Lewis.
30
31         Added version 5 and version 6 of V8 tests with modifications similar
32         to the version 4 tests so they'll work with the SunSpider test harness.
33         Note that these files are external source and therefore don't conform 
34         to WebKit coding standards.
35         https://bugs.webkit.org/show_bug.cgi?id=44638
36
37         * tests/v8-v5: Added.
38         * tests/v8-v5/LIST: Added.
39         * tests/v8-v5/v8-crypto.js: Added.
40         * tests/v8-v5/v8-deltablue.js: Added.
41         * tests/v8-v5/v8-earley-boyer.js: Added.
42         * tests/v8-v5/v8-raytrace.js: Added.
43         * tests/v8-v5/v8-regexp.js: Added.
44         * tests/v8-v5/v8-richards.js: Added.
45         * tests/v8-v5/v8-splay.js: Added.
46         * tests/v8-v6: Added.
47         * tests/v8-v6/LIST: Added.
48         * tests/v8-v6/v8-crypto.js: Added.
49         * tests/v8-v6/v8-deltablue.js: Added.
50         * tests/v8-v6/v8-earley-boyer.js: Added.
51         * tests/v8-v6/v8-raytrace.js: Added.
52         * tests/v8-v6/v8-regexp.js: Added.
53         * tests/v8-v6/v8-richards.js: Added.
54         * tests/v8-v6/v8-splay.js: Added.
55
56 2010-07-15  Stephanie Lewis  <slewis@apple.com>
57
58         Reviewed by Geoff Garen.
59
60         http://bugs.webkit.org/show_bug.cgi?id=42406
61         Output location of results file.
62
63         * sunspider:
64
65 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
66
67         Reviewed by Mark Rowe.
68
69         Browser-hosted version of SunSpider runs tests twice as many times as advertised
70         https://bugs.webkit.org/show_bug.cgi?id=39201
71
72         * make-hosted: Make sure to reset test list after every test suite.
73
74 2010-05-07  Adam Roben  <aroben@apple.com>
75
76         Fix typo in sunspider-compare-results.js that caused incorrect
77         two-sample t statistics to be computed
78
79         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
80         two-sample t statistics
81
82         Reviewed by Darin Adler.
83
84         * resources/sunspider-compare-results.js:
85         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
86         variable to match the definition of a two-sample t statistic. Looks
87         like this was just a simple typo.
88
89 2010-04-05  Darin Adler  <darin@apple.com>
90
91         Updated the ignore property for changes to the contents here.
92
93         * .: Modified property svn:ignore.
94
95 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
96
97         Reviewed by Adam Barth.
98
99         Add sunspider-0.9.1 to the site in preparation for announcing it.
100         https://bugs.webkit.org/show_bug.cgi?id=35206
101
102         * hosted/sunspider.css: Remove tabs, which I noticed where present while
103         copying to the WebKitSite directory.
104
105 2010-01-11  Oliver Hunt  <oliver@apple.com>
106
107         Reviewed by Darin Adler.
108
109         Sunspider parse-only tests don't run
110         https://bugs.webkit.org/show_bug.cgi?id=33489
111
112         Fix parse-only test list
113
114         * tests/parse-only/LIST:
115
116 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
117
118         Reviewed by Adele Peterson.
119
120         Make SunSpider version more prominent in the title
121         https://bugs.webkit.org/show_bug.cgi?id=32574
122
123         * hosted/sunspider.html:
124         * hosted/versions.html:
125         * resources/driver-TEMPLATE.html:
126         * resources/results-TEMPLATE.html:
127
128 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
129
130         Reviewed by Darin Adler.
131
132         Make SunSpider refuse to compare results across content versions
133         https://bugs.webkit.org/show_bug.cgi?id=32573
134         
135         The results URL now includes a version indicator, and compare mode
136         will refuse to compare between versions.
137
138         * resources/driver-TEMPLATE.html:
139         * resources/results-TEMPLATE.html:
140
141 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
142
143         Reviewed by Darin Adler.
144
145         Some Browser-hosted SunSpider files are not valid HTML5
146         https://bugs.webkit.org/show_bug.cgi?id=32536
147         
148         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
149         close tags and adding a meta charset declaration).
150
151         * hosted/sunspider.html:
152         * hosted/versions.html:
153         * resources/TEMPLATE.html:
154         * resources/driver-TEMPLATE.html:
155         * resources/results-TEMPLATE.html:
156
157 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
158
159         Reviewed by Eric Seidel.
160
161         Make sunspider-0.9.1 the default content set (both command-line and hosted)
162         https://bugs.webkit.org/show_bug.cgi?id=32537
163
164         * hosted/sunspider.html: Change default suite link.
165         * sunspider: Change default suite.
166         * sunspider-compare-results: Likewise.
167
168 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
169
170         Reviewed by Sam Weinig.
171
172         in-browser SunSpider suffers excessive penalty under power management
173         https://bugs.webkit.org/show_bug.cgi?id=32505
174         
175         I have made a few changes to address this:
176         
177         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
178         document.write to reduce triggering of progress UI.
179         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
180         stepped all the way down the whole time.
181         3) Run the test cycle an extra time for warmup before the runs that count.
182
183         * hosted/sunspider-record-result.js: Removed.
184         * make-hosted:
185         * resources/TEMPLATE.html:
186         * resources/driver-TEMPLATE.html:
187
188 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
189
190         Reviewed by Gavin Barraclough.
191
192         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
193         https://bugs.webkit.org/show_bug.cgi?id=16806
194
195         Based on a patch by Eric Seidel.
196         
197         Fix the base64 computation to actually compute correct results. The impact on runtime of
198         the test is pretty small, but noticeable for some browsers. But at least it's not
199         doing a wrong and meaningless computation any more.
200         
201         * tests/sunspider-0.9.1/string-base64.js:
202         ():
203         (base64ToString):
204
205 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
206
207         Fixing commit error...
208         
209         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
210
211         * resources/driver-TEMPLATE.html:
212
213 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
214
215         Reviewed by Sam Weinig.
216
217         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
218         https://bugs.webkit.org/show_bug.cgi?id=32490
219
220         * hosted/json2.js: Added. Incorporated from json.org
221         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
222         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
223         results.
224
225 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
226
227         Reviewed by Oliver Hunt.
228
229         Enable Web-hosted version of SunSpider to handle multiple versions
230         https://bugs.webkit.org/show_bug.cgi?id=32478
231
232         * make-hosted: Now generate the hosted version to be able to run both the
233         0.9 and the 0.9.1 test suites.
234         * hosted: Modified property svn:ignore.
235         * hosted/sunspider-driver.html: Removed.
236         * hosted/sunspider-results.html: Removed.
237         * hosted/sunspider.html:
238         * hosted/versions.html: Added.
239         * resources/TEMPLATE.html:
240         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
241         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
242
243 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
244
245         Reviewed by Oliver Hunt.
246
247         Give command-line sunspider the ability to handle multiple suites and versions
248         https://bugs.webkit.org/show_bug.cgi?id=32477
249         
250         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
251         enumerating the tests. Current suites are:
252         
253         sunspider-0.9 - riginal SunSpider content 
254         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
255         ubench - "ubench" microbenchmark suite
256         v8-v4 - version 4 of the v8 benchmark
257         parse-only - some parser tests
258         
259         Sunspider can now be run with the --suite= parameter to select a
260         suite. The default is --suite=sunspider-0.9.
261
262         * .: Modified property svn:ignore to ignore new results directories.
263         * make-hosted: Generate based on sunspider-0.9 suite.
264         * resources/TEMPLATE.html: Fix an HTML compliance problem.
265         * resources/sunspider-standalone-driver.js: Become aware of suite names.
266         * sunspider: Modified to handle multiple suites as described above.
267         * sunspider-compare-results: ditto
268         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
269         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
270         * tests/LIST-UBENCH: Removed.
271         * tests/LIST-V8: Removed.
272         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
273         * tests/3d-morph.js: ditto
274         * tests/3d-raytrace.js: ditto
275         * tests/access-binary-trees.js: ditto
276         * tests/access-fannkuch.js: ditto
277         * tests/access-nbody.js: ditto
278         * tests/access-nsieve.js: ditto
279         * tests/bitops-3bit-bits-in-byte.js: ditto
280         * tests/bitops-bits-in-byte.js: ditto
281         * tests/bitops-bitwise-and.js: ditto
282         * tests/bitops-nsieve-bits.js: ditto
283         * tests/controlflow-recursive.js: ditto
284         * tests/crypto-aes.js: ditto
285         * tests/crypto-md5.js: ditto
286         * tests/crypto-sha1.js: ditto
287         * tests/date-format-tofte.js: ditto
288         * tests/date-format-xparb.js: ditto
289         * tests/math-cordic.js: ditto
290         * tests/math-partial-sums.js: ditto
291         * tests/math-spectral-norm.js: ditto
292         * tests/regexp-dna.js: ditto
293         * tests/string-base64.js: ditto
294         * tests/string-fasta.js: ditto
295         * tests/string-tagcloud.js: ditto
296         * tests/string-unpack-code.js: ditto
297         * tests/string-validate-input.js: ditto
298         * tests/sunspider-0.9/: Added
299         * tests/sunspider-0.9/*.js: Moved from one level up
300         * tests/sunspider-0.9.1: Added.
301         * tests/sunspider-0.9.1/*.js: Moved from one level up.
302         * tests/v8-crypto.js: Moved to v8-v4/ directory.
303         * tests/v8-deltablue.js: ditto
304         * tests/v8-earley-boyer.js: ditto
305         * tests/v8-raytrace.js: ditto
306         * tests/v8-regexp.js: ditto
307         * tests/v8-richards.js: ditto
308         * tests/v8-splay.js: ditto
309         * tests/v8-v4: Added.
310
311 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
312
313         Reviewed by Darin Adler.
314
315         Platform specific null device replaced with a platform independent.
316         https://bugs.webkit.org/show_bug.cgi?id=29544
317
318         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
319
320 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
321
322         Reviewed by Maciej Stachowiak.
323
324         [Qt] Make sunspider script work on Windows platform.
325         https://bugs.webkit.org/show_bug.cgi?id=29544
326
327         * sunspider:
328         Using platform specific null device instead of /dev/null.
329
330 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
331
332         Reviewed by Oliver Hunt.
333
334         https://bugs.webkit.org/show_bug.cgi?id=26540
335         Currently the SunSpider test driver lacks an option to run a test suite that
336         will test JavaScriptCore parsing performance only.  This patch adds just such
337         a test suite and option to SunSpider as well as the jsc test shell.  I've included
338         three large javascript source files found in the wild: jquery, mootools and prototype.
339         Combined with the concatenation of all three, these form a new testsuite to measure
340         and test pure JavaScriptCore parsing performance.
341
342         * resources/sunspider-standalone-driver.js:
343           Load the js with parse-only flag if it is part of the parse-only suite.
344         * sunspider:
345           Add the parse-only suite to the list of options.
346         * sunspider-compare-results:
347           Add the parse-only suite to the list of options.
348         * tests/LIST-PARSE-ONLY: Added.
349         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
350           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
351         * tests/parse-only/jquery-1.3.2.js: Added.
352           Latest version from jquery.com.
353         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
354           Latest version from mootools.net.
355         * tests/parse-only/prototype-1.6.0.3.js: Added.
356           Latest version from prototypejs.org.
357
358 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
359
360         Update V8 benchmark to version 4.
361         
362         Reviewed by Cameron Zwarich.
363
364         * tests/LIST-V8:
365         * tests/v8-crypto.js:
366         * tests/v8-deltablue.js:
367         * tests/v8-earley-boyer.js:
368         * tests/v8-raytrace.js:
369         * tests/v8-regexp.js: Added.
370         * tests/v8-richards.js:
371         * tests/v8-splay.js: Added.
372
373 2009-05-26  Olivier DOLE  <odole@pleyo.com>
374
375         Reviewed by Maciej.
376
377         Store sunspider test suite results in a specific directory for each
378         different type of benchmark (sunspider, V8 or ubench).
379
380         * sunspider:
381         * sunspider-compare-results:
382
383 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
384
385         Build fix.
386
387         * SunSpider.make:
388
389 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
390
391         Include SunSpider tests in production builds.
392         
393         Reviewed by Adam Roben.
394
395         * SunSpider.make: Added.
396         * xcopy.excludes: Added.
397
398 2009-03-04  Oliver Hunt  <oliver@apple.com>
399
400         Added a few more tests to ubench
401
402         RS=Gavin Barraclough
403
404         Adding a couple of test to cover too few and too many arguments.  This
405         is needed to prevent regressions in all call paths from my current refactoring.
406
407         * tests/LIST-UBENCH:
408         * tests/ubench/function-correct-args.js: Added.
409         (f):
410         * tests/ubench/function-excess-args.js: Added.
411         (f):
412
413 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
414
415         Fix a bug where when V8's or UBENCH's test suite list file is missing
416         the error message is suggesting that SunSpider's list file is missing.
417
418         Reviewed by Sam Weinig.
419
420         * sunspider:
421         (loadTestsList): Fix the error message to use the right list name.
422
423 2008-10-07  David Hyatt  <hyatt@apple.com>
424
425         https://bugs.webkit.org/show_bug.cgi?id=18989
426
427         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
428         we want all browsers to be doing the same thing on the test.
429
430         Reviewed by Maciej
431
432         * tests/regexp-dna.js:
433
434 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
435
436         Rubber-stamped by Maciej Stachowiak.
437
438         The bleeding edge branch of V8 now supports passing fileanmes after
439         '-f', so the SunSpider script should be changed to do this again.
440         This allows all 3 major JavaScript engines to be used with the
441         unmodified SunSpider script.
442
443         * sunspider:
444
445 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
446
447         Reviewed by Maciej Stachowiak.
448
449         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
450         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
451         correct the punctuation in its documentation.
452
453         * sunspider:
454
455 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
456
457         Reviewed by Oliver Hunt.
458
459         Bug 20616: Incorporate V8 benchmarks in testing
460         <https://bugs.webkit.org/show_bug.cgi?id=20616>
461
462         This is a first attempt at running the V8 benchmarks within SunSpider,
463         with the --v8 option. The number of iterations is fixed for each
464         benchmark, and was chosen to make each of the benchmarks run between
465         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
466
467         * sunspider:
468         * tests/LIST-V8: Added.
469         * tests/v8-crypto.js: Added.
470         * tests/v8-deltablue.js: Added.
471         * tests/v8-earley-boyer.js: Added.
472         * tests/v8-raytrace.js: Added.
473         * tests/v8-richards.js: Added.
474
475 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
476
477         Not reviewed.
478
479         Remove a local change that was accidentally committed in r36034.
480
481         * resources/sunspider-standalone-driver.js:
482
483 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
484
485         Rubber-stamped by Oliver Hunt.
486
487         Make SunSpider work with JavaScript shells that don't take filenames
488         after a '-f' argument.
489
490         * sunspider:
491
492 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
493
494         Reviewed by Timothy.
495
496         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
497
498         * sunspider:
499         * tests/LIST-SQUIRRELFISH: Removed.
500
501 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
502
503         Rubber-stamped by Mark Rowe.
504
505         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
506
507         * sunspider:
508
509 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
510
511         Rubber stamped by Oliver.
512         
513         - add newly working tests to squirrelfish list
514
515         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
516
517 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
518
519         Rubber stamped by Oliver.
520         
521         - move ubench tests properly.
522
523         * tests/ubench: Added.
524         * tests/ubench/function-closure.js: Added.
525         * tests/ubench/function-empty.js: Added.
526         * tests/ubench/function-missing-args.js: Added.
527         * tests/ubench/function-sum.js: Added.
528         * tests/ubench/loop-empty-resolve.js: Added.
529         * tests/ubench/loop-empty.js: Added.
530         * tests/ubench/loop-sum.js: Added.
531
532 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
533
534         Rubber stamped by Oliver.
535
536         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
537
538 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
539
540         Reviewed by Oliver.
541
542         - make --squirrelfish mode use the real harness, but a reduced set of tests
543
544         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
545
546         * sunspider:
547         * tests/LIST-SQUIRRELFISH:
548         * tests/LIST-UBENCH: Added.
549         * tests/squirrelfish: Moved to tests/ubench
550
551 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
552
553         Reviewed by Maciej Stachowiak.
554         
555         Removed an unused test.
556
557         * tests/squirrelfish/loop-resolve.js: Removed.
558
559 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
560
561         Reviewed by Oliver.
562         
563         Add newly runnable tests to --squirrelfish mode.
564
565         * tests/LIST-SQUIRRELFISH:
566
567 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
568
569         Reviewed by Maciej Stachowiak.
570         
571         A little more love for --squirrelfish mode:
572         
573         Fixed a misplaced sort. Results properly sort now.
574         
575         Added a test list just for squirrelfish, and updated it to include
576         all currently passing SunSpider tests.
577         
578         Fixed the pruning regexp to match 3d-morph.js.
579
580         * sunspider:
581
582 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
583
584         Reviewed by Oliver Hunt.
585         
586         --squirrelfish mode: pared down tests for squirrelfish to chew on.
587
588         * sunspider:
589         * tests/LIST:
590         * tests/squirrelfish: Added.
591         * tests/squirrelfish/function-closure.js: Added.
592         * tests/squirrelfish/function-empty.js: Added.
593         * tests/squirrelfish/function-missing-args.js: Added.
594         * tests/squirrelfish/function-sum.js: Added.
595         * tests/squirrelfish/loop-empty-resolve.js: Added.
596         * tests/squirrelfish/loop-empty.js: Added.
597         * tests/squirrelfish/loop-resolve.js: Added.
598         * tests/squirrelfish/loop-sum.js: Added.
599
600 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
601
602         Reviewed by Geoff.
603
604         * sunspider: Pause update daemon on Darwin to make results reliable
605         again (workaround for <rdar://problem/5811127>).
606
607 2008-02-07  Eric Seidel  <eric@webkit.org>
608
609         Reviewed by Sam.
610
611         * sunspider: add an error message when ./tests/LIST is not found
612
613 2007-12-29  Alp Toker  <alp@atoker.com>
614
615         Fix typo.
616
617         * hosted/sunspider.html:
618
619 2007-12-29  Alp Toker  <alp@atoker.com>
620
621         Reviewed by Maciej.
622
623         Provide the current URL in the SunSpider results page. Useful for
624         copying benchmark results out of browsers without a location bar or
625         where the location bar can't handle long URLs.
626
627         * hosted/sunspider-results.html:
628
629 2007-12-20  Eric Seidel  <eric@webkit.org>
630
631         Reviewed by mjs.
632
633         * sunspider: increase default number of runs for more accurate numbers
634
635 2007-12-18  Mark Rowe  <mrowe@apple.com>
636
637         Fix typo.
638
639         * hosted/sunspider.html:
640
641 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
642
643         Reviewed by Eric.
644         
645         - give an extra digit of precision for small differences.
646
647         * resources/sunspider-compare-results.js:
648
649 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
650
651         Reviewed by Eric.
652         
653         - work around IE layout bugs
654
655         * hosted/sunspider.css:
656
657 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
658
659         Reviewed by Mark.
660         
661         - fix compare feature
662
663         * hosted/sunspider-results.html:
664
665 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
666
667         Reviewed by Mark.
668         
669         - Change the format of the output URL to be shorter, to meet IE constraints
670
671         * hosted/sunspider-driver.html:
672         * hosted/sunspider-results.html:
673
674 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
675
676         Reviewed by Oliver.
677         
678         - Add some explanatory test and a dumb logo to SunSpider web hosted version
679
680         * hosted/sunspider-driver.html:
681         * hosted/sunspider-results.html:
682         * hosted/sunspider.css:
683         * hosted/sunspider.html:
684
685 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
686
687         Reviewed by Oliver.
688         
689         - Make the web-hosted version of SunSpider look a little bit less drab.
690
691         * hosted/sunspider-driver.html:
692         * hosted/sunspider-results.html:
693         * hosted/sunspider.css: Added.
694         * hosted/sunspider.html:
695         * resources/TEMPLATE.html:
696
697 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
698
699         Reviewed by Sam.
700         
701         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
702         
703         A sample of what new-style results would look like:
704         
705         TEST                   COMPARISON            FROM                 TO             DETAILS
706         ...
707         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
708         ...
709             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
710         
711         * resources/sunspider-compare-results.js:
712
713 2007-11-14  Eric Seidel  <eric@webkit.org>
714
715         Reviewed by Sam.
716
717         * sunspider: add --shark-cache for L2 Cache Miss profiling
718
719 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
720
721         Reviewed by Darin Adler.
722
723         http://bugs.webkit.org/show_bug.cgi?id=15925
724         SunSpider should check for Shark being installed
725
726         * sunspider: Explicitly test that shark command line tool is installed.
727
728 2007-11-04  Darin Adler  <darin@apple.com>
729
730         * hosted/sunspider.html: Tweaked formatting a bit.
731
732 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
733
734         Reviewed by Darin Adler.
735
736         - Add compare mode and a description of the benchmark to the hosted version.
737
738         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
739         * hosted/sunspider.html: Added intro text.
740         * make-hosted: Copy new file.
741         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
742         a function.
743         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
744         expects.
745         * sunspider-compare-results: Use sunspider-standalone-compare.js.
746
747 2007-11-02  Darin Adler  <darin@apple.com>
748
749         * sunspider: Changed "--base" to "--set-baseline".
750
751 2007-11-02  Darin Adler  <darin@apple.com>
752
753         Reviewed by Maciej.
754
755         * sunspider: Added a "--base" option that records the current test as a baseline.
756         Moved the code to find the newest .mshark file into a function (since I needed to use
757         it in the other script and I prefer to copy/paste a function).
758
759         * sunspider-compare-results: Added logic to compare the baseline with the last
760         run when you don't pass any parameters.
761
762 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
763
764         Reviewed by Eric.
765         
766         - Added list of ECMAScript3 features that SunSpider does not cover currently.
767
768         * UNCOVERED: Added.
769
770 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
771
772         Reviewed by Oliver.
773         
774         - Give better error messages when sunspider-compare-results fails
775
776         * sunspider-compare-results:
777
778 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
779
780         Reviewed by Darin Adler.
781         
782         - fix Date tests to work in command-line SpiderMonkey
783
784         * tests/date-format-tofte.js:
785         * tests/date-format-xparb.js:
786
787 2007-10-29  Darin Adler  <darin@apple.com>
788
789         Reviewed by Maciej.
790
791         * sunspider: Added --shark20 option, to run Shark at its highest sample
792         resolution instead of its default.
793
794 2007-10-28  Adam Roben  <aroben@apple.com>
795
796         Fix SunSpider on Windows
797
798         Reviewed by Eric.
799
800         * sunspider: Don't use colons in filenames.
801
802 2007-10-28  Darin Adler  <darin@apple.com>
803
804         Reviewed by Adam.
805
806         * resources/sunspider-analyze-results.js: Tweak the output format a little.
807         Change so that when there's only one run we don't write out confidence
808         intervals at all rather than writing "NaN" over and over again.
809
810         * sunspider: Use the time and date as a suffix on the results file, that way
811         all the results are left behind in the tmp directory. This will make it easy
812         for us to add features that compare past results in the future. Also rename
813         the .mshark files using the same scheme.
814
815         * sunspider-compare-results: Relax the parsing rules so we can parse the
816         results file exactly as it's emitted from sunspider.
817
818 2007-10-25  Adam Roben  <aroben@apple.com>
819
820         Strip carriage returns from results
821
822         Carriage returns were screwing up sunspider-compare-results on
823         Windows.
824
825         Rubberstamped by Oliver.
826
827         * sunspider:
828
829 2007-10-24  Darin Adler  <darin@apple.com>
830
831         Reviewed by Eric.
832
833         * sunspider: Open the profile in shark after completing a run.
834         * sunspider-compare-results: Tiny perl tweaks.
835
836 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
837
838         Reviewed by Darin Adler.
839
840         - Don't hardcode my path to testkjs
841         
842         * sunspider-compare-results:
843
844 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
845
846         Reviewed by Darin Adler.
847         
848         - Add a compare script to compare results, and improve formatting of standard results a bit.
849
850         * resources/sunspider-analyze-results.js:
851         * resources/sunspider-compare-results.js: Added.
852         * sunspider-compare-results: Added.
853
854 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
855
856         Reviewed by Darin Adler.
857
858         * resources/sunspider-analyze-results.js: Do the error range
859         properly, using the t-distribution instead of 1.96 (which was
860         based on the normal distribution).
861         
862         * sunspider: Print results in a way that will be friendlier to the
863         soon-to-come compare mode.
864
865 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
866
867         Reviewed by Review.
868         
869         - add more tests (probably the final set for now)
870         
871         Already balanced. Added date, regexp, control flow, and a few
872         more string and object/array access tests.
873         
874         * tests/LIST:
875         * tests/access-fannkuch.js: Added.
876         * tests/access-nbody.js: Added.
877         * tests/controlflow-recursive.js: Added.
878         * tests/date-format-tofte.js: Added.
879         * tests/date-format-xparb.js: Added.
880         * tests/regexp-dna.js: Added.
881         * tests/string-validate-input.js: Added.
882
883 2007-10-21  Eric Seidel  <eric@webkit.org>
884
885         Reviewed by mjs.
886
887         * sunspider: make --shark only sample from testkjs
888
889 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
890
891         Rubber stamped by Eric.
892         
893         - added crypto tests (already balanced)
894
895         * tests/LIST:
896         * tests/crypto-aes.js: Added.
897         * tests/crypto-md5.js: Added.
898         * tests/crypto-sha1.js: Added.
899
900 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
901
902         Reviewed by Adam.
903         
904         - rebalance new tests
905         
906         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
907         
908         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
909
910         This balances them with the existing tests.
911
912         * tests/access-binary-trees.js:
913         * tests/access-nsieve.js:
914         * tests/bitops-nsieve-bits.js:
915         * tests/math-partial-sums.js:
916         * tests/math-spectral-norm.js:
917         * tests/string-fasta.js:
918
919 2007-10-20  Eric Seidel  <eric@webkit.org>
920
921         Reviewed by darin.
922         
923         Add improved argument handling to sunspider, including
924         --runs=<number>, --tests=<pattern, --shark, and --help
925         Also re-factor code into subroutines
926
927         * sunspider: improved argument handling
928
929 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
930
931         Reviewed by Mark.
932
933         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
934         
935         * TODO:
936         * tests/LIST:
937         * tests/access-binary-trees.js: Added.
938         * tests/access-nsieve.js: Added.
939         * tests/bitops-nsieve-bits.js: Added.
940         * tests/math-partial-sums.js: Added.
941         * tests/math-spectral-norm.js: Added.
942         * tests/string-fasta.js: Added.
943
944 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
945
946         Reviewed by Darin Adler.
947
948         * TODO: Updated to note areas that are now well-covered.
949         
950         - Change the tests and the driver to avoid leaving large object
951         graphs hanging around, since that throws off the subsequent tests.
952         
953         * resources/sunspider-standalone-driver.js:
954         * tests/3d-cube.js:
955         * tests/3d-morph.js:
956         * tests/3d-raytrace.js:
957         * tests/bitops-bitwise-and.js:
958         * tests/string-base64.js:
959         * tests/string-tagcloud.js:
960
961 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
962
963         Reviewed by Nikolas.
964         
965         - Rebalance test complexity.
966         
967         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
968         
969         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
970
971         However, for some tests, IE7 was a huge outlier compared to other
972         major browsers. For those tests, I used Firefox 2.0.0.8 on the
973         same OS and hardware as a normalization basis.
974         
975         * tests/3d-morph.js:
976         * tests/3d-raytrace.js:
977         * tests/bitops-3bit-bits-in-byte.js:
978         * tests/bitops-bits-in-byte.js:
979         * tests/bitops-bitwise-and.js:
980         * tests/math-cordic.js:
981         * tests/string-base64.js:
982         * tests/string-tagcloud.js:
983         * tests/string-unpack-code.js:
984
985 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
986
987         Reviewed by Oliver.
988         
989         - Fix the browser-hosted driver to be compatible with Safari 2
990
991         * hosted/sunspider-driver.html:
992
993 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
994
995         Rubber stamped by Hyatt.
996         
997         - Add BSD license notice to cordic test, the original author said to do so.
998
999         * tests/math-cordic.js:
1000
1001 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1002
1003         Reviewed by Eric.
1004         
1005         - Support Windows and Windows browsers.
1006
1007         * hosted/sunspider-results.html: Display results correctly in IE.
1008         * sunspider: tweaks to work on Windows, and with spaces in the path.
1009
1010 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1011
1012         Rubber stamped by Oliver.
1013
1014         * tests/string-tagcloud.js: Remove stray print()
1015
1016 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1017
1018         Reviewed by Oliver.
1019         
1020         Add some string test cases.
1021
1022         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1023         * tests/string-tagcloud.js: Added. Parses a JSON string and
1024         generates tagcloud HTML markup based on the data.        
1025         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1026         four of the most popular JavaScript libraries.
1027         * tests/LIST: Added the new tests.
1028
1029 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1030
1031         Reviewed by Anders.
1032
1033         * TODO: Updated with info on how to do the stats properly.
1034
1035         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1036         avoid overly slowing down the benchmark.
1037
1038         Added some 3D-related test cases.
1039         
1040         * tests/3d-cube.js: Added.
1041         * tests/3d-morph.js: Added.
1042         * tests/3d-raytrace.js: Added.
1043         * tests/LIST:
1044
1045 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1046
1047         Reviewed by Darin Adler.
1048
1049         - New JavaScript benchmark
1050         http://bugs.webkit.org/show_bug.cgi?id=15515
1051         
1052         A start on a new JavaScript benchmark, with standalone and
1053         browser-hosted drivers.
1054         
1055         The test content is still incomplete and the drivers could use
1056         some more features but there is enough here to make a good start.
1057         
1058         The drivers compute a 95% confidence interval on the mean for the
1059         whole test, each category, and each individual test to make it
1060         easier to tell whether differences are statistically
1061         significant. The confidence interval can be narrowed by running
1062         the test more times.
1063         
1064         * TODO: Added. Remaining things that need to be done.
1065         * sunspider: Added. Perl script that acts as the standalone test driver.
1066         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1067         of standalone test driver.
1068         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1069         analysis code.
1070         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1071         * make-hosted: Added. Script to generate browser-hosted tests.
1072         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1073         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1074         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1075         tests.
1076         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1077         tests.
1078         * tests/LIST: Added. List of tests to use.
1079         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1080         * tests/bitops-bits-in-byte.js: Added. Ditto.
1081         * tests/bitops-bitwise-and.js: Added. Ditto.
1082         * tests/math-cordic.js: Added. Ditto.
1083