2010-09-08 Adam Barth <abarth@webkit.org>
[WebKit-https.git] / SunSpider / ChangeLog
1 2010-08-27  Michael Saboff  <msaboff@apple.com>
2
3         Reviewed by Stephanie Lewis.
4
5         Added version 5 and version 6 of V8 tests with modifications similar
6         to the version 4 tests so they'll work with the SunSpider test harness.
7         Note that these files are external source and therefore don't conform 
8         to WebKit coding standards.
9         https://bugs.webkit.org/show_bug.cgi?id=44638
10
11         * tests/v8-v5: Added.
12         * tests/v8-v5/LIST: Added.
13         * tests/v8-v5/v8-crypto.js: Added.
14         * tests/v8-v5/v8-deltablue.js: Added.
15         * tests/v8-v5/v8-earley-boyer.js: Added.
16         * tests/v8-v5/v8-raytrace.js: Added.
17         * tests/v8-v5/v8-regexp.js: Added.
18         * tests/v8-v5/v8-richards.js: Added.
19         * tests/v8-v5/v8-splay.js: Added.
20         * tests/v8-v6: Added.
21         * tests/v8-v6/LIST: Added.
22         * tests/v8-v6/v8-crypto.js: Added.
23         * tests/v8-v6/v8-deltablue.js: Added.
24         * tests/v8-v6/v8-earley-boyer.js: Added.
25         * tests/v8-v6/v8-raytrace.js: Added.
26         * tests/v8-v6/v8-regexp.js: Added.
27         * tests/v8-v6/v8-richards.js: Added.
28         * tests/v8-v6/v8-splay.js: Added.
29
30 2010-07-15  Stephanie Lewis  <slewis@apple.com>
31
32         Reviewed by Geoff Garen.
33
34         http://bugs.webkit.org/show_bug.cgi?id=42406
35         Output location of results file.
36
37         * sunspider:
38
39 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
40
41         Reviewed by Mark Rowe.
42
43         Browser-hosted version of SunSpider runs tests twice as many times as advertised
44         https://bugs.webkit.org/show_bug.cgi?id=39201
45
46         * make-hosted: Make sure to reset test list after every test suite.
47
48 2010-05-07  Adam Roben  <aroben@apple.com>
49
50         Fix typo in sunspider-compare-results.js that caused incorrect
51         two-sample t statistics to be computed
52
53         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
54         two-sample t statistics
55
56         Reviewed by Darin Adler.
57
58         * resources/sunspider-compare-results.js:
59         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
60         variable to match the definition of a two-sample t statistic. Looks
61         like this was just a simple typo.
62
63 2010-04-05  Darin Adler  <darin@apple.com>
64
65         Updated the ignore property for changes to the contents here.
66
67         * .: Modified property svn:ignore.
68
69 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
70
71         Reviewed by Adam Barth.
72
73         Add sunspider-0.9.1 to the site in preparation for announcing it.
74         https://bugs.webkit.org/show_bug.cgi?id=35206
75
76         * hosted/sunspider.css: Remove tabs, which I noticed where present while
77         copying to the WebKitSite directory.
78
79 2010-01-11  Oliver Hunt  <oliver@apple.com>
80
81         Reviewed by Darin Adler.
82
83         Sunspider parse-only tests don't run
84         https://bugs.webkit.org/show_bug.cgi?id=33489
85
86         Fix parse-only test list
87
88         * tests/parse-only/LIST:
89
90 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
91
92         Reviewed by Adele Peterson.
93
94         Make SunSpider version more prominent in the title
95         https://bugs.webkit.org/show_bug.cgi?id=32574
96
97         * hosted/sunspider.html:
98         * hosted/versions.html:
99         * resources/driver-TEMPLATE.html:
100         * resources/results-TEMPLATE.html:
101
102 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
103
104         Reviewed by Darin Adler.
105
106         Make SunSpider refuse to compare results across content versions
107         https://bugs.webkit.org/show_bug.cgi?id=32573
108         
109         The results URL now includes a version indicator, and compare mode
110         will refuse to compare between versions.
111
112         * resources/driver-TEMPLATE.html:
113         * resources/results-TEMPLATE.html:
114
115 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
116
117         Reviewed by Darin Adler.
118
119         Some Browser-hosted SunSpider files are not valid HTML5
120         https://bugs.webkit.org/show_bug.cgi?id=32536
121         
122         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
123         close tags and adding a meta charset declaration).
124
125         * hosted/sunspider.html:
126         * hosted/versions.html:
127         * resources/TEMPLATE.html:
128         * resources/driver-TEMPLATE.html:
129         * resources/results-TEMPLATE.html:
130
131 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
132
133         Reviewed by Eric Seidel.
134
135         Make sunspider-0.9.1 the default content set (both command-line and hosted)
136         https://bugs.webkit.org/show_bug.cgi?id=32537
137
138         * hosted/sunspider.html: Change default suite link.
139         * sunspider: Change default suite.
140         * sunspider-compare-results: Likewise.
141
142 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
143
144         Reviewed by Sam Weinig.
145
146         in-browser SunSpider suffers excessive penalty under power management
147         https://bugs.webkit.org/show_bug.cgi?id=32505
148         
149         I have made a few changes to address this:
150         
151         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
152         document.write to reduce triggering of progress UI.
153         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
154         stepped all the way down the whole time.
155         3) Run the test cycle an extra time for warmup before the runs that count.
156
157         * hosted/sunspider-record-result.js: Removed.
158         * make-hosted:
159         * resources/TEMPLATE.html:
160         * resources/driver-TEMPLATE.html:
161
162 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
163
164         Reviewed by Gavin Barraclough.
165
166         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
167         https://bugs.webkit.org/show_bug.cgi?id=16806
168
169         Based on a patch by Eric Seidel.
170         
171         Fix the base64 computation to actually compute correct results. The impact on runtime of
172         the test is pretty small, but noticeable for some browsers. But at least it's not
173         doing a wrong and meaningless computation any more.
174         
175         * tests/sunspider-0.9.1/string-base64.js:
176         ():
177         (base64ToString):
178
179 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
180
181         Fixing commit error...
182         
183         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
184
185         * resources/driver-TEMPLATE.html:
186
187 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
188
189         Reviewed by Sam Weinig.
190
191         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
192         https://bugs.webkit.org/show_bug.cgi?id=32490
193
194         * hosted/json2.js: Added. Incorporated from json.org
195         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
196         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
197         results.
198
199 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
200
201         Reviewed by Oliver Hunt.
202
203         Enable Web-hosted version of SunSpider to handle multiple versions
204         https://bugs.webkit.org/show_bug.cgi?id=32478
205
206         * make-hosted: Now generate the hosted version to be able to run both the
207         0.9 and the 0.9.1 test suites.
208         * hosted: Modified property svn:ignore.
209         * hosted/sunspider-driver.html: Removed.
210         * hosted/sunspider-results.html: Removed.
211         * hosted/sunspider.html:
212         * hosted/versions.html: Added.
213         * resources/TEMPLATE.html:
214         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
215         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
216
217 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
218
219         Reviewed by Oliver Hunt.
220
221         Give command-line sunspider the ability to handle multiple suites and versions
222         https://bugs.webkit.org/show_bug.cgi?id=32477
223         
224         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
225         enumerating the tests. Current suites are:
226         
227         sunspider-0.9 - riginal SunSpider content 
228         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
229         ubench - "ubench" microbenchmark suite
230         v8-v4 - version 4 of the v8 benchmark
231         parse-only - some parser tests
232         
233         Sunspider can now be run with the --suite= parameter to select a
234         suite. The default is --suite=sunspider-0.9.
235
236         * .: Modified property svn:ignore to ignore new results directories.
237         * make-hosted: Generate based on sunspider-0.9 suite.
238         * resources/TEMPLATE.html: Fix an HTML compliance problem.
239         * resources/sunspider-standalone-driver.js: Become aware of suite names.
240         * sunspider: Modified to handle multiple suites as described above.
241         * sunspider-compare-results: ditto
242         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
243         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
244         * tests/LIST-UBENCH: Removed.
245         * tests/LIST-V8: Removed.
246         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
247         * tests/3d-morph.js: ditto
248         * tests/3d-raytrace.js: ditto
249         * tests/access-binary-trees.js: ditto
250         * tests/access-fannkuch.js: ditto
251         * tests/access-nbody.js: ditto
252         * tests/access-nsieve.js: ditto
253         * tests/bitops-3bit-bits-in-byte.js: ditto
254         * tests/bitops-bits-in-byte.js: ditto
255         * tests/bitops-bitwise-and.js: ditto
256         * tests/bitops-nsieve-bits.js: ditto
257         * tests/controlflow-recursive.js: ditto
258         * tests/crypto-aes.js: ditto
259         * tests/crypto-md5.js: ditto
260         * tests/crypto-sha1.js: ditto
261         * tests/date-format-tofte.js: ditto
262         * tests/date-format-xparb.js: ditto
263         * tests/math-cordic.js: ditto
264         * tests/math-partial-sums.js: ditto
265         * tests/math-spectral-norm.js: ditto
266         * tests/regexp-dna.js: ditto
267         * tests/string-base64.js: ditto
268         * tests/string-fasta.js: ditto
269         * tests/string-tagcloud.js: ditto
270         * tests/string-unpack-code.js: ditto
271         * tests/string-validate-input.js: ditto
272         * tests/sunspider-0.9/: Added
273         * tests/sunspider-0.9/*.js: Moved from one level up
274         * tests/sunspider-0.9.1: Added.
275         * tests/sunspider-0.9.1/*.js: Moved from one level up.
276         * tests/v8-crypto.js: Moved to v8-v4/ directory.
277         * tests/v8-deltablue.js: ditto
278         * tests/v8-earley-boyer.js: ditto
279         * tests/v8-raytrace.js: ditto
280         * tests/v8-regexp.js: ditto
281         * tests/v8-richards.js: ditto
282         * tests/v8-splay.js: ditto
283         * tests/v8-v4: Added.
284
285 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
286
287         Reviewed by Darin Adler.
288
289         Platform specific null device replaced with a platform independent.
290         https://bugs.webkit.org/show_bug.cgi?id=29544
291
292         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
293
294 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
295
296         Reviewed by Maciej Stachowiak.
297
298         [Qt] Make sunspider script work on Windows platform.
299         https://bugs.webkit.org/show_bug.cgi?id=29544
300
301         * sunspider:
302         Using platform specific null device instead of /dev/null.
303
304 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
305
306         Reviewed by Oliver Hunt.
307
308         https://bugs.webkit.org/show_bug.cgi?id=26540
309         Currently the SunSpider test driver lacks an option to run a test suite that
310         will test JavaScriptCore parsing performance only.  This patch adds just such
311         a test suite and option to SunSpider as well as the jsc test shell.  I've included
312         three large javascript source files found in the wild: jquery, mootools and prototype.
313         Combined with the concatenation of all three, these form a new testsuite to measure
314         and test pure JavaScriptCore parsing performance.
315
316         * resources/sunspider-standalone-driver.js:
317           Load the js with parse-only flag if it is part of the parse-only suite.
318         * sunspider:
319           Add the parse-only suite to the list of options.
320         * sunspider-compare-results:
321           Add the parse-only suite to the list of options.
322         * tests/LIST-PARSE-ONLY: Added.
323         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
324           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
325         * tests/parse-only/jquery-1.3.2.js: Added.
326           Latest version from jquery.com.
327         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
328           Latest version from mootools.net.
329         * tests/parse-only/prototype-1.6.0.3.js: Added.
330           Latest version from prototypejs.org.
331
332 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
333
334         Update V8 benchmark to version 4.
335         
336         Reviewed by Cameron Zwarich.
337
338         * tests/LIST-V8:
339         * tests/v8-crypto.js:
340         * tests/v8-deltablue.js:
341         * tests/v8-earley-boyer.js:
342         * tests/v8-raytrace.js:
343         * tests/v8-regexp.js: Added.
344         * tests/v8-richards.js:
345         * tests/v8-splay.js: Added.
346
347 2009-05-26  Olivier DOLE  <odole@pleyo.com>
348
349         Reviewed by Maciej.
350
351         Store sunspider test suite results in a specific directory for each
352         different type of benchmark (sunspider, V8 or ubench).
353
354         * sunspider:
355         * sunspider-compare-results:
356
357 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
358
359         Build fix.
360
361         * SunSpider.make:
362
363 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
364
365         Include SunSpider tests in production builds.
366         
367         Reviewed by Adam Roben.
368
369         * SunSpider.make: Added.
370         * xcopy.excludes: Added.
371
372 2009-03-04  Oliver Hunt  <oliver@apple.com>
373
374         Added a few more tests to ubench
375
376         RS=Gavin Barraclough
377
378         Adding a couple of test to cover too few and too many arguments.  This
379         is needed to prevent regressions in all call paths from my current refactoring.
380
381         * tests/LIST-UBENCH:
382         * tests/ubench/function-correct-args.js: Added.
383         (f):
384         * tests/ubench/function-excess-args.js: Added.
385         (f):
386
387 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
388
389         Fix a bug where when V8's or UBENCH's test suite list file is missing
390         the error message is suggesting that SunSpider's list file is missing.
391
392         Reviewed by Sam Weinig.
393
394         * sunspider:
395         (loadTestsList): Fix the error message to use the right list name.
396
397 2008-10-07  David Hyatt  <hyatt@apple.com>
398
399         https://bugs.webkit.org/show_bug.cgi?id=18989
400
401         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
402         we want all browsers to be doing the same thing on the test.
403
404         Reviewed by Maciej
405
406         * tests/regexp-dna.js:
407
408 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
409
410         Rubber-stamped by Maciej Stachowiak.
411
412         The bleeding edge branch of V8 now supports passing fileanmes after
413         '-f', so the SunSpider script should be changed to do this again.
414         This allows all 3 major JavaScript engines to be used with the
415         unmodified SunSpider script.
416
417         * sunspider:
418
419 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
420
421         Reviewed by Maciej Stachowiak.
422
423         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
424         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
425         correct the punctuation in its documentation.
426
427         * sunspider:
428
429 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
430
431         Reviewed by Oliver Hunt.
432
433         Bug 20616: Incorporate V8 benchmarks in testing
434         <https://bugs.webkit.org/show_bug.cgi?id=20616>
435
436         This is a first attempt at running the V8 benchmarks within SunSpider,
437         with the --v8 option. The number of iterations is fixed for each
438         benchmark, and was chosen to make each of the benchmarks run between
439         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
440
441         * sunspider:
442         * tests/LIST-V8: Added.
443         * tests/v8-crypto.js: Added.
444         * tests/v8-deltablue.js: Added.
445         * tests/v8-earley-boyer.js: Added.
446         * tests/v8-raytrace.js: Added.
447         * tests/v8-richards.js: Added.
448
449 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
450
451         Not reviewed.
452
453         Remove a local change that was accidentally committed in r36034.
454
455         * resources/sunspider-standalone-driver.js:
456
457 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
458
459         Rubber-stamped by Oliver Hunt.
460
461         Make SunSpider work with JavaScript shells that don't take filenames
462         after a '-f' argument.
463
464         * sunspider:
465
466 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
467
468         Reviewed by Timothy.
469
470         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
471
472         * sunspider:
473         * tests/LIST-SQUIRRELFISH: Removed.
474
475 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
476
477         Rubber-stamped by Mark Rowe.
478
479         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
480
481         * sunspider:
482
483 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
484
485         Rubber stamped by Oliver.
486         
487         - add newly working tests to squirrelfish list
488
489         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
490
491 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
492
493         Rubber stamped by Oliver.
494         
495         - move ubench tests properly.
496
497         * tests/ubench: Added.
498         * tests/ubench/function-closure.js: Added.
499         * tests/ubench/function-empty.js: Added.
500         * tests/ubench/function-missing-args.js: Added.
501         * tests/ubench/function-sum.js: Added.
502         * tests/ubench/loop-empty-resolve.js: Added.
503         * tests/ubench/loop-empty.js: Added.
504         * tests/ubench/loop-sum.js: Added.
505
506 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
507
508         Rubber stamped by Oliver.
509
510         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
511
512 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
513
514         Reviewed by Oliver.
515
516         - make --squirrelfish mode use the real harness, but a reduced set of tests
517
518         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
519
520         * sunspider:
521         * tests/LIST-SQUIRRELFISH:
522         * tests/LIST-UBENCH: Added.
523         * tests/squirrelfish: Moved to tests/ubench
524
525 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
526
527         Reviewed by Maciej Stachowiak.
528         
529         Removed an unused test.
530
531         * tests/squirrelfish/loop-resolve.js: Removed.
532
533 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
534
535         Reviewed by Oliver.
536         
537         Add newly runnable tests to --squirrelfish mode.
538
539         * tests/LIST-SQUIRRELFISH:
540
541 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
542
543         Reviewed by Maciej Stachowiak.
544         
545         A little more love for --squirrelfish mode:
546         
547         Fixed a misplaced sort. Results properly sort now.
548         
549         Added a test list just for squirrelfish, and updated it to include
550         all currently passing SunSpider tests.
551         
552         Fixed the pruning regexp to match 3d-morph.js.
553
554         * sunspider:
555
556 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
557
558         Reviewed by Oliver Hunt.
559         
560         --squirrelfish mode: pared down tests for squirrelfish to chew on.
561
562         * sunspider:
563         * tests/LIST:
564         * tests/squirrelfish: Added.
565         * tests/squirrelfish/function-closure.js: Added.
566         * tests/squirrelfish/function-empty.js: Added.
567         * tests/squirrelfish/function-missing-args.js: Added.
568         * tests/squirrelfish/function-sum.js: Added.
569         * tests/squirrelfish/loop-empty-resolve.js: Added.
570         * tests/squirrelfish/loop-empty.js: Added.
571         * tests/squirrelfish/loop-resolve.js: Added.
572         * tests/squirrelfish/loop-sum.js: Added.
573
574 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
575
576         Reviewed by Geoff.
577
578         * sunspider: Pause update daemon on Darwin to make results reliable
579         again (workaround for <rdar://problem/5811127>).
580
581 2008-02-07  Eric Seidel  <eric@webkit.org>
582
583         Reviewed by Sam.
584
585         * sunspider: add an error message when ./tests/LIST is not found
586
587 2007-12-29  Alp Toker  <alp@atoker.com>
588
589         Fix typo.
590
591         * hosted/sunspider.html:
592
593 2007-12-29  Alp Toker  <alp@atoker.com>
594
595         Reviewed by Maciej.
596
597         Provide the current URL in the SunSpider results page. Useful for
598         copying benchmark results out of browsers without a location bar or
599         where the location bar can't handle long URLs.
600
601         * hosted/sunspider-results.html:
602
603 2007-12-20  Eric Seidel  <eric@webkit.org>
604
605         Reviewed by mjs.
606
607         * sunspider: increase default number of runs for more accurate numbers
608
609 2007-12-18  Mark Rowe  <mrowe@apple.com>
610
611         Fix typo.
612
613         * hosted/sunspider.html:
614
615 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
616
617         Reviewed by Eric.
618         
619         - give an extra digit of precision for small differences.
620
621         * resources/sunspider-compare-results.js:
622
623 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
624
625         Reviewed by Eric.
626         
627         - work around IE layout bugs
628
629         * hosted/sunspider.css:
630
631 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
632
633         Reviewed by Mark.
634         
635         - fix compare feature
636
637         * hosted/sunspider-results.html:
638
639 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
640
641         Reviewed by Mark.
642         
643         - Change the format of the output URL to be shorter, to meet IE constraints
644
645         * hosted/sunspider-driver.html:
646         * hosted/sunspider-results.html:
647
648 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
649
650         Reviewed by Oliver.
651         
652         - Add some explanatory test and a dumb logo to SunSpider web hosted version
653
654         * hosted/sunspider-driver.html:
655         * hosted/sunspider-results.html:
656         * hosted/sunspider.css:
657         * hosted/sunspider.html:
658
659 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
660
661         Reviewed by Oliver.
662         
663         - Make the web-hosted version of SunSpider look a little bit less drab.
664
665         * hosted/sunspider-driver.html:
666         * hosted/sunspider-results.html:
667         * hosted/sunspider.css: Added.
668         * hosted/sunspider.html:
669         * resources/TEMPLATE.html:
670
671 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
672
673         Reviewed by Sam.
674         
675         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
676         
677         A sample of what new-style results would look like:
678         
679         TEST                   COMPARISON            FROM                 TO             DETAILS
680         ...
681         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
682         ...
683             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
684         
685         * resources/sunspider-compare-results.js:
686
687 2007-11-14  Eric Seidel  <eric@webkit.org>
688
689         Reviewed by Sam.
690
691         * sunspider: add --shark-cache for L2 Cache Miss profiling
692
693 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
694
695         Reviewed by Darin Adler.
696
697         http://bugs.webkit.org/show_bug.cgi?id=15925
698         SunSpider should check for Shark being installed
699
700         * sunspider: Explicitly test that shark command line tool is installed.
701
702 2007-11-04  Darin Adler  <darin@apple.com>
703
704         * hosted/sunspider.html: Tweaked formatting a bit.
705
706 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
707
708         Reviewed by Darin Adler.
709
710         - Add compare mode and a description of the benchmark to the hosted version.
711
712         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
713         * hosted/sunspider.html: Added intro text.
714         * make-hosted: Copy new file.
715         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
716         a function.
717         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
718         expects.
719         * sunspider-compare-results: Use sunspider-standalone-compare.js.
720
721 2007-11-02  Darin Adler  <darin@apple.com>
722
723         * sunspider: Changed "--base" to "--set-baseline".
724
725 2007-11-02  Darin Adler  <darin@apple.com>
726
727         Reviewed by Maciej.
728
729         * sunspider: Added a "--base" option that records the current test as a baseline.
730         Moved the code to find the newest .mshark file into a function (since I needed to use
731         it in the other script and I prefer to copy/paste a function).
732
733         * sunspider-compare-results: Added logic to compare the baseline with the last
734         run when you don't pass any parameters.
735
736 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
737
738         Reviewed by Eric.
739         
740         - Added list of ECMAScript3 features that SunSpider does not cover currently.
741
742         * UNCOVERED: Added.
743
744 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
745
746         Reviewed by Oliver.
747         
748         - Give better error messages when sunspider-compare-results fails
749
750         * sunspider-compare-results:
751
752 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
753
754         Reviewed by Darin Adler.
755         
756         - fix Date tests to work in command-line SpiderMonkey
757
758         * tests/date-format-tofte.js:
759         * tests/date-format-xparb.js:
760
761 2007-10-29  Darin Adler  <darin@apple.com>
762
763         Reviewed by Maciej.
764
765         * sunspider: Added --shark20 option, to run Shark at its highest sample
766         resolution instead of its default.
767
768 2007-10-28  Adam Roben  <aroben@apple.com>
769
770         Fix SunSpider on Windows
771
772         Reviewed by Eric.
773
774         * sunspider: Don't use colons in filenames.
775
776 2007-10-28  Darin Adler  <darin@apple.com>
777
778         Reviewed by Adam.
779
780         * resources/sunspider-analyze-results.js: Tweak the output format a little.
781         Change so that when there's only one run we don't write out confidence
782         intervals at all rather than writing "NaN" over and over again.
783
784         * sunspider: Use the time and date as a suffix on the results file, that way
785         all the results are left behind in the tmp directory. This will make it easy
786         for us to add features that compare past results in the future. Also rename
787         the .mshark files using the same scheme.
788
789         * sunspider-compare-results: Relax the parsing rules so we can parse the
790         results file exactly as it's emitted from sunspider.
791
792 2007-10-25  Adam Roben  <aroben@apple.com>
793
794         Strip carriage returns from results
795
796         Carriage returns were screwing up sunspider-compare-results on
797         Windows.
798
799         Rubberstamped by Oliver.
800
801         * sunspider:
802
803 2007-10-24  Darin Adler  <darin@apple.com>
804
805         Reviewed by Eric.
806
807         * sunspider: Open the profile in shark after completing a run.
808         * sunspider-compare-results: Tiny perl tweaks.
809
810 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
811
812         Reviewed by Darin Adler.
813
814         - Don't hardcode my path to testkjs
815         
816         * sunspider-compare-results:
817
818 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
819
820         Reviewed by Darin Adler.
821         
822         - Add a compare script to compare results, and improve formatting of standard results a bit.
823
824         * resources/sunspider-analyze-results.js:
825         * resources/sunspider-compare-results.js: Added.
826         * sunspider-compare-results: Added.
827
828 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
829
830         Reviewed by Darin Adler.
831
832         * resources/sunspider-analyze-results.js: Do the error range
833         properly, using the t-distribution instead of 1.96 (which was
834         based on the normal distribution).
835         
836         * sunspider: Print results in a way that will be friendlier to the
837         soon-to-come compare mode.
838
839 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
840
841         Reviewed by Review.
842         
843         - add more tests (probably the final set for now)
844         
845         Already balanced. Added date, regexp, control flow, and a few
846         more string and object/array access tests.
847         
848         * tests/LIST:
849         * tests/access-fannkuch.js: Added.
850         * tests/access-nbody.js: Added.
851         * tests/controlflow-recursive.js: Added.
852         * tests/date-format-tofte.js: Added.
853         * tests/date-format-xparb.js: Added.
854         * tests/regexp-dna.js: Added.
855         * tests/string-validate-input.js: Added.
856
857 2007-10-21  Eric Seidel  <eric@webkit.org>
858
859         Reviewed by mjs.
860
861         * sunspider: make --shark only sample from testkjs
862
863 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
864
865         Rubber stamped by Eric.
866         
867         - added crypto tests (already balanced)
868
869         * tests/LIST:
870         * tests/crypto-aes.js: Added.
871         * tests/crypto-md5.js: Added.
872         * tests/crypto-sha1.js: Added.
873
874 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
875
876         Reviewed by Adam.
877         
878         - rebalance new tests
879         
880         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
881         
882         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
883
884         This balances them with the existing tests.
885
886         * tests/access-binary-trees.js:
887         * tests/access-nsieve.js:
888         * tests/bitops-nsieve-bits.js:
889         * tests/math-partial-sums.js:
890         * tests/math-spectral-norm.js:
891         * tests/string-fasta.js:
892
893 2007-10-20  Eric Seidel  <eric@webkit.org>
894
895         Reviewed by darin.
896         
897         Add improved argument handling to sunspider, including
898         --runs=<number>, --tests=<pattern, --shark, and --help
899         Also re-factor code into subroutines
900
901         * sunspider: improved argument handling
902
903 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
904
905         Reviewed by Mark.
906
907         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
908         
909         * TODO:
910         * tests/LIST:
911         * tests/access-binary-trees.js: Added.
912         * tests/access-nsieve.js: Added.
913         * tests/bitops-nsieve-bits.js: Added.
914         * tests/math-partial-sums.js: Added.
915         * tests/math-spectral-norm.js: Added.
916         * tests/string-fasta.js: Added.
917
918 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
919
920         Reviewed by Darin Adler.
921
922         * TODO: Updated to note areas that are now well-covered.
923         
924         - Change the tests and the driver to avoid leaving large object
925         graphs hanging around, since that throws off the subsequent tests.
926         
927         * resources/sunspider-standalone-driver.js:
928         * tests/3d-cube.js:
929         * tests/3d-morph.js:
930         * tests/3d-raytrace.js:
931         * tests/bitops-bitwise-and.js:
932         * tests/string-base64.js:
933         * tests/string-tagcloud.js:
934
935 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
936
937         Reviewed by Nikolas.
938         
939         - Rebalance test complexity.
940         
941         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
942         
943         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
944
945         However, for some tests, IE7 was a huge outlier compared to other
946         major browsers. For those tests, I used Firefox 2.0.0.8 on the
947         same OS and hardware as a normalization basis.
948         
949         * tests/3d-morph.js:
950         * tests/3d-raytrace.js:
951         * tests/bitops-3bit-bits-in-byte.js:
952         * tests/bitops-bits-in-byte.js:
953         * tests/bitops-bitwise-and.js:
954         * tests/math-cordic.js:
955         * tests/string-base64.js:
956         * tests/string-tagcloud.js:
957         * tests/string-unpack-code.js:
958
959 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
960
961         Reviewed by Oliver.
962         
963         - Fix the browser-hosted driver to be compatible with Safari 2
964
965         * hosted/sunspider-driver.html:
966
967 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
968
969         Rubber stamped by Hyatt.
970         
971         - Add BSD license notice to cordic test, the original author said to do so.
972
973         * tests/math-cordic.js:
974
975 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
976
977         Reviewed by Eric.
978         
979         - Support Windows and Windows browsers.
980
981         * hosted/sunspider-results.html: Display results correctly in IE.
982         * sunspider: tweaks to work on Windows, and with spaces in the path.
983
984 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
985
986         Rubber stamped by Oliver.
987
988         * tests/string-tagcloud.js: Remove stray print()
989
990 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
991
992         Reviewed by Oliver.
993         
994         Add some string test cases.
995
996         * tests/string-base64.js: Added. Does base64 encoding and decoding.
997         * tests/string-tagcloud.js: Added. Parses a JSON string and
998         generates tagcloud HTML markup based on the data.        
999         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1000         four of the most popular JavaScript libraries.
1001         * tests/LIST: Added the new tests.
1002
1003 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1004
1005         Reviewed by Anders.
1006
1007         * TODO: Updated with info on how to do the stats properly.
1008
1009         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1010         avoid overly slowing down the benchmark.
1011
1012         Added some 3D-related test cases.
1013         
1014         * tests/3d-cube.js: Added.
1015         * tests/3d-morph.js: Added.
1016         * tests/3d-raytrace.js: Added.
1017         * tests/LIST:
1018
1019 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1020
1021         Reviewed by Darin Adler.
1022
1023         - New JavaScript benchmark
1024         http://bugs.webkit.org/show_bug.cgi?id=15515
1025         
1026         A start on a new JavaScript benchmark, with standalone and
1027         browser-hosted drivers.
1028         
1029         The test content is still incomplete and the drivers could use
1030         some more features but there is enough here to make a good start.
1031         
1032         The drivers compute a 95% confidence interval on the mean for the
1033         whole test, each category, and each individual test to make it
1034         easier to tell whether differences are statistically
1035         significant. The confidence interval can be narrowed by running
1036         the test more times.
1037         
1038         * TODO: Added. Remaining things that need to be done.
1039         * sunspider: Added. Perl script that acts as the standalone test driver.
1040         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1041         of standalone test driver.
1042         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1043         analysis code.
1044         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1045         * make-hosted: Added. Script to generate browser-hosted tests.
1046         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1047         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1048         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1049         tests.
1050         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1051         tests.
1052         * tests/LIST: Added. List of tests to use.
1053         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1054         * tests/bitops-bits-in-byte.js: Added. Ditto.
1055         * tests/bitops-bitwise-and.js: Added. Ditto.
1056         * tests/math-cordic.js: Added. Ditto.
1057