2011-07-02 Maciej Stachowiak <mjs@apple.com>
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
2
3         Sunspider 0.9.1 harness doesn't actually close() its test documents
4         https://bugs.webkit.org/show_bug.cgi?id=47045
5
6         Reviewed by Daniel Bates.
7
8         * hosted/versions.html:
9         * resources/driver-TEMPLATE.html:
10
11 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
12
13         Not Reviewed. 
14         
15         Fix obviou typo in previous commit.
16
17         * tests/sunspider-1.0/bitops-bitwise-and.js:
18
19 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
20
21         Reviewed by Anders Carlsson.
22
23         SunSpider: all four bitops benchmarks can be replaced with NOP
24         https://bugs.webkit.org/show_bug.cgi?id=38446
25
26         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
27         (TimeFunc): Save the result.
28         * tests/sunspider-1.0/bitops-bits-in-byte.js:
29         (TimeFunc): Save the result.
30         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
31         save the results.
32         * tests/sunspider-1.0/bitops-nsieve-bits.js:
33         (sieve): Save the result.
34
35 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
36
37         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
38         https://bugs.webkit.org/show_bug.cgi?id=60937
39
40         Reviewed by Dan Bernstein.
41
42         * tests/sunspider-1.0/string-validate-input.js:
43         (doTest): Rename the variable to username
44
45 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
46
47         Reviewed by Darin Adler.
48
49         SunSpider: The main function of math-cordic is dead code
50         https://bugs.webkit.org/show_bug.cgi?id=63863
51         
52         Try to prevent math-cordic from being eliminated as dead code or
53         compiled down to a constant.
54
55         * tests/sunspider-1.0/math-cordic.js:
56         (cordicsincos):
57         (cordic):
58
59 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
60
61         Reviewed by Adam Barth.
62
63         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
64         https://bugs.webkit.org/show_bug.cgi?id=63782
65
66         * make-hosted: Learn about the new directory.
67         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
68
69 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
70
71         Reviewed by Gavin Barraclough.
72
73         Improved accuracy of command-line SunSpider.
74
75         * resources/sunspider-standalone-driver.js: Use the returned number of
76         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
77         ourselves, for slightly more accurate numbers.
78
79 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
80
81         Reviewed by Mark Rowe.
82         https://bugs.webkit.org/show_bug.cgi?id=45924
83
84         Modify the SunSpider harness to allow a path to be passed to the --suite
85         argument, supress the normal prepend-"tests/" behaviour for suite names
86         containing a '/'.
87
88         Also wrap the code in an anonymous function to move variables out of global
89         scope (currently the harness cannot handle running tests the overwrite the
90         variable 'j').
91
92         * resources/sunspider-standalone-driver.js:
93         * sunspider:
94
95 2010-08-27  Michael Saboff  <msaboff@apple.com>
96
97         Reviewed by Stephanie Lewis.
98
99         Added version 5 and version 6 of V8 tests with modifications similar
100         to the version 4 tests so they'll work with the SunSpider test harness.
101         Note that these files are external source and therefore don't conform 
102         to WebKit coding standards.
103         https://bugs.webkit.org/show_bug.cgi?id=44638
104
105         * tests/v8-v5: Added.
106         * tests/v8-v5/LIST: Added.
107         * tests/v8-v5/v8-crypto.js: Added.
108         * tests/v8-v5/v8-deltablue.js: Added.
109         * tests/v8-v5/v8-earley-boyer.js: Added.
110         * tests/v8-v5/v8-raytrace.js: Added.
111         * tests/v8-v5/v8-regexp.js: Added.
112         * tests/v8-v5/v8-richards.js: Added.
113         * tests/v8-v5/v8-splay.js: Added.
114         * tests/v8-v6: Added.
115         * tests/v8-v6/LIST: Added.
116         * tests/v8-v6/v8-crypto.js: Added.
117         * tests/v8-v6/v8-deltablue.js: Added.
118         * tests/v8-v6/v8-earley-boyer.js: Added.
119         * tests/v8-v6/v8-raytrace.js: Added.
120         * tests/v8-v6/v8-regexp.js: Added.
121         * tests/v8-v6/v8-richards.js: Added.
122         * tests/v8-v6/v8-splay.js: Added.
123
124 2010-07-15  Stephanie Lewis  <slewis@apple.com>
125
126         Reviewed by Geoff Garen.
127
128         http://bugs.webkit.org/show_bug.cgi?id=42406
129         Output location of results file.
130
131         * sunspider:
132
133 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
134
135         Reviewed by Mark Rowe.
136
137         Browser-hosted version of SunSpider runs tests twice as many times as advertised
138         https://bugs.webkit.org/show_bug.cgi?id=39201
139
140         * make-hosted: Make sure to reset test list after every test suite.
141
142 2010-05-07  Adam Roben  <aroben@apple.com>
143
144         Fix typo in sunspider-compare-results.js that caused incorrect
145         two-sample t statistics to be computed
146
147         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
148         two-sample t statistics
149
150         Reviewed by Darin Adler.
151
152         * resources/sunspider-compare-results.js:
153         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
154         variable to match the definition of a two-sample t statistic. Looks
155         like this was just a simple typo.
156
157 2010-04-05  Darin Adler  <darin@apple.com>
158
159         Updated the ignore property for changes to the contents here.
160
161         * .: Modified property svn:ignore.
162
163 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
164
165         Reviewed by Adam Barth.
166
167         Add sunspider-0.9.1 to the site in preparation for announcing it.
168         https://bugs.webkit.org/show_bug.cgi?id=35206
169
170         * hosted/sunspider.css: Remove tabs, which I noticed where present while
171         copying to the WebKitSite directory.
172
173 2010-01-11  Oliver Hunt  <oliver@apple.com>
174
175         Reviewed by Darin Adler.
176
177         Sunspider parse-only tests don't run
178         https://bugs.webkit.org/show_bug.cgi?id=33489
179
180         Fix parse-only test list
181
182         * tests/parse-only/LIST:
183
184 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
185
186         Reviewed by Adele Peterson.
187
188         Make SunSpider version more prominent in the title
189         https://bugs.webkit.org/show_bug.cgi?id=32574
190
191         * hosted/sunspider.html:
192         * hosted/versions.html:
193         * resources/driver-TEMPLATE.html:
194         * resources/results-TEMPLATE.html:
195
196 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
197
198         Reviewed by Darin Adler.
199
200         Make SunSpider refuse to compare results across content versions
201         https://bugs.webkit.org/show_bug.cgi?id=32573
202         
203         The results URL now includes a version indicator, and compare mode
204         will refuse to compare between versions.
205
206         * resources/driver-TEMPLATE.html:
207         * resources/results-TEMPLATE.html:
208
209 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
210
211         Reviewed by Darin Adler.
212
213         Some Browser-hosted SunSpider files are not valid HTML5
214         https://bugs.webkit.org/show_bug.cgi?id=32536
215         
216         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
217         close tags and adding a meta charset declaration).
218
219         * hosted/sunspider.html:
220         * hosted/versions.html:
221         * resources/TEMPLATE.html:
222         * resources/driver-TEMPLATE.html:
223         * resources/results-TEMPLATE.html:
224
225 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
226
227         Reviewed by Eric Seidel.
228
229         Make sunspider-0.9.1 the default content set (both command-line and hosted)
230         https://bugs.webkit.org/show_bug.cgi?id=32537
231
232         * hosted/sunspider.html: Change default suite link.
233         * sunspider: Change default suite.
234         * sunspider-compare-results: Likewise.
235
236 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
237
238         Reviewed by Sam Weinig.
239
240         in-browser SunSpider suffers excessive penalty under power management
241         https://bugs.webkit.org/show_bug.cgi?id=32505
242         
243         I have made a few changes to address this:
244         
245         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
246         document.write to reduce triggering of progress UI.
247         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
248         stepped all the way down the whole time.
249         3) Run the test cycle an extra time for warmup before the runs that count.
250
251         * hosted/sunspider-record-result.js: Removed.
252         * make-hosted:
253         * resources/TEMPLATE.html:
254         * resources/driver-TEMPLATE.html:
255
256 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
257
258         Reviewed by Gavin Barraclough.
259
260         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
261         https://bugs.webkit.org/show_bug.cgi?id=16806
262
263         Based on a patch by Eric Seidel.
264         
265         Fix the base64 computation to actually compute correct results. The impact on runtime of
266         the test is pretty small, but noticeable for some browsers. But at least it's not
267         doing a wrong and meaningless computation any more.
268         
269         * tests/sunspider-0.9.1/string-base64.js:
270         ():
271         (base64ToString):
272
273 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
274
275         Fixing commit error...
276         
277         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
278
279         * resources/driver-TEMPLATE.html:
280
281 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
282
283         Reviewed by Sam Weinig.
284
285         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
286         https://bugs.webkit.org/show_bug.cgi?id=32490
287
288         * hosted/json2.js: Added. Incorporated from json.org
289         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
290         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
291         results.
292
293 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
294
295         Reviewed by Oliver Hunt.
296
297         Enable Web-hosted version of SunSpider to handle multiple versions
298         https://bugs.webkit.org/show_bug.cgi?id=32478
299
300         * make-hosted: Now generate the hosted version to be able to run both the
301         0.9 and the 0.9.1 test suites.
302         * hosted: Modified property svn:ignore.
303         * hosted/sunspider-driver.html: Removed.
304         * hosted/sunspider-results.html: Removed.
305         * hosted/sunspider.html:
306         * hosted/versions.html: Added.
307         * resources/TEMPLATE.html:
308         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
309         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
310
311 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
312
313         Reviewed by Oliver Hunt.
314
315         Give command-line sunspider the ability to handle multiple suites and versions
316         https://bugs.webkit.org/show_bug.cgi?id=32477
317         
318         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
319         enumerating the tests. Current suites are:
320         
321         sunspider-0.9 - riginal SunSpider content 
322         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
323         ubench - "ubench" microbenchmark suite
324         v8-v4 - version 4 of the v8 benchmark
325         parse-only - some parser tests
326         
327         Sunspider can now be run with the --suite= parameter to select a
328         suite. The default is --suite=sunspider-0.9.
329
330         * .: Modified property svn:ignore to ignore new results directories.
331         * make-hosted: Generate based on sunspider-0.9 suite.
332         * resources/TEMPLATE.html: Fix an HTML compliance problem.
333         * resources/sunspider-standalone-driver.js: Become aware of suite names.
334         * sunspider: Modified to handle multiple suites as described above.
335         * sunspider-compare-results: ditto
336         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
337         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
338         * tests/LIST-UBENCH: Removed.
339         * tests/LIST-V8: Removed.
340         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
341         * tests/3d-morph.js: ditto
342         * tests/3d-raytrace.js: ditto
343         * tests/access-binary-trees.js: ditto
344         * tests/access-fannkuch.js: ditto
345         * tests/access-nbody.js: ditto
346         * tests/access-nsieve.js: ditto
347         * tests/bitops-3bit-bits-in-byte.js: ditto
348         * tests/bitops-bits-in-byte.js: ditto
349         * tests/bitops-bitwise-and.js: ditto
350         * tests/bitops-nsieve-bits.js: ditto
351         * tests/controlflow-recursive.js: ditto
352         * tests/crypto-aes.js: ditto
353         * tests/crypto-md5.js: ditto
354         * tests/crypto-sha1.js: ditto
355         * tests/date-format-tofte.js: ditto
356         * tests/date-format-xparb.js: ditto
357         * tests/math-cordic.js: ditto
358         * tests/math-partial-sums.js: ditto
359         * tests/math-spectral-norm.js: ditto
360         * tests/regexp-dna.js: ditto
361         * tests/string-base64.js: ditto
362         * tests/string-fasta.js: ditto
363         * tests/string-tagcloud.js: ditto
364         * tests/string-unpack-code.js: ditto
365         * tests/string-validate-input.js: ditto
366         * tests/sunspider-0.9/: Added
367         * tests/sunspider-0.9/*.js: Moved from one level up
368         * tests/sunspider-0.9.1: Added.
369         * tests/sunspider-0.9.1/*.js: Moved from one level up.
370         * tests/v8-crypto.js: Moved to v8-v4/ directory.
371         * tests/v8-deltablue.js: ditto
372         * tests/v8-earley-boyer.js: ditto
373         * tests/v8-raytrace.js: ditto
374         * tests/v8-regexp.js: ditto
375         * tests/v8-richards.js: ditto
376         * tests/v8-splay.js: ditto
377         * tests/v8-v4: Added.
378
379 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
380
381         Reviewed by Darin Adler.
382
383         Platform specific null device replaced with a platform independent.
384         https://bugs.webkit.org/show_bug.cgi?id=29544
385
386         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
387
388 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
389
390         Reviewed by Maciej Stachowiak.
391
392         [Qt] Make sunspider script work on Windows platform.
393         https://bugs.webkit.org/show_bug.cgi?id=29544
394
395         * sunspider:
396         Using platform specific null device instead of /dev/null.
397
398 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
399
400         Reviewed by Oliver Hunt.
401
402         https://bugs.webkit.org/show_bug.cgi?id=26540
403         Currently the SunSpider test driver lacks an option to run a test suite that
404         will test JavaScriptCore parsing performance only.  This patch adds just such
405         a test suite and option to SunSpider as well as the jsc test shell.  I've included
406         three large javascript source files found in the wild: jquery, mootools and prototype.
407         Combined with the concatenation of all three, these form a new testsuite to measure
408         and test pure JavaScriptCore parsing performance.
409
410         * resources/sunspider-standalone-driver.js:
411           Load the js with parse-only flag if it is part of the parse-only suite.
412         * sunspider:
413           Add the parse-only suite to the list of options.
414         * sunspider-compare-results:
415           Add the parse-only suite to the list of options.
416         * tests/LIST-PARSE-ONLY: Added.
417         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
418           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
419         * tests/parse-only/jquery-1.3.2.js: Added.
420           Latest version from jquery.com.
421         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
422           Latest version from mootools.net.
423         * tests/parse-only/prototype-1.6.0.3.js: Added.
424           Latest version from prototypejs.org.
425
426 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
427
428         Update V8 benchmark to version 4.
429         
430         Reviewed by Cameron Zwarich.
431
432         * tests/LIST-V8:
433         * tests/v8-crypto.js:
434         * tests/v8-deltablue.js:
435         * tests/v8-earley-boyer.js:
436         * tests/v8-raytrace.js:
437         * tests/v8-regexp.js: Added.
438         * tests/v8-richards.js:
439         * tests/v8-splay.js: Added.
440
441 2009-05-26  Olivier DOLE  <odole@pleyo.com>
442
443         Reviewed by Maciej.
444
445         Store sunspider test suite results in a specific directory for each
446         different type of benchmark (sunspider, V8 or ubench).
447
448         * sunspider:
449         * sunspider-compare-results:
450
451 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
452
453         Build fix.
454
455         * SunSpider.make:
456
457 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
458
459         Include SunSpider tests in production builds.
460         
461         Reviewed by Adam Roben.
462
463         * SunSpider.make: Added.
464         * xcopy.excludes: Added.
465
466 2009-03-04  Oliver Hunt  <oliver@apple.com>
467
468         Added a few more tests to ubench
469
470         RS=Gavin Barraclough
471
472         Adding a couple of test to cover too few and too many arguments.  This
473         is needed to prevent regressions in all call paths from my current refactoring.
474
475         * tests/LIST-UBENCH:
476         * tests/ubench/function-correct-args.js: Added.
477         (f):
478         * tests/ubench/function-excess-args.js: Added.
479         (f):
480
481 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
482
483         Fix a bug where when V8's or UBENCH's test suite list file is missing
484         the error message is suggesting that SunSpider's list file is missing.
485
486         Reviewed by Sam Weinig.
487
488         * sunspider:
489         (loadTestsList): Fix the error message to use the right list name.
490
491 2008-10-07  David Hyatt  <hyatt@apple.com>
492
493         https://bugs.webkit.org/show_bug.cgi?id=18989
494
495         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
496         we want all browsers to be doing the same thing on the test.
497
498         Reviewed by Maciej
499
500         * tests/regexp-dna.js:
501
502 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
503
504         Rubber-stamped by Maciej Stachowiak.
505
506         The bleeding edge branch of V8 now supports passing fileanmes after
507         '-f', so the SunSpider script should be changed to do this again.
508         This allows all 3 major JavaScript engines to be used with the
509         unmodified SunSpider script.
510
511         * sunspider:
512
513 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
514
515         Reviewed by Maciej Stachowiak.
516
517         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
518         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
519         correct the punctuation in its documentation.
520
521         * sunspider:
522
523 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
524
525         Reviewed by Oliver Hunt.
526
527         Bug 20616: Incorporate V8 benchmarks in testing
528         <https://bugs.webkit.org/show_bug.cgi?id=20616>
529
530         This is a first attempt at running the V8 benchmarks within SunSpider,
531         with the --v8 option. The number of iterations is fixed for each
532         benchmark, and was chosen to make each of the benchmarks run between
533         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
534
535         * sunspider:
536         * tests/LIST-V8: Added.
537         * tests/v8-crypto.js: Added.
538         * tests/v8-deltablue.js: Added.
539         * tests/v8-earley-boyer.js: Added.
540         * tests/v8-raytrace.js: Added.
541         * tests/v8-richards.js: Added.
542
543 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
544
545         Not reviewed.
546
547         Remove a local change that was accidentally committed in r36034.
548
549         * resources/sunspider-standalone-driver.js:
550
551 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
552
553         Rubber-stamped by Oliver Hunt.
554
555         Make SunSpider work with JavaScript shells that don't take filenames
556         after a '-f' argument.
557
558         * sunspider:
559
560 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
561
562         Reviewed by Timothy.
563
564         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
565
566         * sunspider:
567         * tests/LIST-SQUIRRELFISH: Removed.
568
569 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
570
571         Rubber-stamped by Mark Rowe.
572
573         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
574
575         * sunspider:
576
577 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
578
579         Rubber stamped by Oliver.
580         
581         - add newly working tests to squirrelfish list
582
583         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
584
585 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
586
587         Rubber stamped by Oliver.
588         
589         - move ubench tests properly.
590
591         * tests/ubench: Added.
592         * tests/ubench/function-closure.js: Added.
593         * tests/ubench/function-empty.js: Added.
594         * tests/ubench/function-missing-args.js: Added.
595         * tests/ubench/function-sum.js: Added.
596         * tests/ubench/loop-empty-resolve.js: Added.
597         * tests/ubench/loop-empty.js: Added.
598         * tests/ubench/loop-sum.js: Added.
599
600 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
601
602         Rubber stamped by Oliver.
603
604         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
605
606 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
607
608         Reviewed by Oliver.
609
610         - make --squirrelfish mode use the real harness, but a reduced set of tests
611
612         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
613
614         * sunspider:
615         * tests/LIST-SQUIRRELFISH:
616         * tests/LIST-UBENCH: Added.
617         * tests/squirrelfish: Moved to tests/ubench
618
619 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
620
621         Reviewed by Maciej Stachowiak.
622         
623         Removed an unused test.
624
625         * tests/squirrelfish/loop-resolve.js: Removed.
626
627 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
628
629         Reviewed by Oliver.
630         
631         Add newly runnable tests to --squirrelfish mode.
632
633         * tests/LIST-SQUIRRELFISH:
634
635 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
636
637         Reviewed by Maciej Stachowiak.
638         
639         A little more love for --squirrelfish mode:
640         
641         Fixed a misplaced sort. Results properly sort now.
642         
643         Added a test list just for squirrelfish, and updated it to include
644         all currently passing SunSpider tests.
645         
646         Fixed the pruning regexp to match 3d-morph.js.
647
648         * sunspider:
649
650 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
651
652         Reviewed by Oliver Hunt.
653         
654         --squirrelfish mode: pared down tests for squirrelfish to chew on.
655
656         * sunspider:
657         * tests/LIST:
658         * tests/squirrelfish: Added.
659         * tests/squirrelfish/function-closure.js: Added.
660         * tests/squirrelfish/function-empty.js: Added.
661         * tests/squirrelfish/function-missing-args.js: Added.
662         * tests/squirrelfish/function-sum.js: Added.
663         * tests/squirrelfish/loop-empty-resolve.js: Added.
664         * tests/squirrelfish/loop-empty.js: Added.
665         * tests/squirrelfish/loop-resolve.js: Added.
666         * tests/squirrelfish/loop-sum.js: Added.
667
668 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
669
670         Reviewed by Geoff.
671
672         * sunspider: Pause update daemon on Darwin to make results reliable
673         again (workaround for <rdar://problem/5811127>).
674
675 2008-02-07  Eric Seidel  <eric@webkit.org>
676
677         Reviewed by Sam.
678
679         * sunspider: add an error message when ./tests/LIST is not found
680
681 2007-12-29  Alp Toker  <alp@atoker.com>
682
683         Fix typo.
684
685         * hosted/sunspider.html:
686
687 2007-12-29  Alp Toker  <alp@atoker.com>
688
689         Reviewed by Maciej.
690
691         Provide the current URL in the SunSpider results page. Useful for
692         copying benchmark results out of browsers without a location bar or
693         where the location bar can't handle long URLs.
694
695         * hosted/sunspider-results.html:
696
697 2007-12-20  Eric Seidel  <eric@webkit.org>
698
699         Reviewed by mjs.
700
701         * sunspider: increase default number of runs for more accurate numbers
702
703 2007-12-18  Mark Rowe  <mrowe@apple.com>
704
705         Fix typo.
706
707         * hosted/sunspider.html:
708
709 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
710
711         Reviewed by Eric.
712         
713         - give an extra digit of precision for small differences.
714
715         * resources/sunspider-compare-results.js:
716
717 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
718
719         Reviewed by Eric.
720         
721         - work around IE layout bugs
722
723         * hosted/sunspider.css:
724
725 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
726
727         Reviewed by Mark.
728         
729         - fix compare feature
730
731         * hosted/sunspider-results.html:
732
733 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
734
735         Reviewed by Mark.
736         
737         - Change the format of the output URL to be shorter, to meet IE constraints
738
739         * hosted/sunspider-driver.html:
740         * hosted/sunspider-results.html:
741
742 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
743
744         Reviewed by Oliver.
745         
746         - Add some explanatory test and a dumb logo to SunSpider web hosted version
747
748         * hosted/sunspider-driver.html:
749         * hosted/sunspider-results.html:
750         * hosted/sunspider.css:
751         * hosted/sunspider.html:
752
753 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
754
755         Reviewed by Oliver.
756         
757         - Make the web-hosted version of SunSpider look a little bit less drab.
758
759         * hosted/sunspider-driver.html:
760         * hosted/sunspider-results.html:
761         * hosted/sunspider.css: Added.
762         * hosted/sunspider.html:
763         * resources/TEMPLATE.html:
764
765 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
766
767         Reviewed by Sam.
768         
769         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
770         
771         A sample of what new-style results would look like:
772         
773         TEST                   COMPARISON            FROM                 TO             DETAILS
774         ...
775         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
776         ...
777             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
778         
779         * resources/sunspider-compare-results.js:
780
781 2007-11-14  Eric Seidel  <eric@webkit.org>
782
783         Reviewed by Sam.
784
785         * sunspider: add --shark-cache for L2 Cache Miss profiling
786
787 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
788
789         Reviewed by Darin Adler.
790
791         http://bugs.webkit.org/show_bug.cgi?id=15925
792         SunSpider should check for Shark being installed
793
794         * sunspider: Explicitly test that shark command line tool is installed.
795
796 2007-11-04  Darin Adler  <darin@apple.com>
797
798         * hosted/sunspider.html: Tweaked formatting a bit.
799
800 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
801
802         Reviewed by Darin Adler.
803
804         - Add compare mode and a description of the benchmark to the hosted version.
805
806         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
807         * hosted/sunspider.html: Added intro text.
808         * make-hosted: Copy new file.
809         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
810         a function.
811         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
812         expects.
813         * sunspider-compare-results: Use sunspider-standalone-compare.js.
814
815 2007-11-02  Darin Adler  <darin@apple.com>
816
817         * sunspider: Changed "--base" to "--set-baseline".
818
819 2007-11-02  Darin Adler  <darin@apple.com>
820
821         Reviewed by Maciej.
822
823         * sunspider: Added a "--base" option that records the current test as a baseline.
824         Moved the code to find the newest .mshark file into a function (since I needed to use
825         it in the other script and I prefer to copy/paste a function).
826
827         * sunspider-compare-results: Added logic to compare the baseline with the last
828         run when you don't pass any parameters.
829
830 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
831
832         Reviewed by Eric.
833         
834         - Added list of ECMAScript3 features that SunSpider does not cover currently.
835
836         * UNCOVERED: Added.
837
838 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
839
840         Reviewed by Oliver.
841         
842         - Give better error messages when sunspider-compare-results fails
843
844         * sunspider-compare-results:
845
846 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
847
848         Reviewed by Darin Adler.
849         
850         - fix Date tests to work in command-line SpiderMonkey
851
852         * tests/date-format-tofte.js:
853         * tests/date-format-xparb.js:
854
855 2007-10-29  Darin Adler  <darin@apple.com>
856
857         Reviewed by Maciej.
858
859         * sunspider: Added --shark20 option, to run Shark at its highest sample
860         resolution instead of its default.
861
862 2007-10-28  Adam Roben  <aroben@apple.com>
863
864         Fix SunSpider on Windows
865
866         Reviewed by Eric.
867
868         * sunspider: Don't use colons in filenames.
869
870 2007-10-28  Darin Adler  <darin@apple.com>
871
872         Reviewed by Adam.
873
874         * resources/sunspider-analyze-results.js: Tweak the output format a little.
875         Change so that when there's only one run we don't write out confidence
876         intervals at all rather than writing "NaN" over and over again.
877
878         * sunspider: Use the time and date as a suffix on the results file, that way
879         all the results are left behind in the tmp directory. This will make it easy
880         for us to add features that compare past results in the future. Also rename
881         the .mshark files using the same scheme.
882
883         * sunspider-compare-results: Relax the parsing rules so we can parse the
884         results file exactly as it's emitted from sunspider.
885
886 2007-10-25  Adam Roben  <aroben@apple.com>
887
888         Strip carriage returns from results
889
890         Carriage returns were screwing up sunspider-compare-results on
891         Windows.
892
893         Rubberstamped by Oliver.
894
895         * sunspider:
896
897 2007-10-24  Darin Adler  <darin@apple.com>
898
899         Reviewed by Eric.
900
901         * sunspider: Open the profile in shark after completing a run.
902         * sunspider-compare-results: Tiny perl tweaks.
903
904 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
905
906         Reviewed by Darin Adler.
907
908         - Don't hardcode my path to testkjs
909         
910         * sunspider-compare-results:
911
912 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
913
914         Reviewed by Darin Adler.
915         
916         - Add a compare script to compare results, and improve formatting of standard results a bit.
917
918         * resources/sunspider-analyze-results.js:
919         * resources/sunspider-compare-results.js: Added.
920         * sunspider-compare-results: Added.
921
922 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
923
924         Reviewed by Darin Adler.
925
926         * resources/sunspider-analyze-results.js: Do the error range
927         properly, using the t-distribution instead of 1.96 (which was
928         based on the normal distribution).
929         
930         * sunspider: Print results in a way that will be friendlier to the
931         soon-to-come compare mode.
932
933 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
934
935         Reviewed by Review.
936         
937         - add more tests (probably the final set for now)
938         
939         Already balanced. Added date, regexp, control flow, and a few
940         more string and object/array access tests.
941         
942         * tests/LIST:
943         * tests/access-fannkuch.js: Added.
944         * tests/access-nbody.js: Added.
945         * tests/controlflow-recursive.js: Added.
946         * tests/date-format-tofte.js: Added.
947         * tests/date-format-xparb.js: Added.
948         * tests/regexp-dna.js: Added.
949         * tests/string-validate-input.js: Added.
950
951 2007-10-21  Eric Seidel  <eric@webkit.org>
952
953         Reviewed by mjs.
954
955         * sunspider: make --shark only sample from testkjs
956
957 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
958
959         Rubber stamped by Eric.
960         
961         - added crypto tests (already balanced)
962
963         * tests/LIST:
964         * tests/crypto-aes.js: Added.
965         * tests/crypto-md5.js: Added.
966         * tests/crypto-sha1.js: Added.
967
968 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
969
970         Reviewed by Adam.
971         
972         - rebalance new tests
973         
974         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
975         
976         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
977
978         This balances them with the existing tests.
979
980         * tests/access-binary-trees.js:
981         * tests/access-nsieve.js:
982         * tests/bitops-nsieve-bits.js:
983         * tests/math-partial-sums.js:
984         * tests/math-spectral-norm.js:
985         * tests/string-fasta.js:
986
987 2007-10-20  Eric Seidel  <eric@webkit.org>
988
989         Reviewed by darin.
990         
991         Add improved argument handling to sunspider, including
992         --runs=<number>, --tests=<pattern, --shark, and --help
993         Also re-factor code into subroutines
994
995         * sunspider: improved argument handling
996
997 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
998
999         Reviewed by Mark.
1000
1001         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1002         
1003         * TODO:
1004         * tests/LIST:
1005         * tests/access-binary-trees.js: Added.
1006         * tests/access-nsieve.js: Added.
1007         * tests/bitops-nsieve-bits.js: Added.
1008         * tests/math-partial-sums.js: Added.
1009         * tests/math-spectral-norm.js: Added.
1010         * tests/string-fasta.js: Added.
1011
1012 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1013
1014         Reviewed by Darin Adler.
1015
1016         * TODO: Updated to note areas that are now well-covered.
1017         
1018         - Change the tests and the driver to avoid leaving large object
1019         graphs hanging around, since that throws off the subsequent tests.
1020         
1021         * resources/sunspider-standalone-driver.js:
1022         * tests/3d-cube.js:
1023         * tests/3d-morph.js:
1024         * tests/3d-raytrace.js:
1025         * tests/bitops-bitwise-and.js:
1026         * tests/string-base64.js:
1027         * tests/string-tagcloud.js:
1028
1029 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1030
1031         Reviewed by Nikolas.
1032         
1033         - Rebalance test complexity.
1034         
1035         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1036         
1037         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1038
1039         However, for some tests, IE7 was a huge outlier compared to other
1040         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1041         same OS and hardware as a normalization basis.
1042         
1043         * tests/3d-morph.js:
1044         * tests/3d-raytrace.js:
1045         * tests/bitops-3bit-bits-in-byte.js:
1046         * tests/bitops-bits-in-byte.js:
1047         * tests/bitops-bitwise-and.js:
1048         * tests/math-cordic.js:
1049         * tests/string-base64.js:
1050         * tests/string-tagcloud.js:
1051         * tests/string-unpack-code.js:
1052
1053 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1054
1055         Reviewed by Oliver.
1056         
1057         - Fix the browser-hosted driver to be compatible with Safari 2
1058
1059         * hosted/sunspider-driver.html:
1060
1061 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1062
1063         Rubber stamped by Hyatt.
1064         
1065         - Add BSD license notice to cordic test, the original author said to do so.
1066
1067         * tests/math-cordic.js:
1068
1069 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1070
1071         Reviewed by Eric.
1072         
1073         - Support Windows and Windows browsers.
1074
1075         * hosted/sunspider-results.html: Display results correctly in IE.
1076         * sunspider: tweaks to work on Windows, and with spaces in the path.
1077
1078 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1079
1080         Rubber stamped by Oliver.
1081
1082         * tests/string-tagcloud.js: Remove stray print()
1083
1084 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1085
1086         Reviewed by Oliver.
1087         
1088         Add some string test cases.
1089
1090         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1091         * tests/string-tagcloud.js: Added. Parses a JSON string and
1092         generates tagcloud HTML markup based on the data.        
1093         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1094         four of the most popular JavaScript libraries.
1095         * tests/LIST: Added the new tests.
1096
1097 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1098
1099         Reviewed by Anders.
1100
1101         * TODO: Updated with info on how to do the stats properly.
1102
1103         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1104         avoid overly slowing down the benchmark.
1105
1106         Added some 3D-related test cases.
1107         
1108         * tests/3d-cube.js: Added.
1109         * tests/3d-morph.js: Added.
1110         * tests/3d-raytrace.js: Added.
1111         * tests/LIST:
1112
1113 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1114
1115         Reviewed by Darin Adler.
1116
1117         - New JavaScript benchmark
1118         http://bugs.webkit.org/show_bug.cgi?id=15515
1119         
1120         A start on a new JavaScript benchmark, with standalone and
1121         browser-hosted drivers.
1122         
1123         The test content is still incomplete and the drivers could use
1124         some more features but there is enough here to make a good start.
1125         
1126         The drivers compute a 95% confidence interval on the mean for the
1127         whole test, each category, and each individual test to make it
1128         easier to tell whether differences are statistically
1129         significant. The confidence interval can be narrowed by running
1130         the test more times.
1131         
1132         * TODO: Added. Remaining things that need to be done.
1133         * sunspider: Added. Perl script that acts as the standalone test driver.
1134         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1135         of standalone test driver.
1136         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1137         analysis code.
1138         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1139         * make-hosted: Added. Script to generate browser-hosted tests.
1140         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1141         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1142         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1143         tests.
1144         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1145         tests.
1146         * tests/LIST: Added. List of tests to use.
1147         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1148         * tests/bitops-bits-in-byte.js: Added. Ditto.
1149         * tests/bitops-bitwise-and.js: Added. Ditto.
1150         * tests/math-cordic.js: Added. Ditto.
1151