2011-07-02 Maciej Stachowiak <mjs@apple.com>
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
2
3         Reviewed by Anders Carlsson.
4
5         SunSpider: all four bitops benchmarks can be replaced with NOP
6         https://bugs.webkit.org/show_bug.cgi?id=38446
7
8         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
9         (TimeFunc): Save the result.
10         * tests/sunspider-1.0/bitops-bits-in-byte.js:
11         (TimeFunc): Save the result.
12         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
13         save the results.
14         * tests/sunspider-1.0/bitops-nsieve-bits.js:
15         (sieve): Save the result.
16
17 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
18
19         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
20         https://bugs.webkit.org/show_bug.cgi?id=60937
21
22         Reviewed by Dan Bernstein.
23
24         * tests/sunspider-1.0/string-validate-input.js:
25         (doTest): Rename the variable to username
26
27 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
28
29         Reviewed by Darin Adler.
30
31         SunSpider: The main function of math-cordic is dead code
32         https://bugs.webkit.org/show_bug.cgi?id=63863
33         
34         Try to prevent math-cordic from being eliminated as dead code or
35         compiled down to a constant.
36
37         * tests/sunspider-1.0/math-cordic.js:
38         (cordicsincos):
39         (cordic):
40
41 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
42
43         Reviewed by Adam Barth.
44
45         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
46         https://bugs.webkit.org/show_bug.cgi?id=63782
47
48         * make-hosted: Learn about the new directory.
49         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
50
51 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
52
53         Reviewed by Gavin Barraclough.
54
55         Improved accuracy of command-line SunSpider.
56
57         * resources/sunspider-standalone-driver.js: Use the returned number of
58         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
59         ourselves, for slightly more accurate numbers.
60
61 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
62
63         Reviewed by Mark Rowe.
64         https://bugs.webkit.org/show_bug.cgi?id=45924
65
66         Modify the SunSpider harness to allow a path to be passed to the --suite
67         argument, supress the normal prepend-"tests/" behaviour for suite names
68         containing a '/'.
69
70         Also wrap the code in an anonymous function to move variables out of global
71         scope (currently the harness cannot handle running tests the overwrite the
72         variable 'j').
73
74         * resources/sunspider-standalone-driver.js:
75         * sunspider:
76
77 2010-08-27  Michael Saboff  <msaboff@apple.com>
78
79         Reviewed by Stephanie Lewis.
80
81         Added version 5 and version 6 of V8 tests with modifications similar
82         to the version 4 tests so they'll work with the SunSpider test harness.
83         Note that these files are external source and therefore don't conform 
84         to WebKit coding standards.
85         https://bugs.webkit.org/show_bug.cgi?id=44638
86
87         * tests/v8-v5: Added.
88         * tests/v8-v5/LIST: Added.
89         * tests/v8-v5/v8-crypto.js: Added.
90         * tests/v8-v5/v8-deltablue.js: Added.
91         * tests/v8-v5/v8-earley-boyer.js: Added.
92         * tests/v8-v5/v8-raytrace.js: Added.
93         * tests/v8-v5/v8-regexp.js: Added.
94         * tests/v8-v5/v8-richards.js: Added.
95         * tests/v8-v5/v8-splay.js: Added.
96         * tests/v8-v6: Added.
97         * tests/v8-v6/LIST: Added.
98         * tests/v8-v6/v8-crypto.js: Added.
99         * tests/v8-v6/v8-deltablue.js: Added.
100         * tests/v8-v6/v8-earley-boyer.js: Added.
101         * tests/v8-v6/v8-raytrace.js: Added.
102         * tests/v8-v6/v8-regexp.js: Added.
103         * tests/v8-v6/v8-richards.js: Added.
104         * tests/v8-v6/v8-splay.js: Added.
105
106 2010-07-15  Stephanie Lewis  <slewis@apple.com>
107
108         Reviewed by Geoff Garen.
109
110         http://bugs.webkit.org/show_bug.cgi?id=42406
111         Output location of results file.
112
113         * sunspider:
114
115 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
116
117         Reviewed by Mark Rowe.
118
119         Browser-hosted version of SunSpider runs tests twice as many times as advertised
120         https://bugs.webkit.org/show_bug.cgi?id=39201
121
122         * make-hosted: Make sure to reset test list after every test suite.
123
124 2010-05-07  Adam Roben  <aroben@apple.com>
125
126         Fix typo in sunspider-compare-results.js that caused incorrect
127         two-sample t statistics to be computed
128
129         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
130         two-sample t statistics
131
132         Reviewed by Darin Adler.
133
134         * resources/sunspider-compare-results.js:
135         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
136         variable to match the definition of a two-sample t statistic. Looks
137         like this was just a simple typo.
138
139 2010-04-05  Darin Adler  <darin@apple.com>
140
141         Updated the ignore property for changes to the contents here.
142
143         * .: Modified property svn:ignore.
144
145 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
146
147         Reviewed by Adam Barth.
148
149         Add sunspider-0.9.1 to the site in preparation for announcing it.
150         https://bugs.webkit.org/show_bug.cgi?id=35206
151
152         * hosted/sunspider.css: Remove tabs, which I noticed where present while
153         copying to the WebKitSite directory.
154
155 2010-01-11  Oliver Hunt  <oliver@apple.com>
156
157         Reviewed by Darin Adler.
158
159         Sunspider parse-only tests don't run
160         https://bugs.webkit.org/show_bug.cgi?id=33489
161
162         Fix parse-only test list
163
164         * tests/parse-only/LIST:
165
166 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
167
168         Reviewed by Adele Peterson.
169
170         Make SunSpider version more prominent in the title
171         https://bugs.webkit.org/show_bug.cgi?id=32574
172
173         * hosted/sunspider.html:
174         * hosted/versions.html:
175         * resources/driver-TEMPLATE.html:
176         * resources/results-TEMPLATE.html:
177
178 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
179
180         Reviewed by Darin Adler.
181
182         Make SunSpider refuse to compare results across content versions
183         https://bugs.webkit.org/show_bug.cgi?id=32573
184         
185         The results URL now includes a version indicator, and compare mode
186         will refuse to compare between versions.
187
188         * resources/driver-TEMPLATE.html:
189         * resources/results-TEMPLATE.html:
190
191 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
192
193         Reviewed by Darin Adler.
194
195         Some Browser-hosted SunSpider files are not valid HTML5
196         https://bugs.webkit.org/show_bug.cgi?id=32536
197         
198         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
199         close tags and adding a meta charset declaration).
200
201         * hosted/sunspider.html:
202         * hosted/versions.html:
203         * resources/TEMPLATE.html:
204         * resources/driver-TEMPLATE.html:
205         * resources/results-TEMPLATE.html:
206
207 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
208
209         Reviewed by Eric Seidel.
210
211         Make sunspider-0.9.1 the default content set (both command-line and hosted)
212         https://bugs.webkit.org/show_bug.cgi?id=32537
213
214         * hosted/sunspider.html: Change default suite link.
215         * sunspider: Change default suite.
216         * sunspider-compare-results: Likewise.
217
218 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
219
220         Reviewed by Sam Weinig.
221
222         in-browser SunSpider suffers excessive penalty under power management
223         https://bugs.webkit.org/show_bug.cgi?id=32505
224         
225         I have made a few changes to address this:
226         
227         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
228         document.write to reduce triggering of progress UI.
229         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
230         stepped all the way down the whole time.
231         3) Run the test cycle an extra time for warmup before the runs that count.
232
233         * hosted/sunspider-record-result.js: Removed.
234         * make-hosted:
235         * resources/TEMPLATE.html:
236         * resources/driver-TEMPLATE.html:
237
238 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
239
240         Reviewed by Gavin Barraclough.
241
242         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
243         https://bugs.webkit.org/show_bug.cgi?id=16806
244
245         Based on a patch by Eric Seidel.
246         
247         Fix the base64 computation to actually compute correct results. The impact on runtime of
248         the test is pretty small, but noticeable for some browsers. But at least it's not
249         doing a wrong and meaningless computation any more.
250         
251         * tests/sunspider-0.9.1/string-base64.js:
252         ():
253         (base64ToString):
254
255 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
256
257         Fixing commit error...
258         
259         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
260
261         * resources/driver-TEMPLATE.html:
262
263 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
264
265         Reviewed by Sam Weinig.
266
267         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
268         https://bugs.webkit.org/show_bug.cgi?id=32490
269
270         * hosted/json2.js: Added. Incorporated from json.org
271         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
272         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
273         results.
274
275 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
276
277         Reviewed by Oliver Hunt.
278
279         Enable Web-hosted version of SunSpider to handle multiple versions
280         https://bugs.webkit.org/show_bug.cgi?id=32478
281
282         * make-hosted: Now generate the hosted version to be able to run both the
283         0.9 and the 0.9.1 test suites.
284         * hosted: Modified property svn:ignore.
285         * hosted/sunspider-driver.html: Removed.
286         * hosted/sunspider-results.html: Removed.
287         * hosted/sunspider.html:
288         * hosted/versions.html: Added.
289         * resources/TEMPLATE.html:
290         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
291         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
292
293 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
294
295         Reviewed by Oliver Hunt.
296
297         Give command-line sunspider the ability to handle multiple suites and versions
298         https://bugs.webkit.org/show_bug.cgi?id=32477
299         
300         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
301         enumerating the tests. Current suites are:
302         
303         sunspider-0.9 - riginal SunSpider content 
304         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
305         ubench - "ubench" microbenchmark suite
306         v8-v4 - version 4 of the v8 benchmark
307         parse-only - some parser tests
308         
309         Sunspider can now be run with the --suite= parameter to select a
310         suite. The default is --suite=sunspider-0.9.
311
312         * .: Modified property svn:ignore to ignore new results directories.
313         * make-hosted: Generate based on sunspider-0.9 suite.
314         * resources/TEMPLATE.html: Fix an HTML compliance problem.
315         * resources/sunspider-standalone-driver.js: Become aware of suite names.
316         * sunspider: Modified to handle multiple suites as described above.
317         * sunspider-compare-results: ditto
318         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
319         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
320         * tests/LIST-UBENCH: Removed.
321         * tests/LIST-V8: Removed.
322         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
323         * tests/3d-morph.js: ditto
324         * tests/3d-raytrace.js: ditto
325         * tests/access-binary-trees.js: ditto
326         * tests/access-fannkuch.js: ditto
327         * tests/access-nbody.js: ditto
328         * tests/access-nsieve.js: ditto
329         * tests/bitops-3bit-bits-in-byte.js: ditto
330         * tests/bitops-bits-in-byte.js: ditto
331         * tests/bitops-bitwise-and.js: ditto
332         * tests/bitops-nsieve-bits.js: ditto
333         * tests/controlflow-recursive.js: ditto
334         * tests/crypto-aes.js: ditto
335         * tests/crypto-md5.js: ditto
336         * tests/crypto-sha1.js: ditto
337         * tests/date-format-tofte.js: ditto
338         * tests/date-format-xparb.js: ditto
339         * tests/math-cordic.js: ditto
340         * tests/math-partial-sums.js: ditto
341         * tests/math-spectral-norm.js: ditto
342         * tests/regexp-dna.js: ditto
343         * tests/string-base64.js: ditto
344         * tests/string-fasta.js: ditto
345         * tests/string-tagcloud.js: ditto
346         * tests/string-unpack-code.js: ditto
347         * tests/string-validate-input.js: ditto
348         * tests/sunspider-0.9/: Added
349         * tests/sunspider-0.9/*.js: Moved from one level up
350         * tests/sunspider-0.9.1: Added.
351         * tests/sunspider-0.9.1/*.js: Moved from one level up.
352         * tests/v8-crypto.js: Moved to v8-v4/ directory.
353         * tests/v8-deltablue.js: ditto
354         * tests/v8-earley-boyer.js: ditto
355         * tests/v8-raytrace.js: ditto
356         * tests/v8-regexp.js: ditto
357         * tests/v8-richards.js: ditto
358         * tests/v8-splay.js: ditto
359         * tests/v8-v4: Added.
360
361 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
362
363         Reviewed by Darin Adler.
364
365         Platform specific null device replaced with a platform independent.
366         https://bugs.webkit.org/show_bug.cgi?id=29544
367
368         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
369
370 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
371
372         Reviewed by Maciej Stachowiak.
373
374         [Qt] Make sunspider script work on Windows platform.
375         https://bugs.webkit.org/show_bug.cgi?id=29544
376
377         * sunspider:
378         Using platform specific null device instead of /dev/null.
379
380 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
381
382         Reviewed by Oliver Hunt.
383
384         https://bugs.webkit.org/show_bug.cgi?id=26540
385         Currently the SunSpider test driver lacks an option to run a test suite that
386         will test JavaScriptCore parsing performance only.  This patch adds just such
387         a test suite and option to SunSpider as well as the jsc test shell.  I've included
388         three large javascript source files found in the wild: jquery, mootools and prototype.
389         Combined with the concatenation of all three, these form a new testsuite to measure
390         and test pure JavaScriptCore parsing performance.
391
392         * resources/sunspider-standalone-driver.js:
393           Load the js with parse-only flag if it is part of the parse-only suite.
394         * sunspider:
395           Add the parse-only suite to the list of options.
396         * sunspider-compare-results:
397           Add the parse-only suite to the list of options.
398         * tests/LIST-PARSE-ONLY: Added.
399         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
400           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
401         * tests/parse-only/jquery-1.3.2.js: Added.
402           Latest version from jquery.com.
403         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
404           Latest version from mootools.net.
405         * tests/parse-only/prototype-1.6.0.3.js: Added.
406           Latest version from prototypejs.org.
407
408 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
409
410         Update V8 benchmark to version 4.
411         
412         Reviewed by Cameron Zwarich.
413
414         * tests/LIST-V8:
415         * tests/v8-crypto.js:
416         * tests/v8-deltablue.js:
417         * tests/v8-earley-boyer.js:
418         * tests/v8-raytrace.js:
419         * tests/v8-regexp.js: Added.
420         * tests/v8-richards.js:
421         * tests/v8-splay.js: Added.
422
423 2009-05-26  Olivier DOLE  <odole@pleyo.com>
424
425         Reviewed by Maciej.
426
427         Store sunspider test suite results in a specific directory for each
428         different type of benchmark (sunspider, V8 or ubench).
429
430         * sunspider:
431         * sunspider-compare-results:
432
433 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
434
435         Build fix.
436
437         * SunSpider.make:
438
439 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
440
441         Include SunSpider tests in production builds.
442         
443         Reviewed by Adam Roben.
444
445         * SunSpider.make: Added.
446         * xcopy.excludes: Added.
447
448 2009-03-04  Oliver Hunt  <oliver@apple.com>
449
450         Added a few more tests to ubench
451
452         RS=Gavin Barraclough
453
454         Adding a couple of test to cover too few and too many arguments.  This
455         is needed to prevent regressions in all call paths from my current refactoring.
456
457         * tests/LIST-UBENCH:
458         * tests/ubench/function-correct-args.js: Added.
459         (f):
460         * tests/ubench/function-excess-args.js: Added.
461         (f):
462
463 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
464
465         Fix a bug where when V8's or UBENCH's test suite list file is missing
466         the error message is suggesting that SunSpider's list file is missing.
467
468         Reviewed by Sam Weinig.
469
470         * sunspider:
471         (loadTestsList): Fix the error message to use the right list name.
472
473 2008-10-07  David Hyatt  <hyatt@apple.com>
474
475         https://bugs.webkit.org/show_bug.cgi?id=18989
476
477         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
478         we want all browsers to be doing the same thing on the test.
479
480         Reviewed by Maciej
481
482         * tests/regexp-dna.js:
483
484 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
485
486         Rubber-stamped by Maciej Stachowiak.
487
488         The bleeding edge branch of V8 now supports passing fileanmes after
489         '-f', so the SunSpider script should be changed to do this again.
490         This allows all 3 major JavaScript engines to be used with the
491         unmodified SunSpider script.
492
493         * sunspider:
494
495 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
496
497         Reviewed by Maciej Stachowiak.
498
499         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
500         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
501         correct the punctuation in its documentation.
502
503         * sunspider:
504
505 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
506
507         Reviewed by Oliver Hunt.
508
509         Bug 20616: Incorporate V8 benchmarks in testing
510         <https://bugs.webkit.org/show_bug.cgi?id=20616>
511
512         This is a first attempt at running the V8 benchmarks within SunSpider,
513         with the --v8 option. The number of iterations is fixed for each
514         benchmark, and was chosen to make each of the benchmarks run between
515         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
516
517         * sunspider:
518         * tests/LIST-V8: Added.
519         * tests/v8-crypto.js: Added.
520         * tests/v8-deltablue.js: Added.
521         * tests/v8-earley-boyer.js: Added.
522         * tests/v8-raytrace.js: Added.
523         * tests/v8-richards.js: Added.
524
525 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
526
527         Not reviewed.
528
529         Remove a local change that was accidentally committed in r36034.
530
531         * resources/sunspider-standalone-driver.js:
532
533 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
534
535         Rubber-stamped by Oliver Hunt.
536
537         Make SunSpider work with JavaScript shells that don't take filenames
538         after a '-f' argument.
539
540         * sunspider:
541
542 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
543
544         Reviewed by Timothy.
545
546         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
547
548         * sunspider:
549         * tests/LIST-SQUIRRELFISH: Removed.
550
551 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
552
553         Rubber-stamped by Mark Rowe.
554
555         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
556
557         * sunspider:
558
559 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
560
561         Rubber stamped by Oliver.
562         
563         - add newly working tests to squirrelfish list
564
565         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
566
567 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
568
569         Rubber stamped by Oliver.
570         
571         - move ubench tests properly.
572
573         * tests/ubench: Added.
574         * tests/ubench/function-closure.js: Added.
575         * tests/ubench/function-empty.js: Added.
576         * tests/ubench/function-missing-args.js: Added.
577         * tests/ubench/function-sum.js: Added.
578         * tests/ubench/loop-empty-resolve.js: Added.
579         * tests/ubench/loop-empty.js: Added.
580         * tests/ubench/loop-sum.js: Added.
581
582 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
583
584         Rubber stamped by Oliver.
585
586         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
587
588 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
589
590         Reviewed by Oliver.
591
592         - make --squirrelfish mode use the real harness, but a reduced set of tests
593
594         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
595
596         * sunspider:
597         * tests/LIST-SQUIRRELFISH:
598         * tests/LIST-UBENCH: Added.
599         * tests/squirrelfish: Moved to tests/ubench
600
601 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
602
603         Reviewed by Maciej Stachowiak.
604         
605         Removed an unused test.
606
607         * tests/squirrelfish/loop-resolve.js: Removed.
608
609 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
610
611         Reviewed by Oliver.
612         
613         Add newly runnable tests to --squirrelfish mode.
614
615         * tests/LIST-SQUIRRELFISH:
616
617 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
618
619         Reviewed by Maciej Stachowiak.
620         
621         A little more love for --squirrelfish mode:
622         
623         Fixed a misplaced sort. Results properly sort now.
624         
625         Added a test list just for squirrelfish, and updated it to include
626         all currently passing SunSpider tests.
627         
628         Fixed the pruning regexp to match 3d-morph.js.
629
630         * sunspider:
631
632 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
633
634         Reviewed by Oliver Hunt.
635         
636         --squirrelfish mode: pared down tests for squirrelfish to chew on.
637
638         * sunspider:
639         * tests/LIST:
640         * tests/squirrelfish: Added.
641         * tests/squirrelfish/function-closure.js: Added.
642         * tests/squirrelfish/function-empty.js: Added.
643         * tests/squirrelfish/function-missing-args.js: Added.
644         * tests/squirrelfish/function-sum.js: Added.
645         * tests/squirrelfish/loop-empty-resolve.js: Added.
646         * tests/squirrelfish/loop-empty.js: Added.
647         * tests/squirrelfish/loop-resolve.js: Added.
648         * tests/squirrelfish/loop-sum.js: Added.
649
650 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
651
652         Reviewed by Geoff.
653
654         * sunspider: Pause update daemon on Darwin to make results reliable
655         again (workaround for <rdar://problem/5811127>).
656
657 2008-02-07  Eric Seidel  <eric@webkit.org>
658
659         Reviewed by Sam.
660
661         * sunspider: add an error message when ./tests/LIST is not found
662
663 2007-12-29  Alp Toker  <alp@atoker.com>
664
665         Fix typo.
666
667         * hosted/sunspider.html:
668
669 2007-12-29  Alp Toker  <alp@atoker.com>
670
671         Reviewed by Maciej.
672
673         Provide the current URL in the SunSpider results page. Useful for
674         copying benchmark results out of browsers without a location bar or
675         where the location bar can't handle long URLs.
676
677         * hosted/sunspider-results.html:
678
679 2007-12-20  Eric Seidel  <eric@webkit.org>
680
681         Reviewed by mjs.
682
683         * sunspider: increase default number of runs for more accurate numbers
684
685 2007-12-18  Mark Rowe  <mrowe@apple.com>
686
687         Fix typo.
688
689         * hosted/sunspider.html:
690
691 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
692
693         Reviewed by Eric.
694         
695         - give an extra digit of precision for small differences.
696
697         * resources/sunspider-compare-results.js:
698
699 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
700
701         Reviewed by Eric.
702         
703         - work around IE layout bugs
704
705         * hosted/sunspider.css:
706
707 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
708
709         Reviewed by Mark.
710         
711         - fix compare feature
712
713         * hosted/sunspider-results.html:
714
715 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
716
717         Reviewed by Mark.
718         
719         - Change the format of the output URL to be shorter, to meet IE constraints
720
721         * hosted/sunspider-driver.html:
722         * hosted/sunspider-results.html:
723
724 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
725
726         Reviewed by Oliver.
727         
728         - Add some explanatory test and a dumb logo to SunSpider web hosted version
729
730         * hosted/sunspider-driver.html:
731         * hosted/sunspider-results.html:
732         * hosted/sunspider.css:
733         * hosted/sunspider.html:
734
735 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
736
737         Reviewed by Oliver.
738         
739         - Make the web-hosted version of SunSpider look a little bit less drab.
740
741         * hosted/sunspider-driver.html:
742         * hosted/sunspider-results.html:
743         * hosted/sunspider.css: Added.
744         * hosted/sunspider.html:
745         * resources/TEMPLATE.html:
746
747 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
748
749         Reviewed by Sam.
750         
751         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
752         
753         A sample of what new-style results would look like:
754         
755         TEST                   COMPARISON            FROM                 TO             DETAILS
756         ...
757         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
758         ...
759             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
760         
761         * resources/sunspider-compare-results.js:
762
763 2007-11-14  Eric Seidel  <eric@webkit.org>
764
765         Reviewed by Sam.
766
767         * sunspider: add --shark-cache for L2 Cache Miss profiling
768
769 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
770
771         Reviewed by Darin Adler.
772
773         http://bugs.webkit.org/show_bug.cgi?id=15925
774         SunSpider should check for Shark being installed
775
776         * sunspider: Explicitly test that shark command line tool is installed.
777
778 2007-11-04  Darin Adler  <darin@apple.com>
779
780         * hosted/sunspider.html: Tweaked formatting a bit.
781
782 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
783
784         Reviewed by Darin Adler.
785
786         - Add compare mode and a description of the benchmark to the hosted version.
787
788         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
789         * hosted/sunspider.html: Added intro text.
790         * make-hosted: Copy new file.
791         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
792         a function.
793         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
794         expects.
795         * sunspider-compare-results: Use sunspider-standalone-compare.js.
796
797 2007-11-02  Darin Adler  <darin@apple.com>
798
799         * sunspider: Changed "--base" to "--set-baseline".
800
801 2007-11-02  Darin Adler  <darin@apple.com>
802
803         Reviewed by Maciej.
804
805         * sunspider: Added a "--base" option that records the current test as a baseline.
806         Moved the code to find the newest .mshark file into a function (since I needed to use
807         it in the other script and I prefer to copy/paste a function).
808
809         * sunspider-compare-results: Added logic to compare the baseline with the last
810         run when you don't pass any parameters.
811
812 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
813
814         Reviewed by Eric.
815         
816         - Added list of ECMAScript3 features that SunSpider does not cover currently.
817
818         * UNCOVERED: Added.
819
820 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
821
822         Reviewed by Oliver.
823         
824         - Give better error messages when sunspider-compare-results fails
825
826         * sunspider-compare-results:
827
828 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
829
830         Reviewed by Darin Adler.
831         
832         - fix Date tests to work in command-line SpiderMonkey
833
834         * tests/date-format-tofte.js:
835         * tests/date-format-xparb.js:
836
837 2007-10-29  Darin Adler  <darin@apple.com>
838
839         Reviewed by Maciej.
840
841         * sunspider: Added --shark20 option, to run Shark at its highest sample
842         resolution instead of its default.
843
844 2007-10-28  Adam Roben  <aroben@apple.com>
845
846         Fix SunSpider on Windows
847
848         Reviewed by Eric.
849
850         * sunspider: Don't use colons in filenames.
851
852 2007-10-28  Darin Adler  <darin@apple.com>
853
854         Reviewed by Adam.
855
856         * resources/sunspider-analyze-results.js: Tweak the output format a little.
857         Change so that when there's only one run we don't write out confidence
858         intervals at all rather than writing "NaN" over and over again.
859
860         * sunspider: Use the time and date as a suffix on the results file, that way
861         all the results are left behind in the tmp directory. This will make it easy
862         for us to add features that compare past results in the future. Also rename
863         the .mshark files using the same scheme.
864
865         * sunspider-compare-results: Relax the parsing rules so we can parse the
866         results file exactly as it's emitted from sunspider.
867
868 2007-10-25  Adam Roben  <aroben@apple.com>
869
870         Strip carriage returns from results
871
872         Carriage returns were screwing up sunspider-compare-results on
873         Windows.
874
875         Rubberstamped by Oliver.
876
877         * sunspider:
878
879 2007-10-24  Darin Adler  <darin@apple.com>
880
881         Reviewed by Eric.
882
883         * sunspider: Open the profile in shark after completing a run.
884         * sunspider-compare-results: Tiny perl tweaks.
885
886 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
887
888         Reviewed by Darin Adler.
889
890         - Don't hardcode my path to testkjs
891         
892         * sunspider-compare-results:
893
894 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
895
896         Reviewed by Darin Adler.
897         
898         - Add a compare script to compare results, and improve formatting of standard results a bit.
899
900         * resources/sunspider-analyze-results.js:
901         * resources/sunspider-compare-results.js: Added.
902         * sunspider-compare-results: Added.
903
904 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
905
906         Reviewed by Darin Adler.
907
908         * resources/sunspider-analyze-results.js: Do the error range
909         properly, using the t-distribution instead of 1.96 (which was
910         based on the normal distribution).
911         
912         * sunspider: Print results in a way that will be friendlier to the
913         soon-to-come compare mode.
914
915 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
916
917         Reviewed by Review.
918         
919         - add more tests (probably the final set for now)
920         
921         Already balanced. Added date, regexp, control flow, and a few
922         more string and object/array access tests.
923         
924         * tests/LIST:
925         * tests/access-fannkuch.js: Added.
926         * tests/access-nbody.js: Added.
927         * tests/controlflow-recursive.js: Added.
928         * tests/date-format-tofte.js: Added.
929         * tests/date-format-xparb.js: Added.
930         * tests/regexp-dna.js: Added.
931         * tests/string-validate-input.js: Added.
932
933 2007-10-21  Eric Seidel  <eric@webkit.org>
934
935         Reviewed by mjs.
936
937         * sunspider: make --shark only sample from testkjs
938
939 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
940
941         Rubber stamped by Eric.
942         
943         - added crypto tests (already balanced)
944
945         * tests/LIST:
946         * tests/crypto-aes.js: Added.
947         * tests/crypto-md5.js: Added.
948         * tests/crypto-sha1.js: Added.
949
950 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
951
952         Reviewed by Adam.
953         
954         - rebalance new tests
955         
956         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
957         
958         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
959
960         This balances them with the existing tests.
961
962         * tests/access-binary-trees.js:
963         * tests/access-nsieve.js:
964         * tests/bitops-nsieve-bits.js:
965         * tests/math-partial-sums.js:
966         * tests/math-spectral-norm.js:
967         * tests/string-fasta.js:
968
969 2007-10-20  Eric Seidel  <eric@webkit.org>
970
971         Reviewed by darin.
972         
973         Add improved argument handling to sunspider, including
974         --runs=<number>, --tests=<pattern, --shark, and --help
975         Also re-factor code into subroutines
976
977         * sunspider: improved argument handling
978
979 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
980
981         Reviewed by Mark.
982
983         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
984         
985         * TODO:
986         * tests/LIST:
987         * tests/access-binary-trees.js: Added.
988         * tests/access-nsieve.js: Added.
989         * tests/bitops-nsieve-bits.js: Added.
990         * tests/math-partial-sums.js: Added.
991         * tests/math-spectral-norm.js: Added.
992         * tests/string-fasta.js: Added.
993
994 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
995
996         Reviewed by Darin Adler.
997
998         * TODO: Updated to note areas that are now well-covered.
999         
1000         - Change the tests and the driver to avoid leaving large object
1001         graphs hanging around, since that throws off the subsequent tests.
1002         
1003         * resources/sunspider-standalone-driver.js:
1004         * tests/3d-cube.js:
1005         * tests/3d-morph.js:
1006         * tests/3d-raytrace.js:
1007         * tests/bitops-bitwise-and.js:
1008         * tests/string-base64.js:
1009         * tests/string-tagcloud.js:
1010
1011 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1012
1013         Reviewed by Nikolas.
1014         
1015         - Rebalance test complexity.
1016         
1017         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1018         
1019         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1020
1021         However, for some tests, IE7 was a huge outlier compared to other
1022         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1023         same OS and hardware as a normalization basis.
1024         
1025         * tests/3d-morph.js:
1026         * tests/3d-raytrace.js:
1027         * tests/bitops-3bit-bits-in-byte.js:
1028         * tests/bitops-bits-in-byte.js:
1029         * tests/bitops-bitwise-and.js:
1030         * tests/math-cordic.js:
1031         * tests/string-base64.js:
1032         * tests/string-tagcloud.js:
1033         * tests/string-unpack-code.js:
1034
1035 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1036
1037         Reviewed by Oliver.
1038         
1039         - Fix the browser-hosted driver to be compatible with Safari 2
1040
1041         * hosted/sunspider-driver.html:
1042
1043 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1044
1045         Rubber stamped by Hyatt.
1046         
1047         - Add BSD license notice to cordic test, the original author said to do so.
1048
1049         * tests/math-cordic.js:
1050
1051 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1052
1053         Reviewed by Eric.
1054         
1055         - Support Windows and Windows browsers.
1056
1057         * hosted/sunspider-results.html: Display results correctly in IE.
1058         * sunspider: tweaks to work on Windows, and with spaces in the path.
1059
1060 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1061
1062         Rubber stamped by Oliver.
1063
1064         * tests/string-tagcloud.js: Remove stray print()
1065
1066 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1067
1068         Reviewed by Oliver.
1069         
1070         Add some string test cases.
1071
1072         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1073         * tests/string-tagcloud.js: Added. Parses a JSON string and
1074         generates tagcloud HTML markup based on the data.        
1075         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1076         four of the most popular JavaScript libraries.
1077         * tests/LIST: Added the new tests.
1078
1079 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1080
1081         Reviewed by Anders.
1082
1083         * TODO: Updated with info on how to do the stats properly.
1084
1085         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1086         avoid overly slowing down the benchmark.
1087
1088         Added some 3D-related test cases.
1089         
1090         * tests/3d-cube.js: Added.
1091         * tests/3d-morph.js: Added.
1092         * tests/3d-raytrace.js: Added.
1093         * tests/LIST:
1094
1095 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1096
1097         Reviewed by Darin Adler.
1098
1099         - New JavaScript benchmark
1100         http://bugs.webkit.org/show_bug.cgi?id=15515
1101         
1102         A start on a new JavaScript benchmark, with standalone and
1103         browser-hosted drivers.
1104         
1105         The test content is still incomplete and the drivers could use
1106         some more features but there is enough here to make a good start.
1107         
1108         The drivers compute a 95% confidence interval on the mean for the
1109         whole test, each category, and each individual test to make it
1110         easier to tell whether differences are statistically
1111         significant. The confidence interval can be narrowed by running
1112         the test more times.
1113         
1114         * TODO: Added. Remaining things that need to be done.
1115         * sunspider: Added. Perl script that acts as the standalone test driver.
1116         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1117         of standalone test driver.
1118         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1119         analysis code.
1120         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1121         * make-hosted: Added. Script to generate browser-hosted tests.
1122         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1123         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1124         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1125         tests.
1126         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1127         tests.
1128         * tests/LIST: Added. List of tests to use.
1129         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1130         * tests/bitops-bits-in-byte.js: Added. Ditto.
1131         * tests/bitops-bitwise-and.js: Added. Ditto.
1132         * tests/math-cordic.js: Added. Ditto.
1133