2011-07-02 Maciej Stachowiak <mjs@apple.com>
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
2
3         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
4         https://bugs.webkit.org/show_bug.cgi?id=60937
5
6         Reviewed by Dan Bernstein.
7
8         * tests/sunspider-1.0/string-validate-input.js:
9         (doTest): Rename the variable to username
10
11 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
12
13         Reviewed by Darin Adler.
14
15         SunSpider: The main function of math-cordic is dead code
16         https://bugs.webkit.org/show_bug.cgi?id=63863
17         
18         Try to prevent math-cordic from being eliminated as dead code or
19         compiled down to a constant.
20
21         * tests/sunspider-1.0/math-cordic.js:
22         (cordicsincos):
23         (cordic):
24
25 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
26
27         Reviewed by Adam Barth.
28
29         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
30         https://bugs.webkit.org/show_bug.cgi?id=63782
31
32         * make-hosted: Learn about the new directory.
33         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
34
35 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
36
37         Reviewed by Gavin Barraclough.
38
39         Improved accuracy of command-line SunSpider.
40
41         * resources/sunspider-standalone-driver.js: Use the returned number of
42         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
43         ourselves, for slightly more accurate numbers.
44
45 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
46
47         Reviewed by Mark Rowe.
48         https://bugs.webkit.org/show_bug.cgi?id=45924
49
50         Modify the SunSpider harness to allow a path to be passed to the --suite
51         argument, supress the normal prepend-"tests/" behaviour for suite names
52         containing a '/'.
53
54         Also wrap the code in an anonymous function to move variables out of global
55         scope (currently the harness cannot handle running tests the overwrite the
56         variable 'j').
57
58         * resources/sunspider-standalone-driver.js:
59         * sunspider:
60
61 2010-08-27  Michael Saboff  <msaboff@apple.com>
62
63         Reviewed by Stephanie Lewis.
64
65         Added version 5 and version 6 of V8 tests with modifications similar
66         to the version 4 tests so they'll work with the SunSpider test harness.
67         Note that these files are external source and therefore don't conform 
68         to WebKit coding standards.
69         https://bugs.webkit.org/show_bug.cgi?id=44638
70
71         * tests/v8-v5: Added.
72         * tests/v8-v5/LIST: Added.
73         * tests/v8-v5/v8-crypto.js: Added.
74         * tests/v8-v5/v8-deltablue.js: Added.
75         * tests/v8-v5/v8-earley-boyer.js: Added.
76         * tests/v8-v5/v8-raytrace.js: Added.
77         * tests/v8-v5/v8-regexp.js: Added.
78         * tests/v8-v5/v8-richards.js: Added.
79         * tests/v8-v5/v8-splay.js: Added.
80         * tests/v8-v6: Added.
81         * tests/v8-v6/LIST: Added.
82         * tests/v8-v6/v8-crypto.js: Added.
83         * tests/v8-v6/v8-deltablue.js: Added.
84         * tests/v8-v6/v8-earley-boyer.js: Added.
85         * tests/v8-v6/v8-raytrace.js: Added.
86         * tests/v8-v6/v8-regexp.js: Added.
87         * tests/v8-v6/v8-richards.js: Added.
88         * tests/v8-v6/v8-splay.js: Added.
89
90 2010-07-15  Stephanie Lewis  <slewis@apple.com>
91
92         Reviewed by Geoff Garen.
93
94         http://bugs.webkit.org/show_bug.cgi?id=42406
95         Output location of results file.
96
97         * sunspider:
98
99 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
100
101         Reviewed by Mark Rowe.
102
103         Browser-hosted version of SunSpider runs tests twice as many times as advertised
104         https://bugs.webkit.org/show_bug.cgi?id=39201
105
106         * make-hosted: Make sure to reset test list after every test suite.
107
108 2010-05-07  Adam Roben  <aroben@apple.com>
109
110         Fix typo in sunspider-compare-results.js that caused incorrect
111         two-sample t statistics to be computed
112
113         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
114         two-sample t statistics
115
116         Reviewed by Darin Adler.
117
118         * resources/sunspider-compare-results.js:
119         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
120         variable to match the definition of a two-sample t statistic. Looks
121         like this was just a simple typo.
122
123 2010-04-05  Darin Adler  <darin@apple.com>
124
125         Updated the ignore property for changes to the contents here.
126
127         * .: Modified property svn:ignore.
128
129 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
130
131         Reviewed by Adam Barth.
132
133         Add sunspider-0.9.1 to the site in preparation for announcing it.
134         https://bugs.webkit.org/show_bug.cgi?id=35206
135
136         * hosted/sunspider.css: Remove tabs, which I noticed where present while
137         copying to the WebKitSite directory.
138
139 2010-01-11  Oliver Hunt  <oliver@apple.com>
140
141         Reviewed by Darin Adler.
142
143         Sunspider parse-only tests don't run
144         https://bugs.webkit.org/show_bug.cgi?id=33489
145
146         Fix parse-only test list
147
148         * tests/parse-only/LIST:
149
150 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
151
152         Reviewed by Adele Peterson.
153
154         Make SunSpider version more prominent in the title
155         https://bugs.webkit.org/show_bug.cgi?id=32574
156
157         * hosted/sunspider.html:
158         * hosted/versions.html:
159         * resources/driver-TEMPLATE.html:
160         * resources/results-TEMPLATE.html:
161
162 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
163
164         Reviewed by Darin Adler.
165
166         Make SunSpider refuse to compare results across content versions
167         https://bugs.webkit.org/show_bug.cgi?id=32573
168         
169         The results URL now includes a version indicator, and compare mode
170         will refuse to compare between versions.
171
172         * resources/driver-TEMPLATE.html:
173         * resources/results-TEMPLATE.html:
174
175 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
176
177         Reviewed by Darin Adler.
178
179         Some Browser-hosted SunSpider files are not valid HTML5
180         https://bugs.webkit.org/show_bug.cgi?id=32536
181         
182         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
183         close tags and adding a meta charset declaration).
184
185         * hosted/sunspider.html:
186         * hosted/versions.html:
187         * resources/TEMPLATE.html:
188         * resources/driver-TEMPLATE.html:
189         * resources/results-TEMPLATE.html:
190
191 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
192
193         Reviewed by Eric Seidel.
194
195         Make sunspider-0.9.1 the default content set (both command-line and hosted)
196         https://bugs.webkit.org/show_bug.cgi?id=32537
197
198         * hosted/sunspider.html: Change default suite link.
199         * sunspider: Change default suite.
200         * sunspider-compare-results: Likewise.
201
202 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
203
204         Reviewed by Sam Weinig.
205
206         in-browser SunSpider suffers excessive penalty under power management
207         https://bugs.webkit.org/show_bug.cgi?id=32505
208         
209         I have made a few changes to address this:
210         
211         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
212         document.write to reduce triggering of progress UI.
213         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
214         stepped all the way down the whole time.
215         3) Run the test cycle an extra time for warmup before the runs that count.
216
217         * hosted/sunspider-record-result.js: Removed.
218         * make-hosted:
219         * resources/TEMPLATE.html:
220         * resources/driver-TEMPLATE.html:
221
222 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
223
224         Reviewed by Gavin Barraclough.
225
226         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
227         https://bugs.webkit.org/show_bug.cgi?id=16806
228
229         Based on a patch by Eric Seidel.
230         
231         Fix the base64 computation to actually compute correct results. The impact on runtime of
232         the test is pretty small, but noticeable for some browsers. But at least it's not
233         doing a wrong and meaningless computation any more.
234         
235         * tests/sunspider-0.9.1/string-base64.js:
236         ():
237         (base64ToString):
238
239 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
240
241         Fixing commit error...
242         
243         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
244
245         * resources/driver-TEMPLATE.html:
246
247 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
248
249         Reviewed by Sam Weinig.
250
251         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
252         https://bugs.webkit.org/show_bug.cgi?id=32490
253
254         * hosted/json2.js: Added. Incorporated from json.org
255         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
256         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
257         results.
258
259 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
260
261         Reviewed by Oliver Hunt.
262
263         Enable Web-hosted version of SunSpider to handle multiple versions
264         https://bugs.webkit.org/show_bug.cgi?id=32478
265
266         * make-hosted: Now generate the hosted version to be able to run both the
267         0.9 and the 0.9.1 test suites.
268         * hosted: Modified property svn:ignore.
269         * hosted/sunspider-driver.html: Removed.
270         * hosted/sunspider-results.html: Removed.
271         * hosted/sunspider.html:
272         * hosted/versions.html: Added.
273         * resources/TEMPLATE.html:
274         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
275         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
276
277 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
278
279         Reviewed by Oliver Hunt.
280
281         Give command-line sunspider the ability to handle multiple suites and versions
282         https://bugs.webkit.org/show_bug.cgi?id=32477
283         
284         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
285         enumerating the tests. Current suites are:
286         
287         sunspider-0.9 - riginal SunSpider content 
288         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
289         ubench - "ubench" microbenchmark suite
290         v8-v4 - version 4 of the v8 benchmark
291         parse-only - some parser tests
292         
293         Sunspider can now be run with the --suite= parameter to select a
294         suite. The default is --suite=sunspider-0.9.
295
296         * .: Modified property svn:ignore to ignore new results directories.
297         * make-hosted: Generate based on sunspider-0.9 suite.
298         * resources/TEMPLATE.html: Fix an HTML compliance problem.
299         * resources/sunspider-standalone-driver.js: Become aware of suite names.
300         * sunspider: Modified to handle multiple suites as described above.
301         * sunspider-compare-results: ditto
302         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
303         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
304         * tests/LIST-UBENCH: Removed.
305         * tests/LIST-V8: Removed.
306         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
307         * tests/3d-morph.js: ditto
308         * tests/3d-raytrace.js: ditto
309         * tests/access-binary-trees.js: ditto
310         * tests/access-fannkuch.js: ditto
311         * tests/access-nbody.js: ditto
312         * tests/access-nsieve.js: ditto
313         * tests/bitops-3bit-bits-in-byte.js: ditto
314         * tests/bitops-bits-in-byte.js: ditto
315         * tests/bitops-bitwise-and.js: ditto
316         * tests/bitops-nsieve-bits.js: ditto
317         * tests/controlflow-recursive.js: ditto
318         * tests/crypto-aes.js: ditto
319         * tests/crypto-md5.js: ditto
320         * tests/crypto-sha1.js: ditto
321         * tests/date-format-tofte.js: ditto
322         * tests/date-format-xparb.js: ditto
323         * tests/math-cordic.js: ditto
324         * tests/math-partial-sums.js: ditto
325         * tests/math-spectral-norm.js: ditto
326         * tests/regexp-dna.js: ditto
327         * tests/string-base64.js: ditto
328         * tests/string-fasta.js: ditto
329         * tests/string-tagcloud.js: ditto
330         * tests/string-unpack-code.js: ditto
331         * tests/string-validate-input.js: ditto
332         * tests/sunspider-0.9/: Added
333         * tests/sunspider-0.9/*.js: Moved from one level up
334         * tests/sunspider-0.9.1: Added.
335         * tests/sunspider-0.9.1/*.js: Moved from one level up.
336         * tests/v8-crypto.js: Moved to v8-v4/ directory.
337         * tests/v8-deltablue.js: ditto
338         * tests/v8-earley-boyer.js: ditto
339         * tests/v8-raytrace.js: ditto
340         * tests/v8-regexp.js: ditto
341         * tests/v8-richards.js: ditto
342         * tests/v8-splay.js: ditto
343         * tests/v8-v4: Added.
344
345 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
346
347         Reviewed by Darin Adler.
348
349         Platform specific null device replaced with a platform independent.
350         https://bugs.webkit.org/show_bug.cgi?id=29544
351
352         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
353
354 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
355
356         Reviewed by Maciej Stachowiak.
357
358         [Qt] Make sunspider script work on Windows platform.
359         https://bugs.webkit.org/show_bug.cgi?id=29544
360
361         * sunspider:
362         Using platform specific null device instead of /dev/null.
363
364 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
365
366         Reviewed by Oliver Hunt.
367
368         https://bugs.webkit.org/show_bug.cgi?id=26540
369         Currently the SunSpider test driver lacks an option to run a test suite that
370         will test JavaScriptCore parsing performance only.  This patch adds just such
371         a test suite and option to SunSpider as well as the jsc test shell.  I've included
372         three large javascript source files found in the wild: jquery, mootools and prototype.
373         Combined with the concatenation of all three, these form a new testsuite to measure
374         and test pure JavaScriptCore parsing performance.
375
376         * resources/sunspider-standalone-driver.js:
377           Load the js with parse-only flag if it is part of the parse-only suite.
378         * sunspider:
379           Add the parse-only suite to the list of options.
380         * sunspider-compare-results:
381           Add the parse-only suite to the list of options.
382         * tests/LIST-PARSE-ONLY: Added.
383         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
384           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
385         * tests/parse-only/jquery-1.3.2.js: Added.
386           Latest version from jquery.com.
387         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
388           Latest version from mootools.net.
389         * tests/parse-only/prototype-1.6.0.3.js: Added.
390           Latest version from prototypejs.org.
391
392 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
393
394         Update V8 benchmark to version 4.
395         
396         Reviewed by Cameron Zwarich.
397
398         * tests/LIST-V8:
399         * tests/v8-crypto.js:
400         * tests/v8-deltablue.js:
401         * tests/v8-earley-boyer.js:
402         * tests/v8-raytrace.js:
403         * tests/v8-regexp.js: Added.
404         * tests/v8-richards.js:
405         * tests/v8-splay.js: Added.
406
407 2009-05-26  Olivier DOLE  <odole@pleyo.com>
408
409         Reviewed by Maciej.
410
411         Store sunspider test suite results in a specific directory for each
412         different type of benchmark (sunspider, V8 or ubench).
413
414         * sunspider:
415         * sunspider-compare-results:
416
417 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
418
419         Build fix.
420
421         * SunSpider.make:
422
423 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
424
425         Include SunSpider tests in production builds.
426         
427         Reviewed by Adam Roben.
428
429         * SunSpider.make: Added.
430         * xcopy.excludes: Added.
431
432 2009-03-04  Oliver Hunt  <oliver@apple.com>
433
434         Added a few more tests to ubench
435
436         RS=Gavin Barraclough
437
438         Adding a couple of test to cover too few and too many arguments.  This
439         is needed to prevent regressions in all call paths from my current refactoring.
440
441         * tests/LIST-UBENCH:
442         * tests/ubench/function-correct-args.js: Added.
443         (f):
444         * tests/ubench/function-excess-args.js: Added.
445         (f):
446
447 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
448
449         Fix a bug where when V8's or UBENCH's test suite list file is missing
450         the error message is suggesting that SunSpider's list file is missing.
451
452         Reviewed by Sam Weinig.
453
454         * sunspider:
455         (loadTestsList): Fix the error message to use the right list name.
456
457 2008-10-07  David Hyatt  <hyatt@apple.com>
458
459         https://bugs.webkit.org/show_bug.cgi?id=18989
460
461         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
462         we want all browsers to be doing the same thing on the test.
463
464         Reviewed by Maciej
465
466         * tests/regexp-dna.js:
467
468 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
469
470         Rubber-stamped by Maciej Stachowiak.
471
472         The bleeding edge branch of V8 now supports passing fileanmes after
473         '-f', so the SunSpider script should be changed to do this again.
474         This allows all 3 major JavaScript engines to be used with the
475         unmodified SunSpider script.
476
477         * sunspider:
478
479 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
480
481         Reviewed by Maciej Stachowiak.
482
483         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
484         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
485         correct the punctuation in its documentation.
486
487         * sunspider:
488
489 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
490
491         Reviewed by Oliver Hunt.
492
493         Bug 20616: Incorporate V8 benchmarks in testing
494         <https://bugs.webkit.org/show_bug.cgi?id=20616>
495
496         This is a first attempt at running the V8 benchmarks within SunSpider,
497         with the --v8 option. The number of iterations is fixed for each
498         benchmark, and was chosen to make each of the benchmarks run between
499         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
500
501         * sunspider:
502         * tests/LIST-V8: Added.
503         * tests/v8-crypto.js: Added.
504         * tests/v8-deltablue.js: Added.
505         * tests/v8-earley-boyer.js: Added.
506         * tests/v8-raytrace.js: Added.
507         * tests/v8-richards.js: Added.
508
509 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
510
511         Not reviewed.
512
513         Remove a local change that was accidentally committed in r36034.
514
515         * resources/sunspider-standalone-driver.js:
516
517 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
518
519         Rubber-stamped by Oliver Hunt.
520
521         Make SunSpider work with JavaScript shells that don't take filenames
522         after a '-f' argument.
523
524         * sunspider:
525
526 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
527
528         Reviewed by Timothy.
529
530         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
531
532         * sunspider:
533         * tests/LIST-SQUIRRELFISH: Removed.
534
535 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
536
537         Rubber-stamped by Mark Rowe.
538
539         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
540
541         * sunspider:
542
543 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
544
545         Rubber stamped by Oliver.
546         
547         - add newly working tests to squirrelfish list
548
549         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
550
551 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
552
553         Rubber stamped by Oliver.
554         
555         - move ubench tests properly.
556
557         * tests/ubench: Added.
558         * tests/ubench/function-closure.js: Added.
559         * tests/ubench/function-empty.js: Added.
560         * tests/ubench/function-missing-args.js: Added.
561         * tests/ubench/function-sum.js: Added.
562         * tests/ubench/loop-empty-resolve.js: Added.
563         * tests/ubench/loop-empty.js: Added.
564         * tests/ubench/loop-sum.js: Added.
565
566 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
567
568         Rubber stamped by Oliver.
569
570         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
571
572 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
573
574         Reviewed by Oliver.
575
576         - make --squirrelfish mode use the real harness, but a reduced set of tests
577
578         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
579
580         * sunspider:
581         * tests/LIST-SQUIRRELFISH:
582         * tests/LIST-UBENCH: Added.
583         * tests/squirrelfish: Moved to tests/ubench
584
585 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
586
587         Reviewed by Maciej Stachowiak.
588         
589         Removed an unused test.
590
591         * tests/squirrelfish/loop-resolve.js: Removed.
592
593 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
594
595         Reviewed by Oliver.
596         
597         Add newly runnable tests to --squirrelfish mode.
598
599         * tests/LIST-SQUIRRELFISH:
600
601 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
602
603         Reviewed by Maciej Stachowiak.
604         
605         A little more love for --squirrelfish mode:
606         
607         Fixed a misplaced sort. Results properly sort now.
608         
609         Added a test list just for squirrelfish, and updated it to include
610         all currently passing SunSpider tests.
611         
612         Fixed the pruning regexp to match 3d-morph.js.
613
614         * sunspider:
615
616 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
617
618         Reviewed by Oliver Hunt.
619         
620         --squirrelfish mode: pared down tests for squirrelfish to chew on.
621
622         * sunspider:
623         * tests/LIST:
624         * tests/squirrelfish: Added.
625         * tests/squirrelfish/function-closure.js: Added.
626         * tests/squirrelfish/function-empty.js: Added.
627         * tests/squirrelfish/function-missing-args.js: Added.
628         * tests/squirrelfish/function-sum.js: Added.
629         * tests/squirrelfish/loop-empty-resolve.js: Added.
630         * tests/squirrelfish/loop-empty.js: Added.
631         * tests/squirrelfish/loop-resolve.js: Added.
632         * tests/squirrelfish/loop-sum.js: Added.
633
634 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
635
636         Reviewed by Geoff.
637
638         * sunspider: Pause update daemon on Darwin to make results reliable
639         again (workaround for <rdar://problem/5811127>).
640
641 2008-02-07  Eric Seidel  <eric@webkit.org>
642
643         Reviewed by Sam.
644
645         * sunspider: add an error message when ./tests/LIST is not found
646
647 2007-12-29  Alp Toker  <alp@atoker.com>
648
649         Fix typo.
650
651         * hosted/sunspider.html:
652
653 2007-12-29  Alp Toker  <alp@atoker.com>
654
655         Reviewed by Maciej.
656
657         Provide the current URL in the SunSpider results page. Useful for
658         copying benchmark results out of browsers without a location bar or
659         where the location bar can't handle long URLs.
660
661         * hosted/sunspider-results.html:
662
663 2007-12-20  Eric Seidel  <eric@webkit.org>
664
665         Reviewed by mjs.
666
667         * sunspider: increase default number of runs for more accurate numbers
668
669 2007-12-18  Mark Rowe  <mrowe@apple.com>
670
671         Fix typo.
672
673         * hosted/sunspider.html:
674
675 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
676
677         Reviewed by Eric.
678         
679         - give an extra digit of precision for small differences.
680
681         * resources/sunspider-compare-results.js:
682
683 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
684
685         Reviewed by Eric.
686         
687         - work around IE layout bugs
688
689         * hosted/sunspider.css:
690
691 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
692
693         Reviewed by Mark.
694         
695         - fix compare feature
696
697         * hosted/sunspider-results.html:
698
699 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
700
701         Reviewed by Mark.
702         
703         - Change the format of the output URL to be shorter, to meet IE constraints
704
705         * hosted/sunspider-driver.html:
706         * hosted/sunspider-results.html:
707
708 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
709
710         Reviewed by Oliver.
711         
712         - Add some explanatory test and a dumb logo to SunSpider web hosted version
713
714         * hosted/sunspider-driver.html:
715         * hosted/sunspider-results.html:
716         * hosted/sunspider.css:
717         * hosted/sunspider.html:
718
719 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
720
721         Reviewed by Oliver.
722         
723         - Make the web-hosted version of SunSpider look a little bit less drab.
724
725         * hosted/sunspider-driver.html:
726         * hosted/sunspider-results.html:
727         * hosted/sunspider.css: Added.
728         * hosted/sunspider.html:
729         * resources/TEMPLATE.html:
730
731 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
732
733         Reviewed by Sam.
734         
735         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
736         
737         A sample of what new-style results would look like:
738         
739         TEST                   COMPARISON            FROM                 TO             DETAILS
740         ...
741         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
742         ...
743             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
744         
745         * resources/sunspider-compare-results.js:
746
747 2007-11-14  Eric Seidel  <eric@webkit.org>
748
749         Reviewed by Sam.
750
751         * sunspider: add --shark-cache for L2 Cache Miss profiling
752
753 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
754
755         Reviewed by Darin Adler.
756
757         http://bugs.webkit.org/show_bug.cgi?id=15925
758         SunSpider should check for Shark being installed
759
760         * sunspider: Explicitly test that shark command line tool is installed.
761
762 2007-11-04  Darin Adler  <darin@apple.com>
763
764         * hosted/sunspider.html: Tweaked formatting a bit.
765
766 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
767
768         Reviewed by Darin Adler.
769
770         - Add compare mode and a description of the benchmark to the hosted version.
771
772         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
773         * hosted/sunspider.html: Added intro text.
774         * make-hosted: Copy new file.
775         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
776         a function.
777         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
778         expects.
779         * sunspider-compare-results: Use sunspider-standalone-compare.js.
780
781 2007-11-02  Darin Adler  <darin@apple.com>
782
783         * sunspider: Changed "--base" to "--set-baseline".
784
785 2007-11-02  Darin Adler  <darin@apple.com>
786
787         Reviewed by Maciej.
788
789         * sunspider: Added a "--base" option that records the current test as a baseline.
790         Moved the code to find the newest .mshark file into a function (since I needed to use
791         it in the other script and I prefer to copy/paste a function).
792
793         * sunspider-compare-results: Added logic to compare the baseline with the last
794         run when you don't pass any parameters.
795
796 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
797
798         Reviewed by Eric.
799         
800         - Added list of ECMAScript3 features that SunSpider does not cover currently.
801
802         * UNCOVERED: Added.
803
804 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
805
806         Reviewed by Oliver.
807         
808         - Give better error messages when sunspider-compare-results fails
809
810         * sunspider-compare-results:
811
812 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
813
814         Reviewed by Darin Adler.
815         
816         - fix Date tests to work in command-line SpiderMonkey
817
818         * tests/date-format-tofte.js:
819         * tests/date-format-xparb.js:
820
821 2007-10-29  Darin Adler  <darin@apple.com>
822
823         Reviewed by Maciej.
824
825         * sunspider: Added --shark20 option, to run Shark at its highest sample
826         resolution instead of its default.
827
828 2007-10-28  Adam Roben  <aroben@apple.com>
829
830         Fix SunSpider on Windows
831
832         Reviewed by Eric.
833
834         * sunspider: Don't use colons in filenames.
835
836 2007-10-28  Darin Adler  <darin@apple.com>
837
838         Reviewed by Adam.
839
840         * resources/sunspider-analyze-results.js: Tweak the output format a little.
841         Change so that when there's only one run we don't write out confidence
842         intervals at all rather than writing "NaN" over and over again.
843
844         * sunspider: Use the time and date as a suffix on the results file, that way
845         all the results are left behind in the tmp directory. This will make it easy
846         for us to add features that compare past results in the future. Also rename
847         the .mshark files using the same scheme.
848
849         * sunspider-compare-results: Relax the parsing rules so we can parse the
850         results file exactly as it's emitted from sunspider.
851
852 2007-10-25  Adam Roben  <aroben@apple.com>
853
854         Strip carriage returns from results
855
856         Carriage returns were screwing up sunspider-compare-results on
857         Windows.
858
859         Rubberstamped by Oliver.
860
861         * sunspider:
862
863 2007-10-24  Darin Adler  <darin@apple.com>
864
865         Reviewed by Eric.
866
867         * sunspider: Open the profile in shark after completing a run.
868         * sunspider-compare-results: Tiny perl tweaks.
869
870 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
871
872         Reviewed by Darin Adler.
873
874         - Don't hardcode my path to testkjs
875         
876         * sunspider-compare-results:
877
878 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
879
880         Reviewed by Darin Adler.
881         
882         - Add a compare script to compare results, and improve formatting of standard results a bit.
883
884         * resources/sunspider-analyze-results.js:
885         * resources/sunspider-compare-results.js: Added.
886         * sunspider-compare-results: Added.
887
888 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
889
890         Reviewed by Darin Adler.
891
892         * resources/sunspider-analyze-results.js: Do the error range
893         properly, using the t-distribution instead of 1.96 (which was
894         based on the normal distribution).
895         
896         * sunspider: Print results in a way that will be friendlier to the
897         soon-to-come compare mode.
898
899 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
900
901         Reviewed by Review.
902         
903         - add more tests (probably the final set for now)
904         
905         Already balanced. Added date, regexp, control flow, and a few
906         more string and object/array access tests.
907         
908         * tests/LIST:
909         * tests/access-fannkuch.js: Added.
910         * tests/access-nbody.js: Added.
911         * tests/controlflow-recursive.js: Added.
912         * tests/date-format-tofte.js: Added.
913         * tests/date-format-xparb.js: Added.
914         * tests/regexp-dna.js: Added.
915         * tests/string-validate-input.js: Added.
916
917 2007-10-21  Eric Seidel  <eric@webkit.org>
918
919         Reviewed by mjs.
920
921         * sunspider: make --shark only sample from testkjs
922
923 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
924
925         Rubber stamped by Eric.
926         
927         - added crypto tests (already balanced)
928
929         * tests/LIST:
930         * tests/crypto-aes.js: Added.
931         * tests/crypto-md5.js: Added.
932         * tests/crypto-sha1.js: Added.
933
934 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
935
936         Reviewed by Adam.
937         
938         - rebalance new tests
939         
940         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
941         
942         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
943
944         This balances them with the existing tests.
945
946         * tests/access-binary-trees.js:
947         * tests/access-nsieve.js:
948         * tests/bitops-nsieve-bits.js:
949         * tests/math-partial-sums.js:
950         * tests/math-spectral-norm.js:
951         * tests/string-fasta.js:
952
953 2007-10-20  Eric Seidel  <eric@webkit.org>
954
955         Reviewed by darin.
956         
957         Add improved argument handling to sunspider, including
958         --runs=<number>, --tests=<pattern, --shark, and --help
959         Also re-factor code into subroutines
960
961         * sunspider: improved argument handling
962
963 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
964
965         Reviewed by Mark.
966
967         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
968         
969         * TODO:
970         * tests/LIST:
971         * tests/access-binary-trees.js: Added.
972         * tests/access-nsieve.js: Added.
973         * tests/bitops-nsieve-bits.js: Added.
974         * tests/math-partial-sums.js: Added.
975         * tests/math-spectral-norm.js: Added.
976         * tests/string-fasta.js: Added.
977
978 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
979
980         Reviewed by Darin Adler.
981
982         * TODO: Updated to note areas that are now well-covered.
983         
984         - Change the tests and the driver to avoid leaving large object
985         graphs hanging around, since that throws off the subsequent tests.
986         
987         * resources/sunspider-standalone-driver.js:
988         * tests/3d-cube.js:
989         * tests/3d-morph.js:
990         * tests/3d-raytrace.js:
991         * tests/bitops-bitwise-and.js:
992         * tests/string-base64.js:
993         * tests/string-tagcloud.js:
994
995 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
996
997         Reviewed by Nikolas.
998         
999         - Rebalance test complexity.
1000         
1001         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1002         
1003         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1004
1005         However, for some tests, IE7 was a huge outlier compared to other
1006         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1007         same OS and hardware as a normalization basis.
1008         
1009         * tests/3d-morph.js:
1010         * tests/3d-raytrace.js:
1011         * tests/bitops-3bit-bits-in-byte.js:
1012         * tests/bitops-bits-in-byte.js:
1013         * tests/bitops-bitwise-and.js:
1014         * tests/math-cordic.js:
1015         * tests/string-base64.js:
1016         * tests/string-tagcloud.js:
1017         * tests/string-unpack-code.js:
1018
1019 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1020
1021         Reviewed by Oliver.
1022         
1023         - Fix the browser-hosted driver to be compatible with Safari 2
1024
1025         * hosted/sunspider-driver.html:
1026
1027 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1028
1029         Rubber stamped by Hyatt.
1030         
1031         - Add BSD license notice to cordic test, the original author said to do so.
1032
1033         * tests/math-cordic.js:
1034
1035 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1036
1037         Reviewed by Eric.
1038         
1039         - Support Windows and Windows browsers.
1040
1041         * hosted/sunspider-results.html: Display results correctly in IE.
1042         * sunspider: tweaks to work on Windows, and with spaces in the path.
1043
1044 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1045
1046         Rubber stamped by Oliver.
1047
1048         * tests/string-tagcloud.js: Remove stray print()
1049
1050 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1051
1052         Reviewed by Oliver.
1053         
1054         Add some string test cases.
1055
1056         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1057         * tests/string-tagcloud.js: Added. Parses a JSON string and
1058         generates tagcloud HTML markup based on the data.        
1059         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1060         four of the most popular JavaScript libraries.
1061         * tests/LIST: Added the new tests.
1062
1063 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1064
1065         Reviewed by Anders.
1066
1067         * TODO: Updated with info on how to do the stats properly.
1068
1069         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1070         avoid overly slowing down the benchmark.
1071
1072         Added some 3D-related test cases.
1073         
1074         * tests/3d-cube.js: Added.
1075         * tests/3d-morph.js: Added.
1076         * tests/3d-raytrace.js: Added.
1077         * tests/LIST:
1078
1079 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1080
1081         Reviewed by Darin Adler.
1082
1083         - New JavaScript benchmark
1084         http://bugs.webkit.org/show_bug.cgi?id=15515
1085         
1086         A start on a new JavaScript benchmark, with standalone and
1087         browser-hosted drivers.
1088         
1089         The test content is still incomplete and the drivers could use
1090         some more features but there is enough here to make a good start.
1091         
1092         The drivers compute a 95% confidence interval on the mean for the
1093         whole test, each category, and each individual test to make it
1094         easier to tell whether differences are statistically
1095         significant. The confidence interval can be narrowed by running
1096         the test more times.
1097         
1098         * TODO: Added. Remaining things that need to be done.
1099         * sunspider: Added. Perl script that acts as the standalone test driver.
1100         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1101         of standalone test driver.
1102         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1103         analysis code.
1104         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1105         * make-hosted: Added. Script to generate browser-hosted tests.
1106         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1107         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1108         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1109         tests.
1110         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1111         tests.
1112         * tests/LIST: Added. List of tests to use.
1113         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1114         * tests/bitops-bits-in-byte.js: Added. Ditto.
1115         * tests/bitops-bitwise-and.js: Added. Ditto.
1116         * tests/math-cordic.js: Added. Ditto.
1117