Not Reviewed.
[WebKit-https.git] / PerformanceTests / SunSpider / ChangeLog
1 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
2
3         Not Reviewed. 
4         
5         Fix obviou typo in previous commit.
6
7         * tests/sunspider-1.0/bitops-bitwise-and.js:
8
9 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
10
11         Reviewed by Anders Carlsson.
12
13         SunSpider: all four bitops benchmarks can be replaced with NOP
14         https://bugs.webkit.org/show_bug.cgi?id=38446
15
16         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
17         (TimeFunc): Save the result.
18         * tests/sunspider-1.0/bitops-bits-in-byte.js:
19         (TimeFunc): Save the result.
20         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
21         save the results.
22         * tests/sunspider-1.0/bitops-nsieve-bits.js:
23         (sieve): Save the result.
24
25 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
26
27         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
28         https://bugs.webkit.org/show_bug.cgi?id=60937
29
30         Reviewed by Dan Bernstein.
31
32         * tests/sunspider-1.0/string-validate-input.js:
33         (doTest): Rename the variable to username
34
35 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
36
37         Reviewed by Darin Adler.
38
39         SunSpider: The main function of math-cordic is dead code
40         https://bugs.webkit.org/show_bug.cgi?id=63863
41         
42         Try to prevent math-cordic from being eliminated as dead code or
43         compiled down to a constant.
44
45         * tests/sunspider-1.0/math-cordic.js:
46         (cordicsincos):
47         (cordic):
48
49 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
50
51         Reviewed by Adam Barth.
52
53         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
54         https://bugs.webkit.org/show_bug.cgi?id=63782
55
56         * make-hosted: Learn about the new directory.
57         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
58
59 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
60
61         Reviewed by Gavin Barraclough.
62
63         Improved accuracy of command-line SunSpider.
64
65         * resources/sunspider-standalone-driver.js: Use the returned number of
66         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
67         ourselves, for slightly more accurate numbers.
68
69 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
70
71         Reviewed by Mark Rowe.
72         https://bugs.webkit.org/show_bug.cgi?id=45924
73
74         Modify the SunSpider harness to allow a path to be passed to the --suite
75         argument, supress the normal prepend-"tests/" behaviour for suite names
76         containing a '/'.
77
78         Also wrap the code in an anonymous function to move variables out of global
79         scope (currently the harness cannot handle running tests the overwrite the
80         variable 'j').
81
82         * resources/sunspider-standalone-driver.js:
83         * sunspider:
84
85 2010-08-27  Michael Saboff  <msaboff@apple.com>
86
87         Reviewed by Stephanie Lewis.
88
89         Added version 5 and version 6 of V8 tests with modifications similar
90         to the version 4 tests so they'll work with the SunSpider test harness.
91         Note that these files are external source and therefore don't conform 
92         to WebKit coding standards.
93         https://bugs.webkit.org/show_bug.cgi?id=44638
94
95         * tests/v8-v5: Added.
96         * tests/v8-v5/LIST: Added.
97         * tests/v8-v5/v8-crypto.js: Added.
98         * tests/v8-v5/v8-deltablue.js: Added.
99         * tests/v8-v5/v8-earley-boyer.js: Added.
100         * tests/v8-v5/v8-raytrace.js: Added.
101         * tests/v8-v5/v8-regexp.js: Added.
102         * tests/v8-v5/v8-richards.js: Added.
103         * tests/v8-v5/v8-splay.js: Added.
104         * tests/v8-v6: Added.
105         * tests/v8-v6/LIST: Added.
106         * tests/v8-v6/v8-crypto.js: Added.
107         * tests/v8-v6/v8-deltablue.js: Added.
108         * tests/v8-v6/v8-earley-boyer.js: Added.
109         * tests/v8-v6/v8-raytrace.js: Added.
110         * tests/v8-v6/v8-regexp.js: Added.
111         * tests/v8-v6/v8-richards.js: Added.
112         * tests/v8-v6/v8-splay.js: Added.
113
114 2010-07-15  Stephanie Lewis  <slewis@apple.com>
115
116         Reviewed by Geoff Garen.
117
118         http://bugs.webkit.org/show_bug.cgi?id=42406
119         Output location of results file.
120
121         * sunspider:
122
123 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
124
125         Reviewed by Mark Rowe.
126
127         Browser-hosted version of SunSpider runs tests twice as many times as advertised
128         https://bugs.webkit.org/show_bug.cgi?id=39201
129
130         * make-hosted: Make sure to reset test list after every test suite.
131
132 2010-05-07  Adam Roben  <aroben@apple.com>
133
134         Fix typo in sunspider-compare-results.js that caused incorrect
135         two-sample t statistics to be computed
136
137         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
138         two-sample t statistics
139
140         Reviewed by Darin Adler.
141
142         * resources/sunspider-compare-results.js:
143         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
144         variable to match the definition of a two-sample t statistic. Looks
145         like this was just a simple typo.
146
147 2010-04-05  Darin Adler  <darin@apple.com>
148
149         Updated the ignore property for changes to the contents here.
150
151         * .: Modified property svn:ignore.
152
153 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
154
155         Reviewed by Adam Barth.
156
157         Add sunspider-0.9.1 to the site in preparation for announcing it.
158         https://bugs.webkit.org/show_bug.cgi?id=35206
159
160         * hosted/sunspider.css: Remove tabs, which I noticed where present while
161         copying to the WebKitSite directory.
162
163 2010-01-11  Oliver Hunt  <oliver@apple.com>
164
165         Reviewed by Darin Adler.
166
167         Sunspider parse-only tests don't run
168         https://bugs.webkit.org/show_bug.cgi?id=33489
169
170         Fix parse-only test list
171
172         * tests/parse-only/LIST:
173
174 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
175
176         Reviewed by Adele Peterson.
177
178         Make SunSpider version more prominent in the title
179         https://bugs.webkit.org/show_bug.cgi?id=32574
180
181         * hosted/sunspider.html:
182         * hosted/versions.html:
183         * resources/driver-TEMPLATE.html:
184         * resources/results-TEMPLATE.html:
185
186 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
187
188         Reviewed by Darin Adler.
189
190         Make SunSpider refuse to compare results across content versions
191         https://bugs.webkit.org/show_bug.cgi?id=32573
192         
193         The results URL now includes a version indicator, and compare mode
194         will refuse to compare between versions.
195
196         * resources/driver-TEMPLATE.html:
197         * resources/results-TEMPLATE.html:
198
199 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
200
201         Reviewed by Darin Adler.
202
203         Some Browser-hosted SunSpider files are not valid HTML5
204         https://bugs.webkit.org/show_bug.cgi?id=32536
205         
206         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
207         close tags and adding a meta charset declaration).
208
209         * hosted/sunspider.html:
210         * hosted/versions.html:
211         * resources/TEMPLATE.html:
212         * resources/driver-TEMPLATE.html:
213         * resources/results-TEMPLATE.html:
214
215 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
216
217         Reviewed by Eric Seidel.
218
219         Make sunspider-0.9.1 the default content set (both command-line and hosted)
220         https://bugs.webkit.org/show_bug.cgi?id=32537
221
222         * hosted/sunspider.html: Change default suite link.
223         * sunspider: Change default suite.
224         * sunspider-compare-results: Likewise.
225
226 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
227
228         Reviewed by Sam Weinig.
229
230         in-browser SunSpider suffers excessive penalty under power management
231         https://bugs.webkit.org/show_bug.cgi?id=32505
232         
233         I have made a few changes to address this:
234         
235         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
236         document.write to reduce triggering of progress UI.
237         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
238         stepped all the way down the whole time.
239         3) Run the test cycle an extra time for warmup before the runs that count.
240
241         * hosted/sunspider-record-result.js: Removed.
242         * make-hosted:
243         * resources/TEMPLATE.html:
244         * resources/driver-TEMPLATE.html:
245
246 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
247
248         Reviewed by Gavin Barraclough.
249
250         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
251         https://bugs.webkit.org/show_bug.cgi?id=16806
252
253         Based on a patch by Eric Seidel.
254         
255         Fix the base64 computation to actually compute correct results. The impact on runtime of
256         the test is pretty small, but noticeable for some browsers. But at least it's not
257         doing a wrong and meaningless computation any more.
258         
259         * tests/sunspider-0.9.1/string-base64.js:
260         ():
261         (base64ToString):
262
263 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
264
265         Fixing commit error...
266         
267         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
268
269         * resources/driver-TEMPLATE.html:
270
271 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
272
273         Reviewed by Sam Weinig.
274
275         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
276         https://bugs.webkit.org/show_bug.cgi?id=32490
277
278         * hosted/json2.js: Added. Incorporated from json.org
279         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
280         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
281         results.
282
283 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
284
285         Reviewed by Oliver Hunt.
286
287         Enable Web-hosted version of SunSpider to handle multiple versions
288         https://bugs.webkit.org/show_bug.cgi?id=32478
289
290         * make-hosted: Now generate the hosted version to be able to run both the
291         0.9 and the 0.9.1 test suites.
292         * hosted: Modified property svn:ignore.
293         * hosted/sunspider-driver.html: Removed.
294         * hosted/sunspider-results.html: Removed.
295         * hosted/sunspider.html:
296         * hosted/versions.html: Added.
297         * resources/TEMPLATE.html:
298         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
299         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
300
301 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
302
303         Reviewed by Oliver Hunt.
304
305         Give command-line sunspider the ability to handle multiple suites and versions
306         https://bugs.webkit.org/show_bug.cgi?id=32477
307         
308         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
309         enumerating the tests. Current suites are:
310         
311         sunspider-0.9 - riginal SunSpider content 
312         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
313         ubench - "ubench" microbenchmark suite
314         v8-v4 - version 4 of the v8 benchmark
315         parse-only - some parser tests
316         
317         Sunspider can now be run with the --suite= parameter to select a
318         suite. The default is --suite=sunspider-0.9.
319
320         * .: Modified property svn:ignore to ignore new results directories.
321         * make-hosted: Generate based on sunspider-0.9 suite.
322         * resources/TEMPLATE.html: Fix an HTML compliance problem.
323         * resources/sunspider-standalone-driver.js: Become aware of suite names.
324         * sunspider: Modified to handle multiple suites as described above.
325         * sunspider-compare-results: ditto
326         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
327         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
328         * tests/LIST-UBENCH: Removed.
329         * tests/LIST-V8: Removed.
330         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
331         * tests/3d-morph.js: ditto
332         * tests/3d-raytrace.js: ditto
333         * tests/access-binary-trees.js: ditto
334         * tests/access-fannkuch.js: ditto
335         * tests/access-nbody.js: ditto
336         * tests/access-nsieve.js: ditto
337         * tests/bitops-3bit-bits-in-byte.js: ditto
338         * tests/bitops-bits-in-byte.js: ditto
339         * tests/bitops-bitwise-and.js: ditto
340         * tests/bitops-nsieve-bits.js: ditto
341         * tests/controlflow-recursive.js: ditto
342         * tests/crypto-aes.js: ditto
343         * tests/crypto-md5.js: ditto
344         * tests/crypto-sha1.js: ditto
345         * tests/date-format-tofte.js: ditto
346         * tests/date-format-xparb.js: ditto
347         * tests/math-cordic.js: ditto
348         * tests/math-partial-sums.js: ditto
349         * tests/math-spectral-norm.js: ditto
350         * tests/regexp-dna.js: ditto
351         * tests/string-base64.js: ditto
352         * tests/string-fasta.js: ditto
353         * tests/string-tagcloud.js: ditto
354         * tests/string-unpack-code.js: ditto
355         * tests/string-validate-input.js: ditto
356         * tests/sunspider-0.9/: Added
357         * tests/sunspider-0.9/*.js: Moved from one level up
358         * tests/sunspider-0.9.1: Added.
359         * tests/sunspider-0.9.1/*.js: Moved from one level up.
360         * tests/v8-crypto.js: Moved to v8-v4/ directory.
361         * tests/v8-deltablue.js: ditto
362         * tests/v8-earley-boyer.js: ditto
363         * tests/v8-raytrace.js: ditto
364         * tests/v8-regexp.js: ditto
365         * tests/v8-richards.js: ditto
366         * tests/v8-splay.js: ditto
367         * tests/v8-v4: Added.
368
369 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
370
371         Reviewed by Darin Adler.
372
373         Platform specific null device replaced with a platform independent.
374         https://bugs.webkit.org/show_bug.cgi?id=29544
375
376         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
377
378 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
379
380         Reviewed by Maciej Stachowiak.
381
382         [Qt] Make sunspider script work on Windows platform.
383         https://bugs.webkit.org/show_bug.cgi?id=29544
384
385         * sunspider:
386         Using platform specific null device instead of /dev/null.
387
388 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
389
390         Reviewed by Oliver Hunt.
391
392         https://bugs.webkit.org/show_bug.cgi?id=26540
393         Currently the SunSpider test driver lacks an option to run a test suite that
394         will test JavaScriptCore parsing performance only.  This patch adds just such
395         a test suite and option to SunSpider as well as the jsc test shell.  I've included
396         three large javascript source files found in the wild: jquery, mootools and prototype.
397         Combined with the concatenation of all three, these form a new testsuite to measure
398         and test pure JavaScriptCore parsing performance.
399
400         * resources/sunspider-standalone-driver.js:
401           Load the js with parse-only flag if it is part of the parse-only suite.
402         * sunspider:
403           Add the parse-only suite to the list of options.
404         * sunspider-compare-results:
405           Add the parse-only suite to the list of options.
406         * tests/LIST-PARSE-ONLY: Added.
407         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
408           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
409         * tests/parse-only/jquery-1.3.2.js: Added.
410           Latest version from jquery.com.
411         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
412           Latest version from mootools.net.
413         * tests/parse-only/prototype-1.6.0.3.js: Added.
414           Latest version from prototypejs.org.
415
416 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
417
418         Update V8 benchmark to version 4.
419         
420         Reviewed by Cameron Zwarich.
421
422         * tests/LIST-V8:
423         * tests/v8-crypto.js:
424         * tests/v8-deltablue.js:
425         * tests/v8-earley-boyer.js:
426         * tests/v8-raytrace.js:
427         * tests/v8-regexp.js: Added.
428         * tests/v8-richards.js:
429         * tests/v8-splay.js: Added.
430
431 2009-05-26  Olivier DOLE  <odole@pleyo.com>
432
433         Reviewed by Maciej.
434
435         Store sunspider test suite results in a specific directory for each
436         different type of benchmark (sunspider, V8 or ubench).
437
438         * sunspider:
439         * sunspider-compare-results:
440
441 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
442
443         Build fix.
444
445         * SunSpider.make:
446
447 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
448
449         Include SunSpider tests in production builds.
450         
451         Reviewed by Adam Roben.
452
453         * SunSpider.make: Added.
454         * xcopy.excludes: Added.
455
456 2009-03-04  Oliver Hunt  <oliver@apple.com>
457
458         Added a few more tests to ubench
459
460         RS=Gavin Barraclough
461
462         Adding a couple of test to cover too few and too many arguments.  This
463         is needed to prevent regressions in all call paths from my current refactoring.
464
465         * tests/LIST-UBENCH:
466         * tests/ubench/function-correct-args.js: Added.
467         (f):
468         * tests/ubench/function-excess-args.js: Added.
469         (f):
470
471 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
472
473         Fix a bug where when V8's or UBENCH's test suite list file is missing
474         the error message is suggesting that SunSpider's list file is missing.
475
476         Reviewed by Sam Weinig.
477
478         * sunspider:
479         (loadTestsList): Fix the error message to use the right list name.
480
481 2008-10-07  David Hyatt  <hyatt@apple.com>
482
483         https://bugs.webkit.org/show_bug.cgi?id=18989
484
485         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
486         we want all browsers to be doing the same thing on the test.
487
488         Reviewed by Maciej
489
490         * tests/regexp-dna.js:
491
492 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
493
494         Rubber-stamped by Maciej Stachowiak.
495
496         The bleeding edge branch of V8 now supports passing fileanmes after
497         '-f', so the SunSpider script should be changed to do this again.
498         This allows all 3 major JavaScript engines to be used with the
499         unmodified SunSpider script.
500
501         * sunspider:
502
503 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
504
505         Reviewed by Maciej Stachowiak.
506
507         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
508         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
509         correct the punctuation in its documentation.
510
511         * sunspider:
512
513 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
514
515         Reviewed by Oliver Hunt.
516
517         Bug 20616: Incorporate V8 benchmarks in testing
518         <https://bugs.webkit.org/show_bug.cgi?id=20616>
519
520         This is a first attempt at running the V8 benchmarks within SunSpider,
521         with the --v8 option. The number of iterations is fixed for each
522         benchmark, and was chosen to make each of the benchmarks run between
523         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
524
525         * sunspider:
526         * tests/LIST-V8: Added.
527         * tests/v8-crypto.js: Added.
528         * tests/v8-deltablue.js: Added.
529         * tests/v8-earley-boyer.js: Added.
530         * tests/v8-raytrace.js: Added.
531         * tests/v8-richards.js: Added.
532
533 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
534
535         Not reviewed.
536
537         Remove a local change that was accidentally committed in r36034.
538
539         * resources/sunspider-standalone-driver.js:
540
541 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
542
543         Rubber-stamped by Oliver Hunt.
544
545         Make SunSpider work with JavaScript shells that don't take filenames
546         after a '-f' argument.
547
548         * sunspider:
549
550 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
551
552         Reviewed by Timothy.
553
554         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
555
556         * sunspider:
557         * tests/LIST-SQUIRRELFISH: Removed.
558
559 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
560
561         Rubber-stamped by Mark Rowe.
562
563         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
564
565         * sunspider:
566
567 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
568
569         Rubber stamped by Oliver.
570         
571         - add newly working tests to squirrelfish list
572
573         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
574
575 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
576
577         Rubber stamped by Oliver.
578         
579         - move ubench tests properly.
580
581         * tests/ubench: Added.
582         * tests/ubench/function-closure.js: Added.
583         * tests/ubench/function-empty.js: Added.
584         * tests/ubench/function-missing-args.js: Added.
585         * tests/ubench/function-sum.js: Added.
586         * tests/ubench/loop-empty-resolve.js: Added.
587         * tests/ubench/loop-empty.js: Added.
588         * tests/ubench/loop-sum.js: Added.
589
590 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
591
592         Rubber stamped by Oliver.
593
594         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
595
596 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
597
598         Reviewed by Oliver.
599
600         - make --squirrelfish mode use the real harness, but a reduced set of tests
601
602         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
603
604         * sunspider:
605         * tests/LIST-SQUIRRELFISH:
606         * tests/LIST-UBENCH: Added.
607         * tests/squirrelfish: Moved to tests/ubench
608
609 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
610
611         Reviewed by Maciej Stachowiak.
612         
613         Removed an unused test.
614
615         * tests/squirrelfish/loop-resolve.js: Removed.
616
617 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
618
619         Reviewed by Oliver.
620         
621         Add newly runnable tests to --squirrelfish mode.
622
623         * tests/LIST-SQUIRRELFISH:
624
625 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
626
627         Reviewed by Maciej Stachowiak.
628         
629         A little more love for --squirrelfish mode:
630         
631         Fixed a misplaced sort. Results properly sort now.
632         
633         Added a test list just for squirrelfish, and updated it to include
634         all currently passing SunSpider tests.
635         
636         Fixed the pruning regexp to match 3d-morph.js.
637
638         * sunspider:
639
640 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
641
642         Reviewed by Oliver Hunt.
643         
644         --squirrelfish mode: pared down tests for squirrelfish to chew on.
645
646         * sunspider:
647         * tests/LIST:
648         * tests/squirrelfish: Added.
649         * tests/squirrelfish/function-closure.js: Added.
650         * tests/squirrelfish/function-empty.js: Added.
651         * tests/squirrelfish/function-missing-args.js: Added.
652         * tests/squirrelfish/function-sum.js: Added.
653         * tests/squirrelfish/loop-empty-resolve.js: Added.
654         * tests/squirrelfish/loop-empty.js: Added.
655         * tests/squirrelfish/loop-resolve.js: Added.
656         * tests/squirrelfish/loop-sum.js: Added.
657
658 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
659
660         Reviewed by Geoff.
661
662         * sunspider: Pause update daemon on Darwin to make results reliable
663         again (workaround for <rdar://problem/5811127>).
664
665 2008-02-07  Eric Seidel  <eric@webkit.org>
666
667         Reviewed by Sam.
668
669         * sunspider: add an error message when ./tests/LIST is not found
670
671 2007-12-29  Alp Toker  <alp@atoker.com>
672
673         Fix typo.
674
675         * hosted/sunspider.html:
676
677 2007-12-29  Alp Toker  <alp@atoker.com>
678
679         Reviewed by Maciej.
680
681         Provide the current URL in the SunSpider results page. Useful for
682         copying benchmark results out of browsers without a location bar or
683         where the location bar can't handle long URLs.
684
685         * hosted/sunspider-results.html:
686
687 2007-12-20  Eric Seidel  <eric@webkit.org>
688
689         Reviewed by mjs.
690
691         * sunspider: increase default number of runs for more accurate numbers
692
693 2007-12-18  Mark Rowe  <mrowe@apple.com>
694
695         Fix typo.
696
697         * hosted/sunspider.html:
698
699 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
700
701         Reviewed by Eric.
702         
703         - give an extra digit of precision for small differences.
704
705         * resources/sunspider-compare-results.js:
706
707 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
708
709         Reviewed by Eric.
710         
711         - work around IE layout bugs
712
713         * hosted/sunspider.css:
714
715 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
716
717         Reviewed by Mark.
718         
719         - fix compare feature
720
721         * hosted/sunspider-results.html:
722
723 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
724
725         Reviewed by Mark.
726         
727         - Change the format of the output URL to be shorter, to meet IE constraints
728
729         * hosted/sunspider-driver.html:
730         * hosted/sunspider-results.html:
731
732 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
733
734         Reviewed by Oliver.
735         
736         - Add some explanatory test and a dumb logo to SunSpider web hosted version
737
738         * hosted/sunspider-driver.html:
739         * hosted/sunspider-results.html:
740         * hosted/sunspider.css:
741         * hosted/sunspider.html:
742
743 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
744
745         Reviewed by Oliver.
746         
747         - Make the web-hosted version of SunSpider look a little bit less drab.
748
749         * hosted/sunspider-driver.html:
750         * hosted/sunspider-results.html:
751         * hosted/sunspider.css: Added.
752         * hosted/sunspider.html:
753         * resources/TEMPLATE.html:
754
755 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
756
757         Reviewed by Sam.
758         
759         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
760         
761         A sample of what new-style results would look like:
762         
763         TEST                   COMPARISON            FROM                 TO             DETAILS
764         ...
765         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
766         ...
767             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
768         
769         * resources/sunspider-compare-results.js:
770
771 2007-11-14  Eric Seidel  <eric@webkit.org>
772
773         Reviewed by Sam.
774
775         * sunspider: add --shark-cache for L2 Cache Miss profiling
776
777 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
778
779         Reviewed by Darin Adler.
780
781         http://bugs.webkit.org/show_bug.cgi?id=15925
782         SunSpider should check for Shark being installed
783
784         * sunspider: Explicitly test that shark command line tool is installed.
785
786 2007-11-04  Darin Adler  <darin@apple.com>
787
788         * hosted/sunspider.html: Tweaked formatting a bit.
789
790 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
791
792         Reviewed by Darin Adler.
793
794         - Add compare mode and a description of the benchmark to the hosted version.
795
796         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
797         * hosted/sunspider.html: Added intro text.
798         * make-hosted: Copy new file.
799         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
800         a function.
801         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
802         expects.
803         * sunspider-compare-results: Use sunspider-standalone-compare.js.
804
805 2007-11-02  Darin Adler  <darin@apple.com>
806
807         * sunspider: Changed "--base" to "--set-baseline".
808
809 2007-11-02  Darin Adler  <darin@apple.com>
810
811         Reviewed by Maciej.
812
813         * sunspider: Added a "--base" option that records the current test as a baseline.
814         Moved the code to find the newest .mshark file into a function (since I needed to use
815         it in the other script and I prefer to copy/paste a function).
816
817         * sunspider-compare-results: Added logic to compare the baseline with the last
818         run when you don't pass any parameters.
819
820 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
821
822         Reviewed by Eric.
823         
824         - Added list of ECMAScript3 features that SunSpider does not cover currently.
825
826         * UNCOVERED: Added.
827
828 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
829
830         Reviewed by Oliver.
831         
832         - Give better error messages when sunspider-compare-results fails
833
834         * sunspider-compare-results:
835
836 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
837
838         Reviewed by Darin Adler.
839         
840         - fix Date tests to work in command-line SpiderMonkey
841
842         * tests/date-format-tofte.js:
843         * tests/date-format-xparb.js:
844
845 2007-10-29  Darin Adler  <darin@apple.com>
846
847         Reviewed by Maciej.
848
849         * sunspider: Added --shark20 option, to run Shark at its highest sample
850         resolution instead of its default.
851
852 2007-10-28  Adam Roben  <aroben@apple.com>
853
854         Fix SunSpider on Windows
855
856         Reviewed by Eric.
857
858         * sunspider: Don't use colons in filenames.
859
860 2007-10-28  Darin Adler  <darin@apple.com>
861
862         Reviewed by Adam.
863
864         * resources/sunspider-analyze-results.js: Tweak the output format a little.
865         Change so that when there's only one run we don't write out confidence
866         intervals at all rather than writing "NaN" over and over again.
867
868         * sunspider: Use the time and date as a suffix on the results file, that way
869         all the results are left behind in the tmp directory. This will make it easy
870         for us to add features that compare past results in the future. Also rename
871         the .mshark files using the same scheme.
872
873         * sunspider-compare-results: Relax the parsing rules so we can parse the
874         results file exactly as it's emitted from sunspider.
875
876 2007-10-25  Adam Roben  <aroben@apple.com>
877
878         Strip carriage returns from results
879
880         Carriage returns were screwing up sunspider-compare-results on
881         Windows.
882
883         Rubberstamped by Oliver.
884
885         * sunspider:
886
887 2007-10-24  Darin Adler  <darin@apple.com>
888
889         Reviewed by Eric.
890
891         * sunspider: Open the profile in shark after completing a run.
892         * sunspider-compare-results: Tiny perl tweaks.
893
894 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
895
896         Reviewed by Darin Adler.
897
898         - Don't hardcode my path to testkjs
899         
900         * sunspider-compare-results:
901
902 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
903
904         Reviewed by Darin Adler.
905         
906         - Add a compare script to compare results, and improve formatting of standard results a bit.
907
908         * resources/sunspider-analyze-results.js:
909         * resources/sunspider-compare-results.js: Added.
910         * sunspider-compare-results: Added.
911
912 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
913
914         Reviewed by Darin Adler.
915
916         * resources/sunspider-analyze-results.js: Do the error range
917         properly, using the t-distribution instead of 1.96 (which was
918         based on the normal distribution).
919         
920         * sunspider: Print results in a way that will be friendlier to the
921         soon-to-come compare mode.
922
923 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
924
925         Reviewed by Review.
926         
927         - add more tests (probably the final set for now)
928         
929         Already balanced. Added date, regexp, control flow, and a few
930         more string and object/array access tests.
931         
932         * tests/LIST:
933         * tests/access-fannkuch.js: Added.
934         * tests/access-nbody.js: Added.
935         * tests/controlflow-recursive.js: Added.
936         * tests/date-format-tofte.js: Added.
937         * tests/date-format-xparb.js: Added.
938         * tests/regexp-dna.js: Added.
939         * tests/string-validate-input.js: Added.
940
941 2007-10-21  Eric Seidel  <eric@webkit.org>
942
943         Reviewed by mjs.
944
945         * sunspider: make --shark only sample from testkjs
946
947 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
948
949         Rubber stamped by Eric.
950         
951         - added crypto tests (already balanced)
952
953         * tests/LIST:
954         * tests/crypto-aes.js: Added.
955         * tests/crypto-md5.js: Added.
956         * tests/crypto-sha1.js: Added.
957
958 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
959
960         Reviewed by Adam.
961         
962         - rebalance new tests
963         
964         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
965         
966         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
967
968         This balances them with the existing tests.
969
970         * tests/access-binary-trees.js:
971         * tests/access-nsieve.js:
972         * tests/bitops-nsieve-bits.js:
973         * tests/math-partial-sums.js:
974         * tests/math-spectral-norm.js:
975         * tests/string-fasta.js:
976
977 2007-10-20  Eric Seidel  <eric@webkit.org>
978
979         Reviewed by darin.
980         
981         Add improved argument handling to sunspider, including
982         --runs=<number>, --tests=<pattern, --shark, and --help
983         Also re-factor code into subroutines
984
985         * sunspider: improved argument handling
986
987 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
988
989         Reviewed by Mark.
990
991         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
992         
993         * TODO:
994         * tests/LIST:
995         * tests/access-binary-trees.js: Added.
996         * tests/access-nsieve.js: Added.
997         * tests/bitops-nsieve-bits.js: Added.
998         * tests/math-partial-sums.js: Added.
999         * tests/math-spectral-norm.js: Added.
1000         * tests/string-fasta.js: Added.
1001
1002 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1003
1004         Reviewed by Darin Adler.
1005
1006         * TODO: Updated to note areas that are now well-covered.
1007         
1008         - Change the tests and the driver to avoid leaving large object
1009         graphs hanging around, since that throws off the subsequent tests.
1010         
1011         * resources/sunspider-standalone-driver.js:
1012         * tests/3d-cube.js:
1013         * tests/3d-morph.js:
1014         * tests/3d-raytrace.js:
1015         * tests/bitops-bitwise-and.js:
1016         * tests/string-base64.js:
1017         * tests/string-tagcloud.js:
1018
1019 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1020
1021         Reviewed by Nikolas.
1022         
1023         - Rebalance test complexity.
1024         
1025         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1026         
1027         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1028
1029         However, for some tests, IE7 was a huge outlier compared to other
1030         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1031         same OS and hardware as a normalization basis.
1032         
1033         * tests/3d-morph.js:
1034         * tests/3d-raytrace.js:
1035         * tests/bitops-3bit-bits-in-byte.js:
1036         * tests/bitops-bits-in-byte.js:
1037         * tests/bitops-bitwise-and.js:
1038         * tests/math-cordic.js:
1039         * tests/string-base64.js:
1040         * tests/string-tagcloud.js:
1041         * tests/string-unpack-code.js:
1042
1043 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1044
1045         Reviewed by Oliver.
1046         
1047         - Fix the browser-hosted driver to be compatible with Safari 2
1048
1049         * hosted/sunspider-driver.html:
1050
1051 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1052
1053         Rubber stamped by Hyatt.
1054         
1055         - Add BSD license notice to cordic test, the original author said to do so.
1056
1057         * tests/math-cordic.js:
1058
1059 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1060
1061         Reviewed by Eric.
1062         
1063         - Support Windows and Windows browsers.
1064
1065         * hosted/sunspider-results.html: Display results correctly in IE.
1066         * sunspider: tweaks to work on Windows, and with spaces in the path.
1067
1068 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1069
1070         Rubber stamped by Oliver.
1071
1072         * tests/string-tagcloud.js: Remove stray print()
1073
1074 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1075
1076         Reviewed by Oliver.
1077         
1078         Add some string test cases.
1079
1080         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1081         * tests/string-tagcloud.js: Added. Parses a JSON string and
1082         generates tagcloud HTML markup based on the data.        
1083         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1084         four of the most popular JavaScript libraries.
1085         * tests/LIST: Added the new tests.
1086
1087 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1088
1089         Reviewed by Anders.
1090
1091         * TODO: Updated with info on how to do the stats properly.
1092
1093         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1094         avoid overly slowing down the benchmark.
1095
1096         Added some 3D-related test cases.
1097         
1098         * tests/3d-cube.js: Added.
1099         * tests/3d-morph.js: Added.
1100         * tests/3d-raytrace.js: Added.
1101         * tests/LIST:
1102
1103 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1104
1105         Reviewed by Darin Adler.
1106
1107         - New JavaScript benchmark
1108         http://bugs.webkit.org/show_bug.cgi?id=15515
1109         
1110         A start on a new JavaScript benchmark, with standalone and
1111         browser-hosted drivers.
1112         
1113         The test content is still incomplete and the drivers could use
1114         some more features but there is enough here to make a good start.
1115         
1116         The drivers compute a 95% confidence interval on the mean for the
1117         whole test, each category, and each individual test to make it
1118         easier to tell whether differences are statistically
1119         significant. The confidence interval can be narrowed by running
1120         the test more times.
1121         
1122         * TODO: Added. Remaining things that need to be done.
1123         * sunspider: Added. Perl script that acts as the standalone test driver.
1124         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1125         of standalone test driver.
1126         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1127         analysis code.
1128         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1129         * make-hosted: Added. Script to generate browser-hosted tests.
1130         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1131         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1132         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1133         tests.
1134         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1135         tests.
1136         * tests/LIST: Added. List of tests to use.
1137         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1138         * tests/bitops-bits-in-byte.js: Added. Ditto.
1139         * tests/bitops-bitwise-and.js: Added. Ditto.
1140         * tests/math-cordic.js: Added. Ditto.
1141